The Unsafe Chronicles: Exhibit A: Aliasing Boxes

What I'm getting from this is that Unsafe rust is "basically just C" except the rest of rust isn't. The rust compiler basically assumes that unsafe{/*your code*/} contains pointer-casting magic that magically complies with all of Rust's safety protocols. Except is not entirely clear what the safety protocols actually are.

This is super interesting, please give us more :D

This is really helpful. Thank you for making these.

Why don't you use an IDE? If you did, you'd be able to set a break point and show us the complete state of the program, including the types and values of each of your variables. I think that would go a long way in helping people understand.

I guess I'm silly for thinking Rc<T> would've been appropriate for a structure that would only ever have two aliases haha

I'm not even a rust developer but i really enjoy your videos, and every time after the video i feel like i should try rust :D

I think this highlights a nice convention of the current Rust community: Being considered 'safe' requires that it is sound _whenever a user can compile the code_, regardless of how "stupid" the user's code is. If the user can break safety by doing something that is "obviously" really stupid it is considered unsound. High integrity of the API surface. Libraries that "just work" even for idiots like me.

Thanks for this new series and the usual high quality content. I’d love to see more unsafe topics in the future :)

Really useful, would watch again !

So how did you find out about this subtle requirement of Box not being aliased if it's undocumented? From previous experience? hinted by viewers of the prev. stream?

I don't even write a lot of Rust nowadays but this is super interesting and thought-provoking for me.

Interesting video :) From a C++ user's perspective, PhantomData seems like a very contrived way of doing things. Also, this is getting dangerously close to template metaprogramming ;)

Hi Jon, great video as always. I was wondering, would you have any interest in making a video about pointers in Rust? Specifically, I'd love to learn about the differences between *const T, *mut T, Unique<T>, and NonNull<T>, especially with regards to variance and safety. Because, while I understand what variance is, I don't understand what it actually means. Thanks!

As always, great content!

Hey Jon, can you make a video about variance in combination with unsafe code?

Very interesting! I would like to see more

Your explanation of MaybeUninit doesn't sound accurate to me. MaybeUninit isn't just about validity but also about initialization.

For example MaybeUninit<u8>::uninit(). assume_init() is UB despite u8 allowing any bit pattern. The reason is u8 wasn't written to. You can call assume_init () only if you've written the value AND the value has correct bit pattern.

The name of the struct is actually good. 🙂

The thumbnail matches the content.... crazy stuff. I was watching the stream live and programming at the same time, and you caught me in the act of doing a mem::transmute when you said "don't do it" rofl. It's a toy project, so the transmute is still there :)

(toy project is a python interpreter, the transmute is to willfully ignore mutability rules and let things go wild, because that's how python likes it... it is actually cleaner than adding a ton of refcells and borrows. It is never going to production though.)

Great content! So, MaybeUninit It's sort of C++ volatile?

Your channel is actually the best "How Rust works (under the hood)" Information source I could find. Keep it up. :-)

Jon, since rust uses llvm currently, I was curious if it's hindering it in taking advantage of a lot of rust specific benefits. One thing I was thinking about yesterday is whether or not there could be significant improvements in cpu cache invalidation if all "non mut" cache data is stored together (same cache line). I just got around to watching Scott Meyers 2014 video on cache invalidation the other day and got me thinking about rust and that perhaps this could be a major benefit. I doubt llvm currently supports such design as other languages don't restrict in same way (well, not that I understand anyways). I wonder if they've figured out a way to do this with llvm (frankly, I know very little about details of llvm, only what it does).

Just wanted to throw out a suggestion for a future video. This is in particular related to those like me that are more used to GC languages. Use cases for the functionality in "std::mem". You know, devs with non-native background like me just don't tend to do these direct mem operations in our code (if even allowed in language of choice), so when I see all these mem copy and mem replace operations, I often don't quite understand what's the benefit versus setting variable values. Is there a step skipped (like a more atomic operation and faster?) I just think there's a lot in that module that many jumping ship to Rust from GC languages get tripped up on, and I think there are more of us out there than you think. Certainly not suggesting that you make such a video just for my request - like for me, lol. But if you are looking for appealing topics, I think this is one.

Awesome stuff!

This type of video helps stuff soak into my brain. Thanks, Jon! I look forward to more Unsafe Chronicles.

This was extremely informative, thank you. Towards the end some of this stuff feels started to feel a lot like security with the crate user feeling more like a hacker of some kind, determined to violate your invariants through any means.

I'm pretty sure transmute between Wrapper<T> and Wrapper<U> is always unsound if T != U and Wrapper<_> is #[repr(Rust)] regardless if it's safe to transmute between T and U with the exception of ZSTs.

or to put it simply:

#[repr(transparent)]
struct A { a: bool }

#[repr(transparent)]
struct B { b: u8 }

// implicit #[repr(Rust)]
struct Wrapper<T>(T);

// implicit #[repr(Rust)]
struct RefWrapper<'a, T>(&'a T);

unsafe {
transmute::<bool, u8>(..); // sound
transmute::<A, B>(..); // sound
transmute::<Wrapper<A>, Wrapper<B>>(..); // DANGER
transmute::<RefWrapper<'static, A>, RefWrapper<'static, B>>(..); // DANGER
}

even for

struct Foo<T> {
_0: PhantomData<T>,
vl: u8,
}

unsafe {
transmute::<Foo<bool>, Foo<u8>>(..); // unsound
}

struct C;
struct D;

unsafe {
transmute::<C, D>(..) // sound but may lead to unsafety if `D` has invariants to uphold.
}

The private types sound like a specialized version of encoding state in the type system, which allows for some neat API design with compile time sanity checks.

When you have a struct Something<State> you can then impl Something<SpecificState> and only add functionality applicable to that specific state, meaning that the compiler will alert you when you try to call a function that's not applicable, not only making the developer's life easier but also removing the necessity for sanity checks at runtime.

I'm a bit out of my depth here, but maybe there's reason to combine these things. Stateful API design could probably benefit (slightly) from a guarantee that you can safely cast from one state to the other, maybe there's even reason for an explicit language feature, since you could safely do a state change in place.

impl Something<SomeState> {
fn changes_state(self) -> Something<AnotherState> { ... } // would be kind of nice if this could be Self<AnotherState> and happened in place, which shouldn't be a problem if both states are the same (zero) size
}

Learned a lot from this! Finally somebody who explains things in detail!

As a relative newcomer to Rust, I’ve been working through a backlog of your videos in my spare time, and I think this might have been one of the most interesting so far!

Many of these points brought up here make a lot of sense just looking at it from a memory layout perspective.

But in particular those ‘explain it like I’m 5’ descriptions of the casting syntax really helped me… I might have to rewatch those chunks a couple more times after this 😂.

Is there a special reason to make a raw pointer using Box::into_raw instead of using std::alloc::alloc?

Seems like a classic engineering trade-off at play: the cost for having “safe” code is that writing anything under the hood requires a whole bunch of concerns and rules to always play properly with the “safe” world, which ends up being more complex than just writing C, where only specific rules need to be met in any situation. Thus, I found this video interesting but it certainly has not made writing unsafe code feel any more comfortable to me. I’d just rather not.

At the end you say to avoid writing unsafe code and use something someone else has written instead. So how do we know that someone else’s implementation is actually safe? How do we know they did a better job than what we could do? How do we know they even understood the concerns you allude to in this video? After all, nobody needs a license to write unsafe code and publish a crate containing it.

I feel like deferring this expertise to “other people” is a cop-out and will eventually result in a few elites and the rest of us just blindly trusting them to get it right. No, we should be capable of auditing all of this unsafe code, and to do that we need properly and completely specified rules, and tools to help enforce them. Unit tests that happen to pass are not enough!

Why using manually drop? MaybeUninit says you are responsible for dropping the value, yourself, so, it already is manually drop.