How does Rust achieve compile-time-only pointer safety?

Question

I have read somewhere that in a language that features pointers, it is not possible for the compiler to decide fully at compile time whether all pointers are used correctly and/

Answer 1

Disclaimer: I'm in a bit of a hurry, so this is a bit meandering. Feel free to clean it up.

The One Sneaky Trick That Language Designers Hate™ is basically this: Rust can only reason about the 'static lifetime (used for global variables and other whole-program lifetime things) and the lifetime of stack (i.e. local) variables: it cannot express or reason about the lifetime of heap allocations.

This means a few things. First of all, all of the library types that deal with heap allocations (i.e. Box, Rc, Arc) all own the thing they point to. As a result, they don't actually need lifetimes in order to exist.

Where you do need lifetimes is when you're accessing the contents of a smart pointer. For example:

let mut x: Box = box 0;
*x = 42;

What is happening behind the scenes on that second line is this:

{
    let box_ref: &mut Box = &mut x;
    let heap_ref: &mut i32 = box_ref.deref_mut();
    *heap_ref = 42;
}

In other words, because Box isn't magic, we have to tell the compiler how to turn it into a regular, run of the mill borrowed pointer. This is what the Deref and DerefMut traits are for. This raises the question: what, exactly, is the lifetime of heap_ref?

The answer to this is in the definition of DerefMut (from memory because I'm in a hurry):

trait DerefMut {
    type Target;
    fn deref_mut<'a>(&'a mut self) -> &'a mut Target;
}

Like I said before, Rust absolutely cannot talk about "heap lifetimes". Instead, it has to tie the lifetime of the heap-allocated i32 to the only other lifetime it has on hand: the lifetime of the Box.

What this means is that "complicated" things don't have an expressible lifetime, and thus have to own the thing they manage. When you convert a complicated smart pointer/handle into a simple borrowed pointer, that is the moment that you have to introduce a lifetime, and you usually just use the lifetime of the handle itself.

Actually, I should clarify: by "lifetime of the handle", I really mean "the lifetime of the variable in which the handle is currently being stored": lifetimes are really for storage, not for values. This is typically why newcomers to Rust get tripped up when they can't work out why they can't do something like:

fn thingy<'a>() -> (Box, &'a i32) {
    let x = box 1701;
    (x, &x)
}

"But... I know that the box will continue to live on, why does the compiler say it doesn't?!" Because Rust can't reason about heap lifetimes and must resort to tying the lifetime of &x to the variable x, not the heap allocation it happens to point to.