How does Rust achieve compile-time-only pointer safety?

Question

I have read somewhere that in a language that features pointers, it is not possible for the compiler to decide fully at compile time whether all pointers are used correctly and/

Answer 1

Is it the case that pointer checking isn't done entirely at compile-time, and Rust's smart pointers still introduce some runtime overhead compared to, say, raw pointers in C?

There are special runtime-checks for things that can't be checked at compile time. These are usually found in the cell crate. But in general, Rust checks everything at compile time and should produce the same code as you would in C (if your C-code isn't doing undefined stuff).

Or is it possible that the Rust compiler can't make fully correct decisions, and it sometimes needs to Just Trust The Programmer™, probably using one of the lifetime annotations (the ones with the <'lifetime_ident> syntax)? In this case, does this mean that the pointer/memory safety guarantee is not 100%, and still relies on the programmer writing correct code?

If the compiler cannot make the correct decision you get a compile time error telling you that the compiler cannot verify what you are doing. This might also restrict you from stuff you know is correct, but the compiler doesn't. You can always go to unsafe code in that case. But as you correctly assumed, then the compiler relies partly on the programmer.

The compiler checks the function's implementation, to see if it does exactly what the lifetimes say it does. Then, at the call-site of the function, it checks if the programmer uses the function correctly. This is similar to type-checking. A C++ compiler checks if you are returning an object of the correct type. Then it checks at the call-site if the returned object is stored in a variable of the correct type. At no time can the programmer of a function break the promise (except if unsafe is used, but you can always let the compiler enforce that no unsafe is used in your project)

Rust is continuously improved. More things may get legal in Rust once the compiler becomes smarter.

Another possibility is that Rust pointers are non-"universal" or restricted in some sense, so that the compiler can infer their properties entirely during compile-time, but they are not as useful as e. g. raw pointers in C or smart pointers in C++.

There's a few things that can go wrong in C:

1. dangling pointers
2. double free
3. null pointers
4. wild pointers

These don't happen in safe Rust.

1. You can never have a pointer that points to an object no longer on the stack or heap. That's proven at compile time through lifetimes.
2. You do not have manual memory management in Rust. Use a Box to allocate your objects (similar but not equal to a unique_ptr in C++)
3. Again, no manual memory management. Boxes automatically free memory.
4. In safe Rust you can create a pointer to any location, but you cannot dereference it. Any reference you create always is bound to an object.

There's a few things that can go wrong in C++:

1. everything that can go wrong in C
2. SmartPointers only help you not forget calling free. You can still create dangling references: auto x = make_unique(42); auto& y = *x; x.reset(); y = 99;

Rust fixes those:

1. see above
2. as long as y exists, you may not modify x. This is checked at compile time and cannot be circumvented by more levels of indirection or structs.

I have read somewhere that in a language that features pointers, it is not possible for the compiler to decide fully at compile time whether all pointers are used correctly and/or are valid (refer to an alive object) for various reasons, since that would essentially constitute solving the halting problem.

Rust doesn't prove you all pointers are used correctly. You may still write bogus programs. Rust proves that you are not using invalid pointers. Rust proves that you never have null-pointers. Rust proves that you never have two pointers to the same object, execept if all these pointers are non-mutable (const). Rust does not allow you to write any program (since that would include programs that violate memory safety). Right now Rust still prevents you from writing some useful programs, but there are plans to allow more (legal) programs to be written in safe Rust.

That is not surprising, intuitively, because in this case, we would be able to infer the runtime behavior of a program during compile-time, similarly to what's stated in this related question.

Revisiting the example in your referenced question about the halting problem:

void foo() {
    if (bar() == 0) this->a = 1;
}

The above C++ code would look one of two ways in Rust:

fn foo(&mut self) {
    if self.bar() == 0 {
        self.a = 1;
    }
}

fn foo(&mut self) {
    if bar() == 0 {
        self.a = 1;
    }
}

For an arbitrary bar you cannot prove this, because it might access global state. Rust soon gets const functions, which can be used to compute stuff at compile-time (similar to constexpr). If bar is const, it becomes trivial to prove if self.a is set to 1 at compile-time. Other than that, without pure functions or other restrictions of the function content, you can never prove whether self.a is set to 1 or not.

Rust currently doesn't care whether your code is called or not. It cares whether the memory of self.a still exists during the assignment. self.bar() can never destroy self (except in unsafe code). Therefor self.a will always be available inside the if branch.