Minimizing page faults (and TLB faults) while “walking” a large graph

Question

Problem (think of the mark phase of a GC)

• I have a graph of “objects” that I need to walk, visiting all objects.
• I can store in each object if it has

Answer 1

Page faults aren't necessarily bad, as long as they're not stalling your progress.

This means that if you have a node Node* p with two candidate successors p->left and p->right, it can be useful to pick the nearest (in terms of (char*)p - (char*)p->next) and pre-fetch the other (e.g. with PrefetchVirtualMemory).

How efficient this will be cannot be predicted; it greatly depends on your graph topology. But the prefetch is virtually free when you have enough RAM.

Closer to the CPU, there's cache prefetching. Same idea, different storage