What use is the INVD instruction?

Question

The x86 INVD invalidates the cache hierarchy without writing the contents back to memory, apparently.

I\'m curious, what use is such an instruction? Given h

Answer 1

Excellent question!

One use-case for such a blunt-acting instruction as invd is in specialized or very-early-bootstrap code, such as when the presence or absence of RAM has not yet been verified. Since we might not know whether RAM is present, its size, or even if particular parts of it function properly, or we might not want to rely on it, it's sometimes useful for the CPU to program part of its own cache to operate as RAM and use it as such. This is called Cache-as-RAM (CAR). During setup of CAR, while using CAR, and during teardown of CAR mode, the kernel must ensure nothing is ever written out from that cache to memory.

Cache-as-RAM

Entering CAR

To set up CAR, the CPU must be set to No-Fill Cache Mode and must designate the memory range to be used for CAR as Write-Back. This can be done by the following steps:

1. Set up an MTRR (Memory Type Range Register) to designate a chunk of memory as WB (Write-Back).
2. invd the entire cache, preventing any cached write from being written out and causing chaos.
3. Set caching mode to Normal Cache Mode (cr0.CD=0).
4. While in Normal Cache Mode, "touch" all cachelines of the memory span to be used as CAR by reading it, and thus filling the cache with it. Filling cachelines can only be done in Normal Cache Mode.
5. Set caching mode to No-Fill Cache Mode (cr0.CD=1).

Using CAR

The motivation for setting up CAR is that once set up, all accesses (read/write) within the CAR region will hit cache and will not hit RAM, yet the cache's contents will be addressable and act just like RAM. Therefore, instead of writing assembler code that only ever uses registers, one can now use normal C code, provided that the stack and local/global variables it accesses are restricted to within the CAR region.

Exiting CAR

When CAR is exited, it would be a bad thing for all of the memory writes incurred in this "pseudo-RAM" to suddenly shoot out from cache and trash any actual content at the same address in RAM. So when CAR is exited, once again invd is used to completely delete the contents of the CAR region, and then Normal Cache Mode is set up.

Intel 80486 Manual

Intel alluded to the Cache-as-RAM use in the i486 Microprocessor Programmer's Reference Manual. The Intel 80486 was the CPU that first introduced the invd instruction. Section 12.2 read:

12.2 OPERATION OF THE INTERNAL CACHE

Software controls the operating mode of the cache. Caching can be enabled (its state following reset initialization), caching can be disabled while valid cache lines exist (a mode in which the cache acts like a fast, internal RAM), or caching can be fully disabled.

Precautions must be followed when disabling the cache. Whenever CD is set to 1, the i486 processor will not read external memory if a copy is still in the cache. Whenever NW is set to 1, the i486 processor will not write to external memory if the data is in the cache. This means stale data can develop in the i486 CPU cache. This stale data will not be written to external memory if NW is later set to 0 or that cache line is later overwritten as a result of a cache miss. In general, the cache should be flushed when disabled.

It is possible to freeze data in the cache by loading it using test registers while CD and NW are set. This is useful to provide guaranteed cache hits for time critical interrupt code and data.

Note that all segments should start on 16 byte boundaries to allow programs to align code/data in cache lines.

Examples of Usage

coreboot has a slide-deck presenting their implementation of CAR, which describes the above procedure. The invd instruction is used on Slide 21.

AMD calls it Cache-as-general-storage in §2.3.3: Using L2 Cache as General Storage During Boot.

Other Uses

In certain situations involving cache-incoherency due to DMA (Direct Memory Access) hardware, invd might also prove useful.