Is there an `x86` instruction to tell which core the instruction is being run on?

Question

When I cat /proc/cpuinfo, I see 8 cores, with ID\'s from 0 to 7.

Is there an x86 instruction that will report th

Answer 1

Some newer x86/x86_64 CPUs have the "RDTSCP" variant of RDTSC instruction:

http://ref.x86asm.net/coder32-abc.html#R

RDTSC   EAX EDX IA32_TIM…               0F  31
        P1+         f2              Read Time-Stamp Counter
RDTSCP  EAX EDX ECX ...         0F  01  F9  7   
        C7+         f2              Read Time-Stamp Counter and Processor ID

C7+ means that "0x0F01F9" instruction was introduced in some "Core i7"...

Opcodes

Hex Mnemonic Encoding Long Mode Legacy Mode Description

0F 01 F9 RDTSCP A Valid Valid

Read 64-bit time-stamp counter and 32-bit IA32_TSC_AUX value into EDX:EAX and ECX.

OS should write core id into IA32_TSC_AUX (Linux does), and this value is accessible with RDTSCP.

Linux encodes numa id (<<12) and core id (8bit) into TSC_AUX:

341         if (cpu_has(&cpu_data(cpu), X86_FEATURE_RDTSCP))
342                 write_rdtscp_aux((node << 12) | cpu);
343 
344         /*
345          * Store cpu number in limit so that it can be loaded quickly
346          * in user space in vgetcpu. (12 bits for the CPU and 8 bits for the node)
347          */

In Linux there is also vsyscall getcpu ("__vdso_getcpu") to access cpu id via rdtscp (if cpu has the instruction) or via GDT - GDT_ENTRY_PER_CPU: __getcpu in include/asm/vsyscall.h from 3.13. From the man page:

getcpu() was added in kernel 2.6.19 for x86_64 and i386.

Linux makes a best effort to make this call as fast possible. The intention of getcpu() is to allow programs to make optimizations with per-CPU data or for NUMA optimization.

From some intel manuals: http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-32-ia-64-benchmark-code-execution-paper.pdf#page=15

3.2 Improvements Using RDTSCP Instruction

The RDTSCP instruction is described in the Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2B ([3]) as an assembly instruction that, at the same time, reads the timestamp register and the CPU identifier. The value of the timestamp register is stored into the EDX and EAX registers; the value of the CPU id is stored into the ECX register (“On processors that support the Intel 64 architecture, the high order 32 bits of each of RAX, RDX, and RCX are cleared”). What is interesting in this case is the “pseudo” serializing property of RDTSCP. The manual states:

“The RDTSCP instruction waits until all previous instructions have been executed before reading the counter. However, subsequent instructions may begin execution before the read operation is performed.”

This means that this instruction guarantees that everything that is above its call in the source code is executed before the instruction itself is called. It cannot, however, guarantee that - for optimization purposes - the CPU will not execute, before the RDTSCP call, instructions that, in the source code, are placed after the RDTSCP function call itself. If this happens, a contamination caused by instructions in the source code that come after the RDTSCP will occur in the code under measurement. .

Also, description is available here http://www.felixcloutier.com/x86/RDTSCP.html which is clone of https://github.com/zneak/x86doc

UPDATE: There will be separate instruction RDPID just to read IA32_TSC_AUX register without timestamp counter (as RDTSCP does

https://hjlebbink.github.io/x86doc/html/RDPID.html

Reads the value of the IA32_TSC_AUX MSR (address C0000103H) into the destination register. The value of CS.D and operand-size prefixes (66H and REX.W) do not affect the behavior of the RDPID instruction.

F3 0F C7 /7 RDPID r32 M   N.E./V  RDPID   Read IA32_TSC_AUX into r32.
F3 0F C7 /7 RDPID r64 M   V/N.E.  RDPID   Read IA32_TSC_AUX into r64.

It will be enabled since "Ice Lake" microarchitecture (2018), as declared in https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf 319433-030 OCTOBER 2017

Answer 2

You see 8 “virtual CPUs”, rather than cores, so if you have, say a 4-core Ivy Bridge CPU with 2 hardware threads per core, you can see which pairs of vCPUs share a core via the entries in /sys/devices/system/cpu/cpu[0-7]/topology/thread_siblings_list.

Another answer makes the fine suggestion of using cpuid, but I don't think you can be certain that the cpuid instruction executes on the same vCPU as the instruction-of-interest, unless you pin the thread to a vCPU (in which case it's rather superfluous), since you cannot know that the kernel hasn't migrated your thread from one vCPU to another between the time that the instruction-of-interest was executed and the time the cpuid instruction was executed.

In short, the vast majority of the time, two “close” instructions will execute on the same vCPU, but it isn't guaranteed without pinning, and if you have pinned the thread, you already know what vCPU it's running on, so it's somewhat pointless.

Answer 3

The Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 3A: System Programming Guide, Part 1, section 8.4.5 Identifying Logical Processors in an MP System lists, among others:

This APIC ID is reported by CPUID.0BH:EDX[31:0]

Note that this doesn't directly equate to the linux kernel's numbering. In the kernel there is an x86_cpu_to_apicid table that you can read. Of course the kernel also knows what cpu the code is executing on, without having to consult the APIC:

 * smp_processor_id(): get the current CPU ID.
 *
 * if DEBUG_PREEMPT is enabled then we check whether it is
 * used in a preemption-safe way. (smp_processor_id() is safe
 * if it's used in a preemption-off critical section, or in
 * a thread that is bound to the current CPU.)

Answer 4

Except already described CPUID and RDTSCP instructions also there is new one RDPID instruction (Intel SDM download page) exactly for this purpose.

Description

Reads the value of the IA32_TSC_AUX MSR (address C0000103H) into the destination register. The value of CS.D and operand-size prefixes (66H and REX.W) do not affect the behavior of the RDPID instruction.

Notes:

RDPID reads processor core id as uint32_r or uint64_r, so read value will not in sequential range [0,MAX_CPU_COUNT]

RDPID is new instruction so it is not widely supported by hardware

Answer 5

taskset + __rdtscp runnable example

And at last, for those that want to have some fun with x86 intrinsics + taskset:

rdtscp.c

#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

#include <x86intrin.h>

int main(void) {
    uint32_t pid;
    printf("0x%016" PRIX64 "\n", (uint64_t)__rdtscp(&pid));
    printf("0x%08" PRIX32 "\n", pid);
    return EXIT_SUCCESS;
}

GitHub upstream.

then compile and run while controlling which core it runs on with taskset:

gcc -ggdb3 -O0 -std=c99 -Wall -Wextra -pedantic -o rdtscp.out rdtscp.c
./taskset -c 0 ./rdtscp.out
./taskset -c 1 ./rdtscp.out

then for each run, the second line, which shows the CPU ID, matches the value set by taskset.

Tested in Ubuntu 19.04 amd64 with an Intel Core i7-7820HQ.