I am looking for the __kernel_vsyscall method within the linux kernel which is used to make system calls. I would like to observe its code to learn more about it, but my gre
Amusingly in your internet searches you didn't think to search stackoverflow.com: What is __kernel_vsyscall?
To answer your more specific question it looks like the symbol itself is defined in (for x86) arch/x86/vdso. It's in assembly, not C.
Assuming your current directory is at the head of the linux kernel source, here are the locations of the files that define the __kernel_vsyscall symbol. (all shown below are only for x86, it does not exists in many other hardware architecture yet).
./arch/x86/vdso/vdso32/syscall.S:
__kernel_vsyscall:
./arch/x86/vdso/vdso32/sysenter.S:
__kernel_vsyscall:
./arch/x86/vdso/vdso32/int80.S:
__kernel_vsyscall:
As you can see, it is essentially declared and implemented inside three file: int80.S, sysenter.S, and syscall.S.
Taking syscall.S:
__kernel_vsyscall:
.LSTART_vsyscall:
push %ebp
.Lpush_ebp:
movl %ecx, %ebp
syscall
movl $__USER32_DS, %ecx
movl %ecx, %ss
movl %ebp, %ecx
popl %ebp
And "syscall" above actually resolved to "int 0x80" if you read the above file and "arch/x86/vdso/vdso32/sigreturn.S" combined.
And for sysenter.S, it is using intel assembly instruction "sysenter" to implement system call transition.
And for int80.S, it is using "int 0x80" for the system call transition.
And if you asked which is the method used for syscall implementation, then look into arch/x86/vdso/vdso32-setup.c:
int __init sysenter_setup(void)
{
void *syscall_page = (void *)get_zeroed_page(GFP_ATOMIC);
const void *vsyscall;
size_t vsyscall_len;
vdso32_pages[0] = virt_to_page(syscall_page);
#ifdef CONFIG_X86_32
gate_vma_init();
#endif
if (vdso32_syscall()) {
vsyscall = &vdso32_syscall_start;
vsyscall_len = &vdso32_syscall_end - &vdso32_syscall_start;
} else if (vdso32_sysenter()){
vsyscall = &vdso32_sysenter_start;
vsyscall_len = &vdso32_sysenter_end - &vdso32_sysenter_start;
} else {
vsyscall = &vdso32_int80_start;
vsyscall_len = &vdso32_int80_end - &vdso32_int80_start;
}
memcpy(syscall_page, vsyscall, vsyscall_len);
relocate_vdso(syscall_page);
return 0;
}
As you can see, modern OS preferred the sysenter approach, as it is faster than the int80 approach. (symbolically the "vds32_syscall_start" will fall back to int80).