Say, I hook in a sys_* (e.g. sys_open) handler; when I find the argument passed from the userspace is malicious, I print the stack trace of the userspace. How can I do it? <
oprofile
has support for user space stack traces, and these are computed in the kernel by walking the user space stacks. (But note: it doesn't resolve the symbols; that's done by the reporting tools in user space.)
If I had to solve this problem, I would start looking (again) at the oprofile code and just use/adapt what is there.
Maybe this kind of "malicious" occurrence could simply just be modeled as a kind of oprofile event to be recorded, hmm.
You can send a signal(e.g. SIGBUS, SIGKABRT), which can generate a core dump file, to the target process in kernel.
For example:
do_send_sig_info(SIGABRT, SEND_SIG_FORCED, current, true);
Then you can use gdb to show the backtrace of generated core file.
It's very tricky with limited scope of application.