GCC how to block system calls within a program?

Question

Does anyone tell me how to block some specific system calls within a program, please? I am building a system which takes a piece of C source code, compiles it with gcc and runs

Answer 1

You can't.

Even this program:

#include <stdio.h>

int main()
{
    printf("Hello, World\n");
    return 0;
}

makes at least one system call (to send the string "Hello, World\n" to standard out). System calls are the only way for a program to interact with the outside World. Use the operating system's security model for security.

Edited for this comment:

I meant not all system calls but malicious system calls, e.g. execv() could be used to execute a BASH script which wipes out my data on the disk.

Your operating system already includes mechanisms to stop that sort of thing happening. For instance, in order for a bash script to wipe out your data, the process must already have write access to that data. That means it must have been started by you or root. Your only real option is not to install untrustworthy software.

By the way, depending on your platform, execv is not necessarily a system call. On Linux, it's a C library wrapper for the real system call (execve).

Answer 2

You could run the compiled program by forking it from a wrapper and use the Linux ptrace(2) facility to intercept and inspect all system calls invoked by the program.

The following example code shows a wrapper that runs the /usr/bin/w command, prints each system call invoked by the command, and terminates the command if it tries to invoke the write(2) system call.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <sys/syscall.h>
#include <sys/reg.h>

#define BAD_SYSCALL __NR_write

int main(int argc, char *argv)
{
    pid_t child;
    int status, syscall_nr;

    child = fork();
    if (child == 0) {
        /* In child. */
        ptrace(PTRACE_TRACEME, 0, NULL, NULL);
        execl("/usr/bin/w", NULL, NULL);
        // not reached
    }

    /* In parent. */
    while (1) {
        wait(&status);

        /* Abort loop if child has exited. */
        if (WIFEXITED(status) || WIFSIGNALED(status))
            break;

        /* Obtain syscall number from the child's process context. */
        syscall_nr = ptrace(PTRACE_PEEKUSER, child, 4 * ORIG_EAX, NULL);
        printf("Child wants to execute system call %d: ", syscall_nr);

        if (syscall_nr != BAD_SYSCALL) {
            /* Allow system call. */
            printf("allowed.\n");
            ptrace(PTRACE_SYSCALL, child, NULL, NULL);
        } else {
            /* Terminate child. */
            printf("not allowed. Terminating child.\n");
            ptrace(PTRACE_KILL, child, NULL, NULL);
        }
    }

    exit(EXIT_SUCCESS);
}

You can do much more powerful things using ptrace, such as inspect and change a process' address space (e.g., to obtain and modify the parameters passed to a system call).

A good introduction can be found in this Linux Journal Article and its follow-up.

Answer 3

Some project have similar idea you can take a look at nacl: http://code.google.com/p/nativeclient/

Answer 4

As others have noted, it's impossible for a program to avoid making system calls, they permate the C library all over the place.

However you might be able to make some headway with careful use of the LD_PRELOAD mechanism, if your platform supports it (e.g. Linux): you write a shared library with the same symbol names as those in the C library, which are called instead of the intended libc functions. (For example, Electric Fence is built as a shared library on Debian-based systems and intercepts calls to malloc, free et al.)

I suspect you could use this mechanism to trap or argument-check calls to any libc functions you don't like, and perhaps to note those which you consider unconditionally safe. It might then be reasonable to scan the compiled executable for the code corresponding to INT 80 to trap out any attempts to make raw syscalls (0xcd 0x80 - though beware of false positives). However I have only give this a few moments of thought, I could easily have missed something or this might turn out to be impractical...

Answer 5

Just to illustrate that this is not possible, the following program:

int main() {
    return 0;
}

makes over 20 system calls as reported using strace. The calls include open (twice) which is one of the calls you seem to want to block.

Answer 6

Well, if you just want to block specific calls, why not just do a grep through the source code before attempting to compile it ? And reject programs which use the insecure system calls.