At work one of our target platforms is a resource constrained mini-server running Linux (kernel 2.6.13, custom distribution based on an old Fedora Core). The application is writ
1: Yes. 2: This is divided into two steps: Any system call like fork() is wrapped by the glibc to the kernel. The kernel part of the system-call is in kernel/fork.c 3: I don't know. But I would bet that your kernel has it.
The OOM killer kicks in when Low memory is threatened on 32bit boxes. I've never had an issue with this, but there are ways to keep OOM at bay. This problem could be some OOM configuration issue.
Since you are using a Java application, you should consider moving to 64bit Linux. That should definitely fix it. Most 32bit apps can run on a 64bit kernel with no issues as long as relevant libraries are installed.
You could also try the PAE kernel for 32 bit fedora.