JVM runs on FreeBSD with 18Gb memory. creating core file takes about hour, which is very slow, so i need some way to disable it.
If you really mean a core file, and not something specific to java, then you need to use the ulimit command to set the maximum core file size to something small.
I'm not sure this will work, but if you're using the HotSpot VM, try this in the arguments to the java
command:
-XX:HeapDumpPath=/dev/null
If you want to turn off core dumps altogether, you can use the ulimit approach (make sure to use the -H flag to set the hard limit):
ulimit -c -H 0
If this is an OS crash dump then use ulimit to set the maximum core size to 0.