How to increase the Java stack size?

Question

I asked this question to get to know how to increase the runtime call stack size in the JVM. I\'ve got an answer to this, and I\'ve also got many useful answers and comments

Answer 1

Add this option

--driver-java-options -Xss512m

to your spark-submit command will fix this issue.

Answer 2

I assume you calculated the "depth of 1024" by the recurring lines in the stack trace?

Obviously, the stack trace array length in Throwable seems to be limited to 1024. Try the following program:

public class Test {

    public static void main(String[] args) {

        try {
            System.out.println(fact(1 << 15));
        }
        catch (StackOverflowError e) {
            System.err.println("true recursion level was " + level);
            System.err.println("reported recursion level was " +
                               e.getStackTrace().length);
        }
    }

    private static int level = 0;
    public static long fact(int n) {
        level++;
        return n < 2 ? n : n * fact(n - 1);
    }
}

Answer 3

If you want to play with the thread stack size, you'll want to look at the -Xss option on the Hotspot JVM. It may be something different on non Hotspot VM's since the -X parameters to the JVM are distribution specific, IIRC.

On Hotspot, this looks like java -Xss16M if you want to make the size 16 megs.

Type java -X -help if you want to see all of the distribution specific JVM parameters you can pass in. I am not sure if this works the same on other JVMs, but it prints all of Hotspot specific parameters.

For what it's worth - I would recommend limiting your use of recursive methods in Java. It's not too great at optimizing them - for one the JVM doesn't support tail recursion (see Does the JVM prevent tail call optimizations?). Try refactoring your factorial code above to use a while loop instead of recursive method calls.

Answer 4

Hmm... it works for me and with far less than 999MB of stack:

> java -Xss4m Test
0

(Windows JDK 7, build 17.0-b05 client VM, and Linux JDK 6 - same version information as you posted)

Answer 5

Weird! You are saying that you want to generate a recursion of 1<<15 depth???!!!!

I'd suggest DON'T try it. The size of the stack will be 2^15 * sizeof(stack-frame). I don't know what stack-frame size is, but 2^15 is 32.768. Pretty much... Well, if it stops at 1024 (2^10) you'll have to make it 2^5 times bigger, it is, 32 times bigger than with your actual setting.

Answer 6

The only way to control the size of stack within process is start a new Thread. But you can also control by creating a self-calling sub Java process with the -Xss parameter.

public class TT {
    private static int level = 0;

    public static long fact(int n) {
        level++;
        return n < 2 ? n : n * fact(n - 1);
    }

    public static void main(String[] args) throws InterruptedException {
        Thread t = new Thread(null, null, "TT", 1000000) {
            @Override
            public void run() {
                try {
                    level = 0;
                    System.out.println(fact(1 << 15));
                } catch (StackOverflowError e) {
                    System.err.println("true recursion level was " + level);
                    System.err.println("reported recursion level was "
                            + e.getStackTrace().length);
                }
            }

        };
        t.start();
        t.join();
        try {
            level = 0;
            System.out.println(fact(1 << 15));
        } catch (StackOverflowError e) {
            System.err.println("true recursion level was " + level);
            System.err.println("reported recursion level was "
                    + e.getStackTrace().length);
        }
    }

}