PipedInputStream / PipedOutputStream in a tight “loop” :: Why the “java.lang.OutOfMemoryError: Java heap space”?

问题

I am experimenting with PipedInputStream and PipedOutputStream and can't understand why the following code would result in a Java Heap exhaustion problem. All transient String objects created should be gc-ed. Why then do I get an OutOfMemoryError ?

I am trying to write and read 1000 String objects each 1 million characters long. The below code fails about half-way through even when invoked with -Xmx2g. What's more the trace:

written string #453
read string #453
written string #454
Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space

... reveals that the PipedInputStream is only one String object "behind" the PipedOutputStream. I don't see why garbage collection has failed to reclaim all necessary heap memory.

import java.io.*;
import java.util.*;


class Worker implements Runnable {

    private ObjectOutputStream oos;
    private PipedInputStream   pis;

    public Worker() throws IOException {
        this.pis = new PipedInputStream();
        this.oos = new ObjectOutputStream(new PipedOutputStream( pis ));
    }

    @Override
    public void run() {
        try {
            for (int i = 0 ; i < 1000 ; i++) {
                oos.writeObject(aBigString());
                System.out.printf("written string #%d\n", i);
            }
            oos.flush();
            oos.close();
        } catch (IOException e) {
            throw new RuntimeException(e.getMessage());
        }
    }

    private static String aBigString() {
        StringBuffer sb = new StringBuffer();
        for (int i = 0 ; i < 1000*1000 ; i++)
            sb.append("X");
        return sb.toString();
    }

    public PipedInputStream getInput() {
        return this.pis;
    }
}


public class FooMain {
    public static void main(String args[]) throws IOException, ClassNotFoundException {
        Worker worker = new Worker();
        (new Thread(worker)).start();
        ObjectInputStream ois = new ObjectInputStream(worker.getInput());
        String record = null;
        int i = 0;
        try {
            while (true) {
                record = (String) ois.readObject();
                System.out.printf("read string #%d", i++);
            }
        } catch (EOFException e) {
            ois.close();
            System.out.println("done.");
        }
    }
}

回答1:

This has nothing to do with the Piped streams. you are hitting one of the classic pitfalls of the Object streams. In order to preserve object identity, the streams will hold onto all objects pass through them. If you need to use these streams for a large number of objects, you need to periodically call reset() on the ObjectOutputStream (but beware that object identities are not preserved across reset calls).

回答2:

I'd recommend downloading Visual VM, installing all the plugins, and attaching it to your PID while the code executes. It'll show you memory, threads, objects, CPU, and lots more.

来源：https://stackoverflow.com/questions/17167595/pipedinputstream-pipedoutputstream-in-a-tight-loop-why-the-java-lang-out

标签

java

garbage-collection

heap