Process Builder waitFor() issue and Open file limitations

Question

I have inherited some code:

Process p = new ProcessBuilder(\"/bin/chmod\", \"777\", path).start();
p.waitFor();

Basically, there is for som

Answer 1

It seems unlikely that the process would actually complete without closing the files. Could this be happening in a very large # of threads? Or perhaps some of them are not actually completing (ie, it is hanging at waitFor in some cases)?

Otherwise, I think you will be stuck with increasing the open files limit. Assuming that this is a Unix-like system, the "ulimit" command is probably what you are looking for.

Answer 2

I presume you are running these chmod commands in a loop - otherwise I don't see why you'd get so many exceptions. It's possible that you're hitting a deadlock because you're not reading the output of the spawned processes. That certainly used to bite me back in the pre-ProcessBuilder, Runtime.exec() days.

Change your code snippet to the above pattern:

try {
    ProcessBuilder pb = new ProcessBuilder("/bin/chmod", "777", path);    
    pb.redirectErrorStream(true); // merge stdout, stderr of process

    Process p = pb.start();
    InputStreamReader isr = new  InputStreamReader(p.getInputStream());
    BufferedReader br = new BufferedReader(isr);

    String lineRead;
    while ((lineRead = br.readLine()) != null) {
        // swallow the line, or print it out - System.out.println(lineRead);
    }

    int rc = p.waitFor();
    // TODO error handling for non-zero rc
}
catch (IOException e) {
    e.printStackTrace(); // or log it, or otherwise handle it
}
catch (InterruptedException ie) {
    ie.printStackTrace(); // or log it, or otherwise handle it
} 

(credit: this site) and see if that helps the situation.

Answer 3

Thanks for the help guys, this should sort out a load of weirdness going on elsewhere because of it.

Using your(Vinay) example and the stream closings:

try{ 
  fw.close();

  ProcessBuilder pb = new ProcessBuilder("/bin/chmod", "777", path);

  pb.redirectErrorStream(true); // merge stdout, stderr of process
  p = pb.start();

  InputStreamReader isr = new  InputStreamReader(p.getInputStream());
  BufferedReader br = new BufferedReader(isr);

  String lineRead;
  while ((lineRead = br.readLine()) != null) {
    // swallow the line, or print it out - System.out.println(lineRead);
  }

} catch (Exception ioe) {
  Logger.logException(Logger.WARN, ioe.getMessage(), ioe);
} finally {
  try {
    p.waitFor();//here as there is some snipped code that was causing a different
                // exception which stopped it from getting processed

    //missing these was causing the mass amounts of open 'files'
    p.getInputStream().close();
    p.getOutputStream().close();
    p.getErrorStream().close(); 

  } catch (Exception ioe) {
    Logger.logException(Logger.WARN, ioe.getMessage(), ioe);
  }
}

Got the idea from John B Mathews post.

Answer 4

If you're using JAVA 6, you could also try the new setters (for read,write,execute) on the File object. Might be slower, but it should work.