Apache FOP 1.0 Multithreading - Too many open files err24

问题

We use Apache FOP to convert a whole lot of XML's to AFP's and PDF's. Our current load would be around 25k files per run on a HP-UX system. We have 8 threads in total that are used to initialize and trigger the FOP conversion in a producer-consumer fashion. Recently there have been multiple failures during conversion and when looked up, we've received generic FOP errors like:

**ERROR,2460364,FOToPDF_Thread_11,FOP Exception, something.pdf,Failed to resolve font with embed-url './Fonts/arial.ttf'**

or its an error failing to load the font metrics file although the files are intact with the right permissions. Many other PDF's are generated so this can't be the problem.

We also wind up with:

**java.io.FileNotFoundException: /PDF/20130111130002/something.pdf (Too many open files (errno:24))**

Judging by the logs and volume being processed, this looks like an FOP problem. I've read that FOP has had this issue in the past with the font files. There have been instances where Apache has opened each font file multiple times and not closed the handles resulting in a large number of open files. This was supposed to be fixed, but if it still persists, what would be a good and quick solution to this, apart from posting this on the Apache lists?

Can the HP-UX maxfiles limit for the open file descriptors per process be increased beyond 2048? Would that help? Any other suggestions?

回答1:

The relevant issue on the Apache FOP project is

https://issues.apache.org/jira/browse/FOP-2189

As I commented there, I was not able to identify any open file handle in FOP 1.0. The constructor of FontFileReader that takes an InputStream argument is indeed not closing it, but the caller (who created the stream) is closing it, see lines 94-106 of

http://svn.apache.org/repos/asf/xmlgraphics/fop/tags/fop-1_0/src/java/org/apache/fop/fonts/truetype/TTFFontLoader.java

回答2:

I've checked in the FOP source code. There is a class named FontFileReader that is used to read true type fonts. This class has two constructors as below:

/**
 * Constructor
 *
 * @param fileName filename to read
 * @throws IOException In case of an I/O problem
 */

public FontFileReader(String fileName) throws IOException {
    final File f = new File(fileName);
    InputStream in = new java.io.FileInputStream(f);
    try {
        init(in);
    } finally {
        in.close();
    }
}

/**
* Constructor
*
* @param in InputStream to read from
* @throws IOException In case of an I/O problem
*/
public FontFileReader(InputStream in) throws IOException {
    init(in);
}

As you can see in the second constructor above, the input stream is not closed., but in the constructor above that it is explicitly closed.

There is another class called TTFFontLoader that loads the ttf fonts into memory, this class uses the second constructor (highlighted) to read the font files, so I think that is why the streams are remaining open and must be reaching the process wide limit of open file handles and is giving the “Too many open files” error.

This issue has been resolved since the re-factor of the FOP font subsystem in the current trunk version of FOP code.

来源：https://stackoverflow.com/questions/14374729/apache-fop-1-0-multithreading-too-many-open-files-err24

标签

java

apache

apache-fop