Speeding up file system access?

Question

My app scans part of a file system, and my users reported it was very slow when they were scanning a network drive. Testing my code, I identified the bottleneck: the methods

Answer 1

Here's a before and after code example for using listFiles and using isDirectory to walk a directory tree (my code uses a generic callback to actually do something with each directory and file; if I was coding C# this would be a delegate).

As you can see the listFiles approach is actually more compact and readily understood as well as being marginally faster on a local drive (950 ms vs 1000 ms), and LAN drive (26 seconds, vs 28 seconds), both for 23 thousand files.

It's very possible that for a remote connected drive the speedup could be substantial, but I can't test that from work. A little surprisingly the speedup is still only about 10% across a Windows RAS VPN to a network drive.

New Code

static public int processDirectory(File dir, Callback cbk, FileSelector sel) {
    dir=dir.getAbsoluteFile();
    return _processDirectory(dir.getParentFile(),dir,new Callback.WithParams(cbk,2),sel);
    }

static private int _processDirectory(File par, File fil, Callback.WithParams cbk, FileSelector sel) {
    File[]                              ents=(sel==null ? fil.listFiles() : fil.listFiles(sel));    // listFiles returns null if fil is not a directory
    int                                 cnt=1;

    if(ents!=null) {
        cbk.invoke(fil,null);
        for(int xa=0; xa

Old Code

static public int oldProcessDirectory(File dir, Callback cbk, FileSelector sel) {
    dir=dir.getAbsoluteFile();
    return _processDirectory(dir,new Callback.WithParams(cbk,2),sel);
    }

static private int _processDirectory(File dir, Callback.WithParams cbk, FileSelector sel) {
    File[]                              ents=(sel==null ? dir.listFiles() : dir.listFiles(sel));
    int                                 cnt=1;

    cbk.invoke(dir,null);

    if(ents!=null) {
        for(int xa=0; xa