java - The process cannot access the file because it is being used by another process

Question

I have a piece of code that monitors a directory for addition of files. Whenever a new file is added to the directory, the contents of the file are picked and published on kafka

Answer 1

Looking at your code it seems when one file is picked by thread for publishing again another thread is picking it up for publishing. That's why no one is able to delete it. It must be concurrency issue only. You should redesign code based up on criterion : steps which can be run concurrently and those which cannot be. So steps in the entire process are :

1. pick up a file (main thread should do it)
2. publish a file (call other thread to do it )
3. delete the file (called thread should delete it)
4. check if any file present (again main thread can do it)

Also the moment a file is selected, you can read it into buffer , delete it and then continue with publish. This will make sure that main thread does not assign this file to some other thread.

Answer 2

It is always a better idea to add sleep time in WatchService events:

if (StandardWatchEventKinds.ENTRY_CREATE.equals(event.kind())) {

       try {
            Thread.sleep(3000);
       } catch (InterruptedException e) {
            e.printStackTrace();
       }

// now do your intended jobs ... 

I needed to add sleep time otherwise it wouldn't work for multiple requests and used to get the error:

The process cannot access the file because it is being used by another process

Answer 3

You have to close all the connection which is accessing that file before deleting it.

Answer 4

I had similar problem as well as in the following thread: Multithreading on Queue when trying to upload a dynamically created files to a Queue service and it was taking 2 days for me to resolve. Thanks to Holger who gave answer as above whereby the locking occurs might be due to the creation had not fully done upon reading by another thread, it has saved me a lot of time.

My initial solution as found a lot from the internet, was:

WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
//queueUploadFile(child);
if (kind == ENTRY_CREATE) {
    uploadToQueue(this.queueId, child);
}

I changed it to:

WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
//queueUploadFile(child);
if (kind == ENTRY_MODIFY) {
    uploadToQueue(this.queueId, child);
}

And everything works perfectly. To handle the "somehow" multiple ENTRY_MODIFY events firing (duplicated files uploaded), I perform deletion on the file inside the uploadToQueue() method once it's uploaded.

I hope my approach taken based on the above contribution will also help others with similar problem.