Python watchdog windows wait till copy finishes

Question

I am using the Python watchdog module on a Windows 2012 server to monitor new files appearing on a shared drive. When watchdog notices the new file it kicks off a database resto

Answer 1

I've tried the check filesize - wait - check again routine many have suggested above but it's not very reliable. To make it work better I've added a check if the file is still locked.

    file_done = False
    file_size = -1

    while file_size != os.path.getsize(file_path):
        file_size = os.path.getsize(file_path)
        time.sleep(1)

    while not file_done:
        try:
            os.rename(file_path, file_path)
            file_done = True
        except:
            return True

Answer 2

I would add a comment as this isn't an answer to your question but a different approach... but I don't have enough rep yet. You could try monitoring filesize, if it stops changing you can assume copy has finished:

copying = True
size2 = -1
while copying:
    size = os.path.getsize('name of file being copied')
    if size == size2:
        break
    else:
        size2 = os.path.getsize('name of file being copied')
        time.sleep(2)

Answer 3

I am using a different approach that might not be the most elegant one but is easy to do on any plateform if you have control on the side copying the file.

Just had 'in-progress' to the name of the file until the copying is complete, and then rename the file. You can then have a while loop waiting for the file with the name without 'in-progress' to exist and you're good.

Answer 4

In your on_modified event, just wait until the file is finished being copied, via watching the filesize.

Offering a Simpler Loop:

historicalSize = -1
while (historicalSize != os.path.getsize(filename)):
  historicalSize = os.path.getsize(filename)
  time.sleep(1)
print "file copy has now finished"

Answer 5

I'm using following code to wait until file copied (for Windows only):

from ctypes import windll
import time

def is_file_copy_finished(file_path):
    finished = False

    GENERIC_WRITE         = 1 << 30
    FILE_SHARE_READ       = 0x00000001
    OPEN_EXISTING         = 3
    FILE_ATTRIBUTE_NORMAL = 0x80

    if isinstance(file_path, str):
        file_path_unicode = file_path.decode('utf-8')
    else:
        file_path_unicode = file_path

    h_file = windll.Kernel32.CreateFileW(file_path_unicode, GENERIC_WRITE, FILE_SHARE_READ, None, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, None)

    if h_file != -1:
        windll.Kernel32.CloseHandle(h_file)
        finished = True

    print 'is_file_copy_finished: ' + str(finished)
    return finished

def wait_for_file_copy_finish(file_path):
    while not is_file_copy_finished(file_path):
        time.sleep(0.2)

wait_for_file_copy_finish(r'C:\testfile.txt')

The idea is to try open a file for write with share read mode. It will fail if someone else is writing to it.

Enjoy ;)

Answer 6

I had a similar issue recently with watchdog. A rather simple but not very smart workaround was for me to check the change of file size in a while loop using a two-element list, one for 'past', one for 'now'. Once the the values are equal the copying is finished.

Edit: something like this.

past = 0
now = 1
value = [past, now]
while True:
    # change

    # test
    if value[0] == value[1]:
        break
    else:
        value = [value[1], value[0]]