Monitoring if a file stopped writing in python

Question

I have a program that keeps writing to a file every second. File writing is happening in a thread parallel to the UI. Due to some hardware issue it stops writing sometimes o

Answer 1

To determine whether the file changes on time in a GUI program, you could use standard tools for your event loop, to run a function every interval seconds e.g., here's how to do it in tkinter:

#!/usr/bin/env python3
import logging
import os
import sys
import tkinter
from datetime import datetime
from time import monotonic as timer, localtime

path = sys.argv[1]
interval = 120 # check every 2 minutes

def check(last=[None]):
    mtime = os.path.getmtime(path) # or os.path.getsize(path)
    logging.debug("mtime %s", datetime.fromtimestamp(mtime))
    if last[0] == mtime: #NOTE: it is always False on the first run
        logging.error("file metadata hasn't been updated, exiting..")
        root.destroy() # exit GUI
    else: # schedule the next run
        last[0] = mtime
        root.after(round(1000 * (interval - timer() % interval)), check)


logging.basicConfig(level=logging.DEBUG,
                    filename=os.path.splitext(__file__)[0] + ".log",
                    format="%(asctime)-15s %(message)s", datefmt="%F %T")
root = tkinter.Tk()
root.withdraw() # hide GUI
root.after(round(1000 * (interval - timer() % interval)), check) # start on boundary
root.mainloop()

You could use supervisord, or systemd, or upstart, etc to respawn your script automatically.

See How to run a function periodically in python.

Answer 2

We can try checking for change in file size as a possible solution by doing the following:

import os
from time import sleep
# other imports

while True:
    file1 = os.stat('file.txt') # initial file size
    file1_size = file1.st_size

    # your script here that collects and writes data (increase file size)
    sleep(1)
    file2 = os.stat('file.txt') # updated file size
    file2_size = file2.st_size
    comp = file2_size - file1_size # compares sizes
    if comp == 0:
        restart_program()
    else:
        sleep(5)

You may need to adjust the sleep() function accordingly these are just estimates that I'm using since I can't test your actual code. In the end this is infinite loop that will keep running as long as you want the script to keep writing.

Another solution is to update your file to:

import os
import sys
from time import sleep
# other imports

while True:
    file1 = os.stat('file.txt') # initial file size
    file1_size = file1.st_size

    # your script here that collects and writes data (increase file size)
    sleep(1)
    file2 = os.stat('file.txt') # updated file size
    file2_size = file2.st_size
    comp = file2_size - file1_size # compares sizes
    if comp == 0:
        sys.exit
    else:
        sleep(5)

Then use a secondary program to run your script as such:

import os
from time import sleep, strftime

while True:
    print(strftime("%H:%M:%S), "Starting"))
    system('main.py') # this is another infinite loop that will keep your script running
    print(strftime("%H:%M:%S), "Crashed"))
    sleep(5)

Answer 3

Finally, after tinkering around with timestamp based options, the following seemed to work for me.

try:
    if time.time()-os.stat(filename).st_mtime>6:
        touch(filename)
        raise ValueError("Yikes! Spike")
except ValueError:
    with open('errors.log','a') as log:
        log.write('Spike/App restarting occured at '+ time.strftime(
                "%H:%M:%S")+' on '+datetime.date.today().strftime('%d/%m/%Y')+'\n')
        log.close()
    restart_program()

Earlier, the problem was it would detect that the file stopped writing with the given time interval and continue to satisfy the same.

time.time()-os.stat(filename).st_mtime>6 

But once this condition is satisfied, unless the file timestamp is updated it continues to satisfy this condition and would keep restarting the program. Now in my solution, I 'touched' the file once (touch used from here) the condition is satisfied and now it works as expected.

Thank you all for your inputs.