Optimizing Python Code Using SQLite3 + Mutagen

Question

I\'m in the process of improving an open-source music database, which reads songs in from my collection and stores them to an SQLite database. In turn, I\'m able to leverag

Answer 1

If you feel up to it, you could dive into threading :) - It is quite easy in python.

Create two Queue objects, one for IN and one for OUT and in between multiple threads scan the files. (Worker threads, and Queue's are threadsafe)

It should be only ~80loc more code, and you probably can keep the current functions, just wrap em in appropriate classes. (to make "Thread" take it)

But 20min for 90k songs does not seem to be completely out of line. It requires a lot of random access on the disk, and seeking is slow (10ms). so 90,000files, one seek per file is 15min already.