Choosing a random file from a directory (with a large number of files) in Python

Question

I have a directory with a large number of files (~1mil). I need to choose a random file from this directory. Since there are so many files, os.listdir naturally tak

Answer 1

You may be able to get this running:

http://mail.python.org/pipermail/python-list/2009-July/1213182.html

And that's probably the best possible solution to your problem, but only where n is small - if n goes large then os.listdir is probably just as good for your purpose.

I've hunted around and failed to find any other way to open a file in a directory. If I had more time I'd be inclined to play around a bit and generate my own ~1mil files.

---

I just thought of another way to do this: Assuming the files are constant - you're not getting any more or less - you could keep a list of the filenames in a sqlite database. Then it would be relatively simple to query the database for a name by a random ROWID. I don't know if you'll still be plagued by the long time to search for the correct file, but at least getting a filename should take a short amount.

Of course if the files in the directory are randomly named, you can rename the files(?) and put them into a directory structure like AdamK suggests.