In Python, fastest way to build a list of files in a directory with a certain extension

Question

In Python on a GNU/Linux system, what\'s the fastest way to recursively scan a directory for all .MOV or .AVI files, and to store them in a list? <

Answer 1

You can use os.walk() for recuresive walking and glob.glob() or fnmatch.filter() for file matching:

Check this answer

Answer 2

I'd use os.walk to scan the directory, os.path.splitext to grab the suffix and filter them myself.

suffixes = set(['.AVI', '.MOV'])
for dirpath, dirnames, filenames in os.walk('.'):
    for f in filenames:
        if os.path.splitext(f)[1] in suffixes:
            yield os.path.join(dirpath, f)

Answer 3

pattern = re.compile('.*\.(mov|MOV|avi|mpg)$')

def fileList(source):
   matches = []
   for root, dirnames, filenames in os.walk(source):
       for filename in filter(lambda name:pattern.match(name),filenames):
           matches.append(os.path.join(root, filename))
   return matches

Answer 4

I suggest the use of os.walk and a carefully reading of its documentation.

This may be a one liner approach:

[f for root,dirs,files in os.walk('/your/path') for f in files if is_video(f)]

Where in is_video you check your extensions.

Answer 5

Python 2.x:

import os

def generic_tree_matching(rootdirname, filterfun):
    return [
        os.path.join(dirname, filename)
        for dirname, dirnames, filenames in os.walk(rootdirname)
        for filename in filenames
        if filterfun(filename)]

def matching_ext(rootdirname, extensions):
    "Case sensitive extension matching"
    return generic_tree_matching(
        rootdirname,
        lambda fn: fn.endswith(extensions))

def matching_ext_ci(rootdirname, extensions):
    "Case insensitive extension matching"
    try:
        extensions= extensions.lower()
    except AttributeError: # assume it's a sequence of extensions
        extensions= tuple(
            extension.lower()
            for extension in extensions)
    return generic_tree_matching(
        rootdirname,
        lambda fn: fn.lower().endswith(extensions))

Use either matching_ext or matching_ext_ci with arguments the root folder and an extension or a tuple of extensions:

>>> matching_ext(".", (".mov", ".avi"))

Answer 6

Example for a list of files in current directory. You can expand this for specific paths.

import glob
movlist = glob.glob('*.mov')