How to recursively go through all subdirectories and read files?

Question

I have a root-ish directory containing multiple subdirectories, all of which contain a file name data.txt. What I would like to do is write a script that takes in the \"root

Answer 1

[os.path.join(dirpath, filename) for dirpath, dirnames, filenames in os.walk(rootdir) 
                                 for filename in filenames]

A functional approach to get the tree looks shorter, cleaner and more Pythonic.

You can wrap the os.path.join(dirpath, filename) into any function to process the files you get or save the array of paths for further processing

Answer 2

You need to use absolute paths, your file variable is just a local filename without a directory path. The root variable is that path:

with open('output.txt','w') as fout:
    for root, subFolders, files in os.walk(rootdir):
        if 'data.txt' in files:
            with open(os.path.join(root, 'data.txt'), 'r') as fin:
                for lines in fin:
                    dosomething()