Python - Way to recursively find and replace string in text files

Question

I want to recursively search through a directory with subdirectories of text files and replace every occurrence of {$replace} within the files with the contents of a multi l

Answer 1

To avoid recursing into .svn directories, os.walk() allows you to change the dirs list inplace. To simplify the text replacement in a file without requiring to read the whole file in memory, you could use fileinput module. And to filter filenames using a file pattern, you could use fnmatch module as suggested by @David Sulpy:

#!/usr/bin/env python
from __future__ import print_function
import fnmatch
import os
from fileinput import FileInput

def find_replace(topdir, file_pattern, text, replacement):
    for dirpath, dirs, files in os.walk(topdir, topdown=True):
        dirs[:] = [d for d in dirs if d != '.svn'] # skip .svn dirs
        files = [os.path.join(dirpath, filename)
                 for filename in fnmatch.filter(files, file_pattern)]
        for line in FileInput(files, inplace=True):
            print(line.replace(text, replacement), end='')

find_replace(r"C:\test", "*.php", '{$replace}', "multiline\nreplacement")

Answer 2

Use:

pip3 install manip

This lets you use a decorator to create something like:

@manip(at='.php$', recursive=True) # to apply to subfolders
def replace_on_php(text, find, replacement):
    return text.replace(find, replacement)

Now in your prompt you should be able to call

replace_on_php('explode', 'myCustomExplode', path='./myPhPFiles', modify=True)

and this should make the function apply itself on the entire folder.

Answer 3

Check out os.walk:

import os
replacement = """some
multi-line string"""
for dname, dirs, files in os.walk("some_dir"):
    for fname in files:
        fpath = os.path.join(dname, fname)
        with open(fpath) as f:
            s = f.read()
        s = s.replace("{$replace}", replacement)
        with open(fpath, "w") as f:
            f.write(s)

The above solution has flaws, such as the fact that it opens literally every file it finds, or the fact that each file is read entirely into memory (which would be bad if you had a 1GB text file), but it should be a good starting point.

You also may want to look into the re module if you want to do a more complex find/replace than looking for a specific string.