How to read and write multiple files?

Question

I want to write a program for this: In a folder I have n number of files; first read one file and perform some operation then store result in a separate file. Then

Answer 1

You may find the fileinput module useful. It is designed for exactly this problem.

Answer 2

import sys

# argv is your commandline arguments, argv[0] is your program name, so skip it
for n in sys.argv[1:]:
    print(n) #print out the filename we are currently processing
    input = open(n, "r")
    output = open(n + ".out", "w")
    # do some processing
    input.close()
    output.close()

Then call it like:

./foo.py bar.txt baz.txt

Answer 3

from pylab import * 
import csv 
import os 
import glob 
import re 
x=[] 
y=[]

f=open("one.txt",'w')

for infile in glob.glob(('*.csv')):
    #   print "" +infile
    csv23=csv2rec(""+infile,'rb',delimiter=',')
    for line in csv23:      
        x.append(line[1])
        #  print len(x)
    for i in range(3000,8000):
        y.append(x[i])
    print ""+infile,"\t",mean(y)
    print >>f,""+infile,"\t\t",mean(y)
    del y[:len(y)]
    del x[:len(x)]

Answer 4

I've just learned of the os.walk() command recently, and it may help you here. It allows you to walk down a directory tree structure.

import os
OUTPUT_DIR = 'C:\\RESULTS'
for path, dirs, files in os.walk('.'):
    for file in files:
        read_f = open(os.join(path,file),'r')
        write_f = open(os.path.join(OUTPUT_DIR,file))

        # Do stuff

Answer 5

I have written a blog on how to easily deal with projects with a large number of files and process them.

https://medium.com/@sreekiranar/directory-and-file-handling-in-python-for-real-world-projects-9bc8baf6ba89

Please go through it and you will have a fair idea on file handling using Python.

Answer 6

Combined answer incorporating directory or specific list of filenames arguments:

import sys
import os.path
import glob

def processFile(filename):
    fileHandle = open(filename, "r")
    for line in fileHandle:
        # do some processing
        pass
    fileHandle.close()

def outputResults(filename):
    output_filemask = "out"
    fileHandle = open("%s.%s" % (filename, output_filemask), "w")
    # do some processing
    fileHandle.write('processed\n')
    fileHandle.close()

def processFiles(args):
    input_filemask = "log"
    directory = args[1]
    if os.path.isdir(directory):
        print "processing a directory"
        list_of_files = glob.glob('%s/*.%s' % (directory, input_filemask))
    else:
        print "processing a list of files"
        list_of_files = sys.argv[1:]

    for file_name in list_of_files:
        print file_name
        processFile(file_name)
        outputResults(file_name)

if __name__ == '__main__':
    if (len(sys.argv) > 1):
        processFiles(sys.argv)
    else:
        print 'usage message'