Following on from my How To Combine Text And CSV Files With Python article I decided to create a script to combine CSV or Text files into one file. This isn’t something I need to do on a regular basis but when I do automating it saves a lot of time.
The script is fairly basic. It opens each file within a specified directory, reads the content and appends it to a specified output file.
Although I am usually dealing with CSV files this technique works just fine with plain text files.
#!/usr/bin/env python
# Import module
import os
# Define output filename
OutputFilename = 'combined.csv'
# Define path to input and output files
InputPath = 'D:/MyData'
OutputPath = 'D:'
# Convert forward/backward slashes
InputPath = os.path.normpath(InputPath)
OutputPath = os.path.normpath(OutputPath)
# Define output file and open for writing
filename = os.path.join(OutputPath,OutputFilename)
file_out = open(filename, 'w')
print "Output file opened"
# Loop through each file in input directory
for file in os.listdir(InputPath):
# Define full filename
filename = os.path.join(InputPath,file)
if os.path.isfile(filename):
print " Adding :" + file
file_in = open(filename, 'r')
content = file_in.read()
file_out.write(content)
file_in.close()
# Close output file
file_out.close()
print "Output file closed"
The script uses the os module to provide file and directory manipulation functions. This makes opening, writing and closing files nice and easy.
The only modifications you will need to make are to the OutputFilename, OutputPath and InputPath variables. The “os.path.normpath” and “os.path.join” should mean you don’t need to worry about forward and backward slashes in your paths.
This example reads the entire input file and adds the whole content to the output file. It should be possible to modify this example to read the input file line by line and selectively add lines to the output based on some logic.