Facebook Twitter Instagram
    Trending
    • Hard Drive Repair After Wrong Voltage Mishap
    • How To Send Excel Workbook As Email Using Button
    • Compare EV Chargers
    • Home Assistant Server Monitoring
    • Mitsubishi Outlander Dash Cam Install
    • Left or Right Pad a String with a VBScript Function
    • List Files in a Folder Using VBScript
    • How to Verify Checksum of a File
    RSS Twitter Instagram Facebook Pinterest YouTube
    Technology Spy
    • Tutorials & Help
    • Reviews
    Technology Spy
    You are at:Home»Development»Python»How To Split Text And CSV Files With Python
    Coding Programming Guides

    How To Split Text And CSV Files With Python

    0
    By Matt on May 4, 2013 Python, Tutorials & Help

    Despite the march of technology plain text files are still a popular way to transfer and process data. Comma Separated Value (CSV) files are still used to import and export data between spreadsheets, databases, online tools and other data repositories.

    The script shown below will take a text file and split it into a number of smaller files based on a specified line count. This works for normal text as well as CSV files. I use it to split large data sets into smaller batches for import into database systems.

    #!/usr/bin/env python
    
    # Import module
    import os
    
    # Define file_splitter function
    def file_splitter(fullfilepath, lines=50):
      """Splits a plain text file based on line count."""
      path, filename = os.path.split(fullfilepath)
      basename, ext = os.path.splitext(filename)
    
      # Open source text file
      with open(fullfilepath, 'r') as f_in:
        try:
          # Open first output file
          f_output = os.path.join(path, '{}_{}{}'.format(basename, 0, ext))
          f_out = open(f_output, 'w')
    
          # Read input file one line at a time
          for i, line in enumerate(f_in):
            # When current line can be divided by the line
            # count close the output file and open the next one
            if i % lines == 0:
              f_out.close()
              f_output = os.path.join(path, '{}_{}{}'.format(basename, i, ext))
              f_out = open(f_output, 'w')
    
            # Write current line to output file
            f_out.write(line)
    
        finally:
          # Close last output file
          f_out.close()
    
    # Call function with source text file and line count
    file_splitter('input_file.txt', 20)
    

    This example takes the file “input_file.txt” and splits it into a series of files containing a maximum of 20 lines each. The last file may contain less than 20 depending on the number of lines in the source file.

    The output files are numbered and are based on the input file name. The example would produce a set of output files named :

    • input_file_0.txt
    • input_file_20.txt
    • input_file_40.txt
    • input_file_60.txt
    • etc …

    The exact number of files produced will depend on the number of lines in the source file and the number specified in the call to file_splitter.

    File_splitter must be passed the full path to the file. If it is not in the same location as the Python script you may need to include an appropriate parent path or drive letter.

    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
    Previous ArticleHow To Split A List Into Chunks In Python
    Next Article How To Send HTML Emails In Python

    Related Posts

    Left or Right Pad a String with a VBScript Function

    List Files in a Folder Using VBScript

    How to Verify Checksum of a File

    Leave A Reply Cancel Reply

    This site uses Akismet to reduce spam. Learn how your comment data is processed.

    Recent Posts
    November 20, 2022

    Hard Drive Repair After Wrong Voltage Mishap

    March 22, 2022

    How To Send Excel Workbook As Email Using Button

    January 30, 2022

    Compare EV Chargers

    Categories
    • 3D Printing
    • Android
    • Arduino
    • Development
    • ESP8266
    • Excel
    • Gaming
    • General
    • GIMP
    • Home Automation
    • JavaScript
    • Linux
    • Microsoft Office
    • Mobile Devices
    • Oracle APEX
    • Python
    • Raspberry Pi
    • Reviews
    • Security
    • SharePoint
    • Tutorials & Help
    • VBScript
    Tags
    Android APEX Apple Arduino Black Friday csv Elite Dangerous email ESP-01 ESP8266 EV Excel file handling gaming GIMP Gritin Home Assistant Home Automation IMNEED Joystick Linux lists Lumsing Media os.stat os.walk Phishing Power Python Qi Review scam Security select SharePoint 2010 string Syncwire text text files TrueCrypt Ubuntu USB-C VBA VBscript Windows 10
    Web Tools

    A set of quick and basic online tools for web designers and software developers :

    • Hash Generator
    • Text to HTML List
    • Text to HTML Table
    • URL Encoder and Decoder
    • UNIX Timestamp Calculator
    • LED Resistor Calculator
    • Extract Email from Text
    • Mortgage Calculator
    About

    Welcome to Technology Spy, a site devoted to gadgets, computers, programming and all things technology! You’ll also find product reviews for items I own as well as tutorials, guides and scripts for the software I use.

    Archives
    Other Resources
    • MattsBits
    • Raspberry Pi Spy
    Facebook Twitter Instagram Pinterest

    Entries RSS | Comments RSS

    Copyright © 2022 - All Rights Reserved - Matt Hawkins

    mastodon.social@MattHawkins

    Type above and press Enter to search. Press Esc to cancel.