CSV Splitter for ubuntu?

Question

Is there an application or a command that could to split a csv (preserving the a table header) into smaller equal parts?

I'm trying to import a large 36mb file and the importer is going kaputz.

Here's an example of a windows application: http://download.fyxm.net/CSV-Splitter-33994.html

I guess i could use wine to use this program but would prefer something native of ubuntu.

Answer 1

You can use csvfix.

It's a command line tool for managing CSV files. The download page doesn't have any Linux binaries, but you can compile the source code from a makefile.

Here is a list of commands. In particular, the file_split command looks to be what you're after.

The manual for csvfix can be found here.

Answer 2

So you want to have multiple files which contain part of the entries in the main file and each contains the table header?

If that is what you want, one could write a simple script that takes the first file and the number of files that you want (or the number of lines in each file) and creates those files.

If it does not need to be graphical and you tell me what parameters you want to set, I can try to make such a script for you.

---

This is my quick and dirty solution:

#!/usr/bin/python

import optparse
import sys
import os

parser = optparse.OptionParser(usage="csv-splitter infile", description="Takes a CSV file and splits it into smaller files. The header of the infile is written to each file.")
parser.add_option("-p", "--parts", dest="parts", type="int", default=3, help="number of resulting files")

(options, args) = parser.parse_args()
del parser

infile = args[0]

lines = 0

if not os.path.exists(infile):
    print "file does not exist"
    sys.exit(1)

print "counting lines ..."
with open(infile) as h:
    for line in h:
        lines += 1

lines_per_file = lines/options.parts
print "found %d lines, that makes %d lines per file" % (lines, lines_per_file)

with open(infile) as h:
    header = h.readline()
    read = h.readline()
    for filenumber in xrange(options.parts):
        written_lines = 0
        print "writing to file %d of %d ..." % (filenumber+1, options.parts)
        with open(infile+"-part-"+str(filenumber)+".csv", "w") as outfile:
            outfile.write(header)
            while read != '' and (written_lines <= lines_per_file or filenumber+1 == options.parts):
                outfile.write(read)
                written_lines += 1
                read = h.readline()

print "done"

Answer 3

I also made an online version that allows splitting by lines, file size, or file count. It's quite intuitive and user-friendly.

CSV Live Splitter

Answer 4

I made one because I wanted an online one I could use on my mac..

check this