I have a file containing approximately 10 million lines.
I want to remove all lines in the file that are less than six characters.
How do I do this?
There are many ways to do this.
Reverse way, printing lines of length six or more:
You can save the output in a different file using
The original file will be backed up as
If you do not want to keep a backup:
Using shell, Slower, Don't do this, this is just for the sake of showing another method:
Better use list comprehension to be more Pythonic:
It's very simple:
This is extremely efficient, as
So grep will only output lines having 6 (or more) chars, and the other ones are not outputted by grep so they don't make it to resultfile.
Solution #1: using C
Fastest way: compile and run this C program:
(Trick to substitute
Comparison with all the other solutions proposed to this question except the shell solution (test run on a ~91MB file with 10M lines with an average lenght of 8 characters):
Solution #2: using AWK:
Solution #3: using Perl:
You can use Vim in Ex mode: