I have generated a wordlist.txt
of 11 GB by crunch-3.6
. When I try to open the file with Vi or gedit, I run into problems because of the file size. How can I view this file?
-
8I don't think it's a duplicate. Although its closely related and some answers might be helpful, this one is about viewing and not editing. The accepted answer is not helpful for viewing.– pLumoNov 9, 2017 at 9:07
-
1I agree. Viewing a file and editing a file are not the same thing, and the distinction only becomes more important when the file is very large.– Eliah KaganNov 14, 2017 at 15:58
2 Answers
Don't use a text editor for viewing text.
There are better tools:
View files with less
(Scroll with Space, End, Home, PageUp, PageDown; Search with "/something" ; Leave with q).
From less
manual:
Less does not have to read the entire input file before starting, so with large input files it starts up faster than text editors like vi (1).
Usage:
less wordlist.txt
Consider the use of less -n
:
-n or --line-numbers:
Suppresses line numbers. The default (to use line numbers) may cause less to run more slowly in some cases, especially with a very large input file. Suppressing line numbers with the
-n
option will avoid this problem.
(thanks for suggesting -n option @pipe)
Use grep
to get only the lines you're interested in:
# Show all Lines beginning with A:
grep "^A:" wordlist.txt
# Show all Lines ending with x and use less for better viewing
grep "x$" wordlist.txt | less
Use head
or tail
to get the first or last n lines
head wordlist.txt
tail -n 200 wordlist.txt
For editing text, refer to this question.
-
4Note that
more
will load the whole file in memory, unlikeless
. If it's that big, I'd advise against using it. The best option would definitely be to usegrep
.– NephoNov 9, 2017 at 13:06 -
1My comment might actually be false. I started checking both
more
andless
source code, and both seem to use eitheropen
orfopen
, which doesn't change anything regarding the way the file is loaded, as far as I know– NephoNov 9, 2017 at 13:17 -
2Agreed.
more
is a very dated utility, ifless
is available I don't think of any reason why you'd usemore
.– NephoNov 9, 2017 at 13:18 -
1@Nepho,
more
has one huge advantage overless
: it doesn't support theLESSOPEN
input preprocessor. If you're trying to view the raw text of a man page or something, it's much faster to typemore my_docs.man
than to dig through theless
documentation to figure out how to suppress the preprocessor.– MarkNov 9, 2017 at 23:34 -
1I suggest invoking less using something like
less -nS
for huge files, to speed things up a little.– pipeNov 10, 2017 at 12:09
Often, just "grep" is enough to find what you need.
If you need more "context" around a particular line, then use "grep -n" to find the line numbers of the lines of interest, then use sed to print out a "chunk" of the file around that line:
$ grep -n 'word' file
123:A line with with word in it
$ sed -n '120,125p' file
A line
Another line
The line before
A line with with word in it
The line after
Something else
-
10no need for sed, you can use
grep -B
and-A
to print lines Before and After ...– pLumoNov 9, 2017 at 12:23 -
8