26

I have generated a wordlist.txt of 11 GB by crunch-3.6. When I try to open the file with Vi or gedit, I run into problems because of the file size. How can I view this file?

| improve this question | |
  • 8
    I don't think it's a duplicate. Although its closely related and some answers might be helpful, this one is about viewing and not editing. The accepted answer is not helpful for viewing. – pLumo Nov 9 '17 at 9:07
  • 1
    I agree. Viewing a file and editing a file are not the same thing, and the distinction only becomes more important when the file is very large. – Eliah Kagan Nov 14 '17 at 15:58
46

Don't use a text editor for viewing text.

There are better tools:

View files with less (Scroll with Space, End, Home, PageUp, PageDown; Search with "/something" ; Leave with q).

From less manual:

Less does not have to read the entire input file before starting, so with large input files it starts up faster than text editors like vi (1).

Usage:

less wordlist.txt

Consider the use of less -n:

-n or --line-numbers:

Suppresses line numbers. The default (to use line numbers) may cause less to run more slowly in some cases, especially with a very large input file. Suppressing line numbers with the -n option will avoid this problem.

(thanks for suggesting -n option @pipe)

Use grep to get only the lines you're interested in:

# Show all Lines beginning with A:
grep "^A:"  wordlist.txt

# Show all Lines ending with x and use less for better viewing
grep "x$"  wordlist.txt | less

Use head or tail to get the first or last n lines

head wordlist.txt
tail -n 200 wordlist.txt

For editing text, refer to this question.

| improve this answer | |
  • 4
    Note that more will load the whole file in memory, unlike less. If it's that big, I'd advise against using it. The best option would definitely be to use grep. – Nepho Nov 9 '17 at 13:06
  • 1
    My comment might actually be false. I started checking both more and less source code, and both seem to use either open or fopen, which doesn't change anything regarding the way the file is loaded, as far as I know – Nepho Nov 9 '17 at 13:17
  • 2
    Agreed. more is a very dated utility, if less is available I don't think of any reason why you'd use more. – Nepho Nov 9 '17 at 13:18
  • 1
    @Nepho, more has one huge advantage over less: it doesn't support the LESSOPEN input preprocessor. If you're trying to view the raw text of a man page or something, it's much faster to type more my_docs.man than to dig through the less documentation to figure out how to suppress the preprocessor. – Mark Nov 9 '17 at 23:34
  • 1
    I suggest invoking less using something like less -nS for huge files, to speed things up a little. – pipe Nov 10 '17 at 12:09
11

Often, just "grep" is enough to find what you need.

If you need more "context" around a particular line, then use "grep -n" to find the line numbers of the lines of interest, then use sed to print out a "chunk" of the file around that line: 

$ grep -n 'word' file 
123:A line with with word in it 

$ sed -n '120,125p' file 
A line 
Another line
The line before
A line with with word in it 
The line after
Something else
| improve this answer | |
  • 10
    no need for sed, you can use grep -B and -A to print lines Before and After ... – pLumo Nov 9 '17 at 12:23
  • 7
    @RoVo Or -C for a Chunk (both before and after). – wjandrea Nov 9 '17 at 21:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.