27

I have generated a wordlist.txt of 11 GB by crunch-3.6. When I try to open the file with Vi or gedit, I run into problems because of the file size. How can I view this file?

Improve this question
2
  • 8
    I don't think it's a duplicate. Although its closely related and some answers might be helpful, this one is about viewing and not editing. The accepted answer is not helpful for viewing.
    – pLumo
    Nov 9, 2017 at 9:07
  • 1
    I agree. Viewing a file and editing a file are not the same thing, and the distinction only becomes more important when the file is very large.
    – Eliah Kagan
    Nov 14, 2017 at 15:58

2 Answers 2

Reset to default
48

Don't use a text editor for viewing text.

There are better tools:

View files with less (Scroll with Space, End, Home, PageUp, PageDown; Search with "/something" ; Leave with q).

From less manual:

Less does not have to read the entire input file before starting, so with large input files it starts up faster than text editors like vi (1).

Usage:

less wordlist.txt

Consider the use of less -n:

-n or --line-numbers:

Suppresses line numbers. The default (to use line numbers) may cause less to run more slowly in some cases, especially with a very large input file. Suppressing line numbers with the -n option will avoid this problem.

(thanks for suggesting -n option @pipe)

Use grep to get only the lines you're interested in:

# Show all Lines beginning with A:
grep "^A:"  wordlist.txt

# Show all Lines ending with x and use less for better viewing
grep "x$"  wordlist.txt | less

Use head or tail to get the first or last n lines

head wordlist.txt
tail -n 200 wordlist.txt

For editing text, refer to this question.

Improve this answer
7
  • 4
    Note that more will load the whole file in memory, unlike less. If it's that big, I'd advise against using it. The best option would definitely be to use grep.
    – Nepho
    Nov 9, 2017 at 13:06
  • 1
    My comment might actually be false. I started checking both more and less source code, and both seem to use either open or fopen, which doesn't change anything regarding the way the file is loaded, as far as I know
    – Nepho
    Nov 9, 2017 at 13:17
  • 2
    Agreed. more is a very dated utility, if less is available I don't think of any reason why you'd use more.
    – Nepho
    Nov 9, 2017 at 13:18
  • 1
    @Nepho, more has one huge advantage over less: it doesn't support the LESSOPEN input preprocessor. If you're trying to view the raw text of a man page or something, it's much faster to type more my_docs.man than to dig through the less documentation to figure out how to suppress the preprocessor.
    – Mark
    Nov 9, 2017 at 23:34
  • 1
    I suggest invoking less using something like less -nS for huge files, to speed things up a little.
    – pipe
    Nov 10, 2017 at 12:09
11

Often, just "grep" is enough to find what you need.

If you need more "context" around a particular line, then use "grep -n" to find the line numbers of the lines of interest, then use sed to print out a "chunk" of the file around that line: 

$ grep -n 'word' file 
123:A line with with word in it 

$ sed -n '120,125p' file 
A line 
Another line
The line before
A line with with word in it 
The line after
Something else
Improve this answer
2
  • 10
    no need for sed, you can use grep -B and -A to print lines Before and After ...
    – pLumo
    Nov 9, 2017 at 12:23
  • 8
    @RoVo Or -C for a Chunk (both before and after).
    – wjandrea
    Nov 9, 2017 at 21:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.