9

This question already has an answer here:

I'm using Ubuntu 14.04. I have a large log file:

-rw-r--r-- 1 rails root 1792390124 Jan 10 14:54 /var/log/unicorn/unicorn.log

Nonetheless, I need to open it and find some things in it. I tried vim, but running

vim /var/log/unicorn/unicorn.log

just opens a blank screen and even when I use Ctrl+C, nothing happens. The terminal is hung. Without upgrading any of the hardware on my machine (but I'm open to installing software), how can I open the file and look around (I don't need to edit it, I just need to read some lines from it).

marked as duplicate by muru command-line Jan 11 '17 at 1:24

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

4

When dealing with large files, it's better to minimize the amount of information you're going to deal with, and there are tools for that. One tool that has been mentioned is grep to search for specific lines that you need. Another two tools that I'd recommend using are tail and head that can display specific number of lines or bytes from top of the file ( head ) or from the end ( tail ). This is useful if you only need to read those specific parts of the file, but nothing in between.

Alternative way is to use split command. It can break down a file based on number of bytes or lines. For example,

$ mkdir SPLIT

$ cd SPLIT/                                                                                                       

$ split -l 5  /etc/passwd

$ ls
xaa  xab  xac  xad  xae  xaf  xag  xah  xai  xaj

Now, my /etc/password is broken down into smaller files, 5 lines each, which you can open with vim or other text editor

6

Use the command less. less uses a more efficient way of reading a file into memory. There are 2 equivalent commands: more and most (that last one you need to install). Those put the single page into memory.

But your BEST option for searching would be to use grep. Example:

grep {searchstring} {file}

And have a look at logrotate so you span that log over multiple files.

And split can split your current file into equal parts so you have an easier life looking at that log.

  • Like less (but unlike more), most supports scrolling back as well as forward. It just uses different keys to scroll back one page: less uses B (for back), while most uses U (for up). In most terminals, the Page Up key fine in both, though. (In most, B scrolls all the way to the bottom; and in less, U scrolls up... but only by half a screen.) Fortunately, both less and most accept / for searching forward, ? for searching backward, and H for help, so users accustomed to one can usually use the other with ease. – Eliah Kagan Jan 11 '17 at 2:17
5

It depends on what your goal is.

If you want to scroll through the entire logfile, you can use the less or more commands to get a page-by-page view of the file at hand.

Otherwise, if you know where in the file you're looking for, you can use the head or tail commands to grab a section/excerpt of the file. See man head and man tail for more information as to how to search for exact parts.

If you're looking for a specific string/entry, you can use grep to search the file for your string. You can either use cat file | grep something or grep something file depending on what you want to do.

Alternatively, if you want to do some more advanced things, you could look at using sed or awk for more advanced operations, if need be.

  • I am looking for a place in teh file but the reason (I think) grep alone isn't good enough is because I'm looking for a string sequence (beginning with "Http") that occurs before another string sequence (beginning with "120"). I'm not clear on how to do that with grep alone. – Dave Jan 10 '17 at 20:30
  • @Dave well, if you need to find a specific string in file, post another question, explaining exactly what the pattern is , and I'm sure people can provide solutions to that. This question is in different context , I'd say, so new post would be better – Sergiy Kolodyazhnyy Jan 10 '17 at 20:38
  • Hey you were the one who was asking about why I wanted to open the large text file. I'm happy to keep this question in its original context. – Dave Jan 10 '17 at 20:43
0

If you know exactly what line you want, use grep. But if you are fishing for something that you are not sure about, using less is better, because it will show you context (which is especially useful for log files).

Type the / command from within less to open up the search field. Type what you want then hit enter. Then to continue searching for the same word, type n to go forward and N to go backward. To jump to the end of the file type G, then N or ?.

Not the answer you're looking for? Browse other questions tagged or ask your own question.