How do I open a large file in a terminal (using vim causes terminal to hang)? [duplicate]

Question

I'm using Ubuntu 14.04. I have a large log file:

-rw-r--r-- 1 rails root 1792390124 Jan 10 14:54 /var/log/unicorn/unicorn.log

Nonetheless, I need to open it and find some things in it. I tried vim, but running

vim /var/log/unicorn/unicorn.log

just opens a blank screen and even when I use Ctrl+C, nothing happens. The terminal is hung. Without upgrading any of the hardware on my machine (but I'm open to installing software), how can I open the file and look around (I don't need to edit it, I just need to read some lines from it).

Answer 1

When dealing with large files, it's better to minimize the amount of information you're going to deal with, and there are tools for that. One tool that has been mentioned is grep to search for specific lines that you need. Another two tools that I'd recommend using are tail and head that can display specific number of lines or bytes from top of the file ( head ) or from the end ( tail ). This is useful if you only need to read those specific parts of the file, but nothing in between.

Alternative way is to use split command. It can break down a file based on number of bytes or lines. For example,

$ mkdir SPLIT

$ cd SPLIT/                                                                                                       

$ split -l 5  /etc/passwd

$ ls
xaa  xab  xac  xad  xae  xaf  xag  xah  xai  xaj

Now, my /etc/password is broken down into smaller files, 5 lines each, which you can open with vim or other text editor

Answer 2

Use the command less. less uses a more efficient way of reading a file into memory. There are 2 equivalent commands: more and most (that last one you need to install). Those put the single page into memory.

But your BEST option for searching would be to use grep. Example:

grep {searchstring} {file}

And have a look at logrotate so you span that log over multiple files.

And split can split your current file into equal parts so you have an easier life looking at that log.

Answer 3

It depends on what your goal is.

If you want to scroll through the entire logfile, you can use the less or more commands to get a page-by-page view of the file at hand.

Otherwise, if you know where in the file you're looking for, you can use the head or tail commands to grab a section/excerpt of the file. See man head and man tail for more information as to how to search for exact parts.

If you're looking for a specific string/entry, you can use grep to search the file for your string. You can either use cat file | grep something or grep something file depending on what you want to do.

Alternatively, if you want to do some more advanced things, you could look at using sed or awk for more advanced operations, if need be.

Answer 4

If you know exactly what line you want, use grep. But if you are fishing for something that you are not sure about, using less is better, because it will show you context (which is especially useful for log files).

Type the / command from within less to open up the search field. Type what you want then hit enter. Then to continue searching for the same word, type n to go forward and N to go backward. To jump to the end of the file type G, then N or ?.