I have a file with many thousands (possibly even millions) of lines, however I need all of the contents to be on one line and all spaces removed (there shouldn't be any spaces there, but there might be, and if there are, they should be removed). How can this be achieved?

To give some context, I got the base64 output for a huge file and now need to pipe it into another command, however I need to pipe it in as one huge number, but as the file is made of multiple lines as opposed to having it all on one line, I am unable to pipe it into the program because it will treat every line as a new number and I need the whole thing to be treated as one big one.

I am running Ubuntu GNOME 15.10 with GNOME 3.18, I hope that this can be achieved with a simple command and/or script.


The simpler way: use tr.

tr -d '[[:space:]]' <file1 >file2

(Use sponge if you need to write to the same file.)

Pretty easy with Vim:

vim -Nesc '%s/\_s//g' +wq file

With awk, set RS (record separator) to whitespace and ORS (output record separator) to nothing:

awk -v RS='[[:space:]]' -v ORS= '1' foo

With GNU awk, you can do in-place editing using gawk -i inplace.

  • This would be useful on a less big file... This file is so large that opening it in vim just crashes my machine and uses up all the RAM... And takes an incredibly long time to open... – user364819 Apr 10 '16 at 20:29
  • @ParanoidPanda How large is so large? – muru Apr 10 '16 at 20:31
  • It's 14.4GB in size. – user364819 Apr 10 '16 at 21:02

Pipe it through this perl regex:

perl -p -e 's/[\n\r ]*//g'

AWK can do that by looping all the fields and using printf "%s". Example

cat /etc/passwd | awk '{for(i=1;i<=NF;i++) printf "%s",$i}'

Sample run:

$ printf "to be or not to be\nthat is the question" | \         
> awk '{for(i=1;i<=NF;i++) printf "%s",$i}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy