5

I have file with following content.

This
is,
are,,,
a,,

I want to replace the end of the line with single comma. If there is no comma at the end of the line then add one comma and if there are more than one comma then replace it with single comma.

output is look like this

This,
is,
are,
a,
  • 1
    How shall trailing whitespaces be treated? Replace commas if they are followed by whitespaces in the end of the line as well or not? – Byte Commander Jul 13 '16 at 15:23
19

The simplest approach is to use sed with in-place editing:

sed -i 's/,*$/,/' file

The -i makes the changes to the same file. You can use i.bak to create a file.bak backup file of the original. You can also run it without the -i to see the changes before they're applied. The s/foo/bar/ is the substitution operator. It will replace the first instance of foo with bar. The $ marks the end of the line, and * means "0 or more". So, s/,*$/,/ means "replace 0 or more commas at the end of the line with one comma". If no commas are there, one will be added and if there are more than one, they will be replaced with a single one.

Some other options, for the sake of completion:

  • Perl

    perl -i -pe  's/,*$/,/' file
    

    Same idea as the sed above. This is where sed got the -i idea from.

    If speed is an issue, this one will be the fastest of all the solutions here:

    perl -i -lne 'printf join ",", (grep {$_ ne ""}split(/,/) ); print ","' file
    
  • awk

    awk '{sub(/,*$/,",")}1;' file >newfile
    

    Or, with newer versions of (g)awk:

    awk -iinplace '{sub(/,*$/,",")}1;' file
    
  • Pure shell (slower and less efficient, only included as an example):

    while read line; do echo "${line/%,*/},"; done < file > newfile
    

    The ${var/%foo/bar} will replace any foo from the end of the variable var with bar. Here, we're replacing everything after the last comma, so this won't work if you have multiple commas per line, it only works on your example. The other solutions don't have any of these limitations.

  • 1
    Will sed -i 's/,*$/,/' file add ,, when missing? – Motte001 Jul 13 '16 at 17:01
  • 2
    @Motte001 yes. As I said in the explanation, that will replace 0 or more commas with a single comma. So, if there are no commas, it will add one. – terdon Jul 13 '16 at 17:32
5

In case there are possible commas in between

Slower then the sed option on smaller files, but faster on bigger files (tested on 10MB), is the python option below.

Also if there is the possibility of commas elsewhere in the lines, the long one- liner below will work:

python3 -c "ls = open('file').read().splitlines(); [print( (',').join([s for s in l.split(',') if not s == ''])+',') for l in ls]"

or a little shorter:

python3 -c "[print( (',').join([s for s in l.split(',') if not s == ''])+',') for l in open('f').read().splitlines()]"

...where 'file' is the absolute path to your file, between (single!) quotes.

example

on a file:

something like, for example this
here, read this line, I added some commas,,,,,,,,
are, you convinced or not,
just say something, anything

...the output is:

something like, for example this,
here, read this line, I added some commas,
are, you convinced or not,
just say something, anything,

Explanation

ls = open('file').read().splitlines()

reads the file, splits it into lines

[s for s in l.split(',') if not s == '']

splits the line by the delimeter , removes the (possible) comma(s) from the end of the line

(',').join([s for s in l.split(',') if not s == ''])+','

joins the split sections, adds a comma at the end.

  • All of the approaches in my answer except the shell one (which is bad for many reasons) can deal with commas in between as well. – terdon Jul 13 '16 at 14:27
  • @terdon surprising result (but then again not so surprising): if I make both the sed and python option output to a file, on smaller files, sed is (much) faster, but on bigger files (tested on a 10mb file), the python option is twice as fast on my system. – Jacob Vlijm Jul 13 '16 at 14:45
  • Probably because you're using split instead of a regex. – terdon Jul 13 '16 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.