Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a CSV file like this:

hostname1 | role1 | environment | tag,list | |
hostname2 | role2 | environment | tag,list,longer | |
hostname3 | role3 | environment | | |

I need a sed expression that adds to the column with the tags (the 4th column) a new tag if there are no tags already, and inserts a new tag with a comma if there are existing tags. I have tried this:

sed "s/\(^$server |.*|.*|\) \(.*|.*|$\)/\1 new,\2/" testfile.csv

where server is defined externally like so:


but it doesn't work, and it doesn't deal with the comma if the tag list is empty.

How would you do this with sed?

share|improve this question
I don't know sed scripting, but found a good tutorial here: – ThiagoPonte Jan 3 '13 at 17:15
This is probably better suited on – user98085 Jan 3 '13 at 17:15

A common mistakes when discovering regular expression is to think that they can be used as parser (and so you see questions like "how can I parse an XML file using REGEXs?"). However you can't insert too much logic inside a regular expression: for complex problems, like yours, you either need a parser or more than one regular expression.

If you want to go with the way of regular expressions, you need two: one for the non-empty tags case and one for the empty tags field. The two regex could look like the following ones:

s/^(hostname123 \|.*?\|.*?\|\s*)(\S+\s*\|.*?\|)$/\1new,\2/
s/^(hostname123 ?\|.*?\|.*?\|)\s*(\|.*?\|)$/\1 new \2/

These two expressions can be used either with two sed piped together:

sed 'expression1' | sed 'expression2'

or, better, with one command in the following way:

sed -r -e '/^'"$server"' \|/ { s/^(.*?\|.*?\|.*?\|\s*)(\S+\s*\|.*?\|)$/\1new,\2/; s/^(.*?\|.*?\|.*?\|)\s*(\|.*?\|)$/\1 new \2/ }' testfile.csv

This is more efficient than using two seds because it does not parse the file twice.

share|improve this answer
wow that was quick and awesome. But it doesn't work: $sed -r -e '/^$server \|/ { s/^(.*?\|.*?\|.*?\|\s*)(\S+\s*\|.*?\|)$/\1new,\2/; s/^(.*?\|.*?\|.*?\|)\s*(\|.*?\|)$/\1 new \2/ }' data2.csv hostname1 | role1 | environment | tag,list | | hostname2 | role2 | environment | tag,list,longer | | hostname3 | role3 | environment | | | – Jason Jan 3 '13 at 18:15
Oops, sorry: the problem is that single quotes do not expand variables. The problem should be fixed now! – Andrea Corbellini Jan 3 '13 at 19:12
brilliant! Can you explain one thing (otherwise I will ask a hundred questions). What is the purpose of the '?'s ? What do they do? – Jason Jan 3 '13 at 19:30
@Jason: I'm not a good teacher. The concept is simple, but I don't know how to explain it in a few words. What I can do is to redirect you to the Python documentation (look for *?, +?, ??) which explains the concept very well, IMO. – Andrea Corbellini Jan 3 '13 at 19:34
+1 that is the most polite and considerate RTFM I have ever received. Bravo, bravo! – Jason Jan 3 '13 at 21:23

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.