I'm having a hell of a time getting the proper syntax using sed with variables containing a file path, a regex callback and an arithmatic expression. I don't know how to properly escape things.

I have a file path in a text file with a number next to it. I want to find this line and increment the number.


sed -i -e "s,'$LINE' \([0-9]*\),'$LINE' $((\1+1))," $FILE

For example..

/home/username/Pictures/properties/wallpaper/span/tree.jpg 2

should become..

/home/username/Pictures/properties/wallpaper/span/tree.jpg 3

What is the best way to escape the "." in the file path? How do I properly implement the \1 callback and the arithmetic expression $((\1+1)?

Currently I'm getting bad token \1+1 error.

  • 3
    Have you considered using awk? e.g. awk -v line="$LINE" '$1==line {$2++}1' "$FILE" – steeldriver Apr 28 '16 at 1:52
  • Please, don't use the awk or sed tags unless you're specifically interested in those tools. It's clear here that your primary purpose is getting this done, not using sed or awk specifically. – muru Apr 28 '16 at 6:18
  • well sed is explicit in my Question title and awk was the sensible answer and replacement for sed. I just want people to be able to find this question if they are in a similar situation to me. – deanresin Apr 28 '16 at 6:27

sed and arithmetic do not mix. The right tool for jobs like this is awk.

Consider this test file:

$ cat test.txt
/nonmatching/line 1
/home/username/Pictures/properties/wallpaper/span/tree.jpg 2
/another/non/matching/line 5

We can increment the number of the line that you want with:

$ line="/home/username/Pictures/properties/wallpaper/span/tree.jpg"
$ awk -v f="$line" '$0~f {$NF++} 1' test.txt
/nonmatching/line 1
/home/username/Pictures/properties/wallpaper/span/tree.jpg 3
/another/non/matching/line 5

As you can see, this incremented the number on the line that matched $line.

One limitation is that this code, as written, will not preserve multiple blanks on lines for which the number is incremented. If whitespace is important, we would need more definition of the input format.

How it works

  • -v f="$line"

    This assigns the value of the shell variable line to the awk variable f.

  • $0~f {$NF++}

    Here, $0~f is a condition. It is true if the input line, denoted $0, matches the regex f as defined above. If it matches, then the statement $NF++ is executed. This increments the last field on the line which, for your input, is the number.

  • 1

    This is awk's cryptic shorthand for print-the-line.

Editing a file in-place with awk

If you have a modern GNU awk, you can edit the file in place with:

gawk -i inplace -v f="$line" '$0~f {$NF++} 1' test.txt

With other awks, use:

awk -v f="$line" '$0~f {$NF++} 1' test.txt > tmp.txt && mv -f tmp.txt test.txt

Creating text.txt if it doesn't exist

In the case that the file test.txt is does not contain a line matching $line, this version will such a line with 1 as the count:

gawk -i inplace -v f="$line" '$0~f {n=1; $NF++} 1; END{if(!n) print f,1>>FILENAME}' test.txt

This code adds a variable n. If $line is in the file, n is set to 1. If it isn't, n stays at the default value of 0. At the END of the input file, the value of n is tested. If n is still zero, the line is added.

  • This seems way easier for sure. I'll use your awk solution. Is awk standard in most installs? – deanresin Apr 28 '16 at 1:56
  • Yes, awk is standard with Unix/Linux. (If you are on sun/solaris, be aware that the default awk is old and buggy.) – John1024 Apr 28 '16 at 1:59
  • I'm having trouble getting output. It doesn't seem to be writing to my FILE="/path/to/file.test.text" in awk -v f="$line" '$0~f {$NF++} 1' $FILE. How does awk know I want to read and write to that file? – deanresin Apr 28 '16 at 2:26
  • I have exec &> $(pwd)/echo ${0##*/} | cut -f 1 -d '.'.log at beginning of my script and for some reason awk copies the entire test.txt to test.log with the value incremented in test.log only. – deanresin Apr 28 '16 at 2:30
  • 1
    @deanresin Very good. Also, I just added a version to the answer that uses pure awk. – John1024 Apr 28 '16 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.