3

Lets say I have the following file:

$ cat test.txt
a
-----
b
-----
-----
c
-----
-----
-----
d
-----
e
-----
-----

Now I want to remove all the -----, but only if they're repeating after each other. So the result should look like this:

a
-----
b
-----
c
-----
d
-----
e
-----

I tried grep -Pvz -- "-----\n-----", but that didn't work.

10

That's exactly what the uniq command is made for:

NAME
       uniq - report or omit repeated lines

SYNOPSIS
       uniq [OPTION]... [INPUT [OUTPUT]]

DESCRIPTION
       Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT
       (or standard output).

       With no options, matching lines are merged to the first occurrence.

So

$ uniq test.txt 
a
-----
b
-----
c
-----
d
-----
e
-----

Alternatively, you can use this sed one-liner 69. Delete duplicate, consecutive lines from a file (emulates "uniq") from Sed One-Liners Explained, Part III: Selective Deletion of Certain Lines and Special Applications

sed '$!N; /^\(.*\)\n\1$/!P; D' test.txt

which might be preferred if you want to edit test.txt in place (by adding the -i or --in-place option).

0
0

just do uniq filename.txt As the name states only unique lines are left and the repetitions are merged

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.