0

read the files data line by line and match the line with the next line, if it matches then save that line in the uniqe.txt file.

3
  • sounds like uniq -d input.txt output.txt (only print duplicate lines, one for each group)
    – user986805
    Oct 11, 2021 at 17:39
  • What is your Ubuntu version/release ? Oct 11, 2021 at 17:42
  • so if you were to have Abcde x 3, should result in two Abcde in unique.txt?
    – user986805
    Oct 11, 2021 at 18:14

2 Answers 2

1

You can accomplish this with a simple for-loop

#!/bin/bash

mapfile -t < text.txt
for ((a=0,b=1; $b<${#MAPFILE[@]}; a++,b++)); do
     [[ ${MAPFILE[$a]} = ${MAPFILE[$b]} ]] && echo ${MAPFILE[$a]}
done > unique.txt
0

Not sure if "successive duplicate lines" is a key issue for you. If not then you simply need the Linux command uniq to eliminate duplicate lines in the file with:

uniq -u inputfile.txt > uniqe.txt

If, however, you are only interested in eliminating successive duplicates you can use awk:

awk 'NR == 1 {a=$0; print}  a!=$0 {a=$0; print}' inputfile > uniqe.txt

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .