read the files data line by line and match the line with the next line, if it matches then save that line in the uniqe.txt file.
2 Answers
You can accomplish this with a simple for-loop
#!/bin/bash
mapfile -t < text.txt
for ((a=0,b=1; $b<${#MAPFILE[@]}; a++,b++)); do
[[ ${MAPFILE[$a]} = ${MAPFILE[$b]} ]] && echo ${MAPFILE[$a]}
done > unique.txt
Not sure if "successive duplicate lines" is a key issue for you. If not then you simply need the Linux command uniq to eliminate duplicate lines in the file with:
uniq -u inputfile.txt > uniqe.txt
If, however, you are only interested in eliminating successive duplicates you can use awk:
awk 'NR == 1 {a=$0; print} a!=$0 {a=$0; print}' inputfile > uniqe.txt
uniq -d input.txt output.txt
(only print duplicate lines, one for each group)Abcde x 3
, should result in twoAbcde
in unique.txt?