4

I like what the uniq command does, but it looks for duplicates on different lines. I would like to find duplicates even within the same line. what command can do that?

Consider this line this this line, and that I might want to know how many times "this" appears in the same line.

Is there a command that can do this?

0

2 Answers 2

6

You can do:

grep -Eo '[^[:blank:]]+' file.txt | sort | uniq -c
  • grep -Eo '[^[:blank:]]+' gets the words of the file separated by any whitespace(s)

  • sort sorts the output

  • uniq -c gets the cound of words

Example:

% grep -Eo '[^[:blank:]]+' <<<'this  line this this line' | sort | uniq -c
      2 line
      3 this
4

An other way using awk:

echo "this  line this this line"| \
awk  'BEGIN{print "count", "lineNum"}{print gsub(/\<this\>/,"") "\t" NR}'

count lineNum
3   1
  • Which prints count and line number in which this word found.

  • gsub() function's return value is number of substitution made. So we use that to print the number.

  • NR holds the line number so we use it to print the line number.

2
  • Not quite, this will match any substring of a word that is this, not whole word e.g. it will match foothisbar, thisspam and so on. With gawk, you can match word boundaries with \< and \>, so make it as: gsub(/\<this\>/,"")
    – heemayl
    Oct 2, 2016 at 21:22
  • 1
    Accepted and Corrected.
    – snoop
    Oct 3, 2016 at 1:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .