Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a 20 mb file with text and ids. I need a grep or something to count and show any duplicated ids.

share|improve this question
Post an example of the text and id format - one or two entries. – izx Jun 7 '12 at 7:28
id-s are in a json array and that's how this looks "code": ["590702HV-16", "590702HV-17", "590702HV-18", "590702HV-19", "590702HV-20", "590702HV-21", "590702HV-23"], "name": ["Pandora Silver Bracelet From"], "descr": ["<span class=\"Label33\"><br><font size=\"2\"><font face=\"Verdana\"><span style=\"font-weight: bold;\">Pandora Bracelet\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 – Martin Jun 7 '12 at 8:57
up vote 1 down vote accepted
awk '{print $2}' file_name | sort | uniq -c | sort -n | awk '$1>1 {print $2}' > duplicates
for x in $(< duplicates); do grep -n $x file;

so there if you have the ID's in the second column then it will print out the Duplicates ID's

Note: In the file_name ,place your file name .

share|improve this answer
Based on your „uniq” idea, it could have been: awk '{print $2}' file_name | sort | uniq -d. – lgarzo Jun 7 '12 at 8:03

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.