Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have a 20 mb file with text and ids. I need a grep or something to count and show any duplicated ids.

share|improve this question
1  
Post an example of the text and id format - one or two entries. –  izx Jun 7 '12 at 7:28
    
id-s are in a json array and that's how this looks "code": ["590702HV-16", "590702HV-17", "590702HV-18", "590702HV-19", "590702HV-20", "590702HV-21", "590702HV-23"], "name": ["Pandora Silver Bracelet From"], "descr": ["<span class=\"Label33\"><br><font size=\"2\"><font face=\"Verdana\"><span style=\"font-weight: bold;\">Pandora Bracelet\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 –  Martin Jun 7 '12 at 8:57

1 Answer 1

up vote 1 down vote accepted
awk '{print $2}' file_name | sort | uniq -c | sort -n | awk '$1>1 {print $2}' > duplicates
for x in $(< duplicates); do grep -n $x file;
done

so there if you have the ID's in the second column then it will print out the Duplicates ID's

Note: In the file_name ,place your file name .

share|improve this answer
    
Based on your „uniq” idea, it could have been: awk '{print $2}' file_name | sort | uniq -d. –  lgarzo Jun 7 '12 at 8:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.