Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

let us say i have two files a.txt, b.txt


87621   3bde   NDF
87621   2dfg   NDF
87621   cdef   NDF
87621   abcd   NDF


93291   abcd   NDF
93291   2dfg   NDF
93291   adbf   NDF
93291   gdrg   NDF

My script should create a file , by matching column 2 of both the files and create a file with matching string in new file c.txt like below



I tried several options, but not succeded, can you help !

share|improve this question
Typically you show what you tried, and we can tell you where we find errors. I can think of an awk and a join answer. – glenn jackman Mar 23 '14 at 20:54
up vote 0 down vote accepted
comm -12 <(awk '{print $2}' a.txt | sort) <(awk '{print $2}' b.txt | sort) > c.txt


  • <(...) is a process substitution. i.e. the output of the commands within the (...) is substituted and used as the two inputs to comm.
  • awk '{print $2}' a.txt prints only the second field of each line.
  • sort sorts the output ready for input into comm.
  • comm compares two sorted files. The -12 flags suppress the lines that are unique to each file, only printing the common lines (see man comm for more details).
share|improve this answer

This reads all the lines in a.txt and keeps track of all of its column 2 values. It then reads b.txt and prints out any repeated values of column 2:

$ awk 'NR==FNR {seen[$2]=1; next} seen[$2]==1 {seen[$2]++; print $2}' a.txt b.txt

The awk command can be divided into the two parts. The first part is:

NR==FNR {seen[$2]=1; next}

awk first tests whether the total number of records (lines) processed, NR, is the same as the number of records processed in this file, FNR. This is true only for the first file processed which is a.txt in this case. Thus, for every line in a.txt, the associative array seen with a key set to the value of the second column, $2, is set to a value of 1. This will be used to indicate that this value of column-2 was "seen" in a.txt. The following command is next which tells awk to skip the rest of the commands and go get the next record.

The second part is:

seen[$2]==1 {seen[$2]++; print $2}

Because of the next command in the first part above, this part is only executed by file b.txt. This part begins with a test: it checks to see if the column-2 value, $2, for this record has been seen before. If is has been seen before once, then the commands in braces are executed. The first command in braces, seen[$2]++ increments the value of seen so that we will never process this value of column-2 again. (Consequently, repeat values of column-2 are ignored.) The second command simply prints the value of column-2 that had been seen before in a.txt.

If we can trust that there are no repeated values for column 2, a simpler script is possible:

$ awk '{print $2}' a.txt b.txt | sort | uniq -d
share|improve this answer
This presumes that neither file contains duplicate lines. – Sparhawk Mar 24 '14 at 0:07
@Sparhawk OK. That is handled now too. – John1024 Mar 24 '14 at 0:16
Nice! (Although my awk-fu is not good enough to understand the whole command.) – Sparhawk Mar 24 '14 at 0:19

The following perl one-liner will do the job:

perl -lane 'BEGIN{%h}$h{$ARGV."\n".$F[1]}++;END{foreach(keys(%h)){s/.*\n//;$f{$_}++};for(keys(%f)){print if$f{$_}>1}}' a.txt b.txt > c.txt
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.