Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

let us say i have two files a.txt, b.txt

a.txt

87621   3bde   NDF
87621   2dfg   NDF
87621   cdef   NDF
87621   abcd   NDF

b.txt

93291   abcd   NDF
93291   2dfg   NDF
93291   adbf   NDF
93291   gdrg   NDF

My script should create a file , by matching column 2 of both the files and create a file with matching string in new file c.txt like below

c.txt

2dfg
abcd

I tried several options, but not succeded, can you help !

share|improve this question
2  
Typically you show what you tried, and we can tell you where we find errors. I can think of an awk and a join answer. –  glenn jackman Mar 23 at 20:54

3 Answers 3

up vote 0 down vote accepted
comm -12 <(awk '{print $2}' a.txt | sort) <(awk '{print $2}' b.txt | sort) > c.txt

Explanation

  • <(...) is a process substitution. i.e. the output of the commands within the (...) is substituted and used as the two inputs to comm.
  • awk '{print $2}' a.txt prints only the second field of each line.
  • sort sorts the output ready for input into comm.
  • comm compares two sorted files. The -12 flags suppress the lines that are unique to each file, only printing the common lines (see man comm for more details).
share|improve this answer

This reads all the lines in a.txt and keeps track of all of its column 2 values. It then reads b.txt and prints out any repeated values of column 2:

$ awk 'NR==FNR {seen[$2]=1; next} seen[$2]==1 {seen[$2]++; print $2}' a.txt b.txt
abcd
2dfg

The awk command can be divided into the two parts. The first part is:

NR==FNR {seen[$2]=1; next}

awk first tests whether the total number of records (lines) processed, NR, is the same as the number of records processed in this file, FNR. This is true only for the first file processed which is a.txt in this case. Thus, for every line in a.txt, the associative array seen with a key set to the value of the second column, $2, is set to a value of 1. This will be used to indicate that this value of column-2 was "seen" in a.txt. The following command is next which tells awk to skip the rest of the commands and go get the next record.

The second part is:

seen[$2]==1 {seen[$2]++; print $2}

Because of the next command in the first part above, this part is only executed by file b.txt. This part begins with a test: it checks to see if the column-2 value, $2, for this record has been seen before. If is has been seen before once, then the commands in braces are executed. The first command in braces, seen[$2]++ increments the value of seen so that we will never process this value of column-2 again. (Consequently, repeat values of column-2 are ignored.) The second command simply prints the value of column-2 that had been seen before in a.txt.

If we can trust that there are no repeated values for column 2, a simpler script is possible:

$ awk '{print $2}' a.txt b.txt | sort | uniq -d
2dfg
abcd
share|improve this answer
    
This presumes that neither file contains duplicate lines. –  Sparhawk Mar 24 at 0:07
    
@Sparhawk OK. That is handled now too. –  John1024 Mar 24 at 0:16
    
Nice! (Although my awk-fu is not good enough to understand the whole command.) –  Sparhawk Mar 24 at 0:19

The following perl one-liner will do the job:

perl -lane 'BEGIN{%h}$h{$ARGV."\n".$F[1]}++;END{foreach(keys(%h)){s/.*\n//;$f{$_}++};for(keys(%f)){print if$f{$_}>1}}' a.txt b.txt > c.txt
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.