I need to match two files and print only those lines that after specific character are not followed only by elements from one of the file.

eg I have two files that look like this:


and this:


I want to match them and if it appears that first _ is only followed by elements from second file then don't print them (example is 1A0I.pdb_2CP). If in other lines it appears that _ is followed some other element but also by element from second file as well then just delete this element that is common in both files but print the line (example is 1A00.pdb_HEM_COA and printed line should be like 1A00.pdb_HEM).

Does anybody has any idea how to do that?

  • Could you elaborate a bit more on the examples? Have you tried anything so far? – M. Becerra Jan 25 '18 at 9:28
  • I have tried something in perl but had a hard time matching the lines in which only elements from the second list appear after _. – djordje Jan 25 '18 at 10:16
  • When a non-matching string is present, is it always the first one after the hyphen? if so, you can just trim hyphen-delimited fields from the end until they stop matching – steeldriver Jan 25 '18 at 13:02
  • @steeldriver It doesn't have to be first it can appear in first, second and every other place. So, just whenever it matches delete the string and whenevet it matches "alone" after hyphen then delete whole line.. – djordje Jan 25 '18 at 16:10
  • I was thinking to give it a try in perl tomorrow.. – djordje Jan 25 '18 at 16:10

You could create a perl hash from the lines of the second file

#!/usr/bin/perl -w

use strict;

BEGIN{ $/ = $\ = "\n"; }

my $stringsfile = shift @ARGV;
open(my $fh, '<:encoding(UTF-8)', $stringsfile)
  or die "Could not open file '$stringsfile' $!";

my %h;

while (defined($_ = <$fh>)) {
    chomp $_;
    $h{$_} = 1;

and then split lines of the first (and subsequent) files into hyphen-separated fields, grep for those fields that are not in the hash, and join it all back together and print if the grep returns anything:

while (defined($_ = <ARGV>)) { 
    chomp $_;
    my ($x, @F) = split(/_/, $_, 0);
    my @y = grep({not $h{$_};} @F);
    print join('_', $x, @y) if @y;


$ ./foo.pl file2 file1

Note: if the potential matches are all at the end, then there's a much simpler approach using awk:

awk '
  NR==FNR {a[$0]++; next} 
  {while ($NF in a) NF--} 
  NF>1 {print}
' file2 file1

For the sample data in your question, both approaches produce the same output.

  • Potential matches dont have to be at the end of each line. This perl script worked fine. Thank You! – djordje Jan 29 '18 at 4:09
  • I have just checked it once again and script fails to delete those lines whenever after hyphen only element from second list appears. When I use these examples from the question it works proprely but when I include 100 more elements into file2, it fails to process. Does the size of a file can be a problem? My file2 has around 160 elements and file1 has around 100 lines.. – djordje Jan 29 '18 at 10:16
  • AFAIK there is no line limit - it works for me on the last line of your test file - my guess is that some of your lines have trailing whitespace characters. – steeldriver Jan 29 '18 at 12:24
  • Yeah it also works perfectly for me on these data from the question but having problems with bigger data..it is surely not anything about the script but about input files. Thank You for this clever suggestion I will delete all whitespace for the input and then try! – djordje Jan 29 '18 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.