I am trying to extract the sequence ID and cluster number that occur on different lines within the same text file.

The input looks like

>Cluster 72
0   319aa, >O311_01007... *
>Cluster 73
0   318aa, >1494_00753... *
1   318aa, >1621_00002... at 99.69%
2   318aa, >1622_00575... at 99.37%
3   318aa, >1633_00422... at 99.37%
4   318aa, >O136_00307... at 99.69%
>Cluster 74
0   318aa, >O139_01028... *
1   318aa, >O142_00961... at 99.69%
>Cluster 75
0   318aa, >O300_00856... *

The desired output is the sequence ID in one column and the corresponding cluster number in the second.

>O311_01007  72
>1494_00753  73
>1621_00002  73
>1622_00575  73
>1633_00422  73
>O136_00307  73
>O139_01028  74
>O142_00961  74
>O300_00856  75

Can anyone help with this?

  • Is the sequence ID always going to be the 3d space-separated field on lines that don't start with >? Also, you might be interested in our sister site, Bioinformatics.
    – terdon
    Mar 26 '18 at 17:48

With awk:

awk -F '[. ]*' 'NF == 2 {id = $2; next} {print $3, id}' input-file
  • we split fields on spaces or periods with -F '[. ]*'
  • with lines of two fields, (the >Cluster lines), save the second field as the ID and move to the next line
  • with other lines, print the third field and the saved ID
  • Rather than keying off the number of fields, it might be better to explicitly look for $1 == ">Cluster" instead of NF == 2, depending on what else might be in the file. Mar 26 '18 at 16:29

You can use awk for this:

awk '/>Cluster/{
      print substr($3,2,length($3)-4), c
    }' file

The first block statement is capturing the cluster ID. The second block statement (the default one) is extracting the wanted data, and print it.

  • You don't need to give " " as an argument to print. Just use a comma to separate the arguments and it will use the OFS, default space, to separate the arguments.
    – muru
    Mar 26 '18 at 6:17

Here's an alternative with Ruby as a one-liner :

ruby -ne 'case $_; when /^>Cluster (\d+)/;id = $1;when /, (>\w{4}_\w{5})\.\.\./;puts "#{$1} #{id}";end' input_file

or spread on multiple lines:

ruby -ne 'case $_
when /^>Cluster (\d+)/
  id = $1
when /, (>\w{4}_\w{5})\.\.\./
  puts "#{$1} #{id}"
end' input_file

I guess it's only more readable than the awk version if you know Ruby and regexen. As a bonus, this code might be a bit more robust than simply splitting the lines, because it looks for the surrounding text.



$ perl -ne 'if(/^>.*?(\d+)/){$n=$1;}else{ s/.*(>[^.]+).*/$1 $n/; print}' file 
>O311_01007 72
>1494_00753 73
>1621_00002 73
>1622_00575 73
>1633_00422 73
>O136_00307 73
>O139_01028 74
>O142_00961 74
>O300_00856 75


  • perl -ne: read the input file line by line (-n) and apply the script given by -e to each line.
  • if(/^>.*?(\d+)/){$n=$1;} : if this line starts with a >, find the longest stretch of numbers at the end of the line, and save that as $n.
  • else{ s/.*(>[^.]+).*/$1 $n/; print : if the line doesn't start with >, replace everything with the longest stretch of non-. characters following a > (>[^.]+), i.e. the sequence name ($1 because we have captured the regex match) and the current value of $n.

Or, for a more awk-like approach:

$ perl -lane 'if($#F==1){$n=$F[1]}else{$F[2]=~s/\.+$//; print "$F[2] $n"}' file 
>O311_01007 72
>1494_00753 73
>1621_00002 73
>1622_00575 73
>1633_00422 73
>O136_00307 73
>O139_01028 74
>O142_00961 74
>O300_00856 75

This is just a slightly more cumbersome way of doing the same basic idea as the various awk approaches. I am including it for the sake of completion and for the Perl fans. If you need an explanation, just use the awk solutions :).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.