1

I have hundreds of files and I need to run a command on pairs of such files that only share the initial part of the filename. e.g.:

samtools merge Sample_1_L5.bam Sample_1_L6.bam 
samtools merge Sample_2_L4.bam Sample_2_L5.bam

In essence the pairs share the Sample_x part of the filename and I'd like to make a for loop so as to run the command on all the files in pairs matched on the basis of that initial part of the filename.

Hope this is clear enough.

Thanks for your help! Joanito

Improve this question
0

1 Answer 1

Reset to default
0

The tricky part is going to be keeping the pairs in numeric order. Given:

$ ls *.bam
Sample_1_L5.bam  Sample_2_L4.bam  Sample_3_L10.bam
Sample_1_L6.bam  Sample_2_L5.bam  Sample_3_L9.bam

then using bash and sort with the -V (--version-sort) option

$ printf '%s\0' *.bam | sort -zV | xargs -0 -n2 echo samtools merge
samtools merge Sample_1_L5.bam Sample_1_L6.bam
samtools merge Sample_2_L4.bam Sample_2_L5.bam
samtools merge Sample_3_L9.bam Sample_3_L10.bam

If zsh is available then you can order the glob expansion natively:

 % print -rN -- *.bam(n) | xargs -0 -n2 echo samtools merge
samtools merge Sample_1_L5.bam Sample_1_L6.bam
samtools merge Sample_2_L4.bam Sample_2_L5.bam
samtools merge Sample_3_L9.bam Sample_3_L10.bam

In both cases, remove the echo once you are happy the the proposed mappings are correct.

Improve this answer
6
  • Thank you so much! Not sure about the zsh one, but your first suggestions works well. If I want to add an output filename to the command, how could I do that? e.g. samtools merge Sample1.bam Sample_1_L5.bam Sample_1_L6.bam The output would be the first filename...
    – Joanito Liberti
    Jun 22, 2019 at 17:02
  • @JoanitoLiberti that's probably easiest done by a one-line shell script that derives the output file name by stripping the trailing L* portion ex. xargs -0 -n2 sh -c 'out="${1%_L*}"; echo samtools merge "${out}.bam" "$1" "$2"' sh
    – steeldriver
    Jun 22, 2019 at 23:14
  • Ok, thanks a lot it worked by combining the first part of your first suggestion with the latter one... so: $ printf '%s\0' *.bam | sort -zV | xargs -0 -n2 sh -c 'out="${1%_L*}"; echo samtools merge "${out}.bam" "$1" "$2"' sh
    – Joanito Liberti
    Jun 24, 2019 at 15:42
  • @JoanitoLiberti oops yes sorry if that wasn't clear - also be aware that if you create output files in the same directory as the input files, that will screw up the ordering of subsequent runs (unless you modify the *.bam gob to exclude the output filenames)
    – steeldriver
    Jun 24, 2019 at 18:07
  • are you sure that the command will consider the merged bam files that are created as input (and therefore make a mess with the ordering) if they are created in the input directory? I had actually not thought about this and did create the files in the same directory as the input but the output files look correct to me... When I run the command with echo it spits out the correct list of commands, is this list modified then while the command is running if new bam files are added? It did not seem to be the case.
    – Joanito Liberti
    Jul 9, 2019 at 11:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .