1

I have been looking to all possible answers about "copying file from multiple directories to one single directory" but I cannot find my answer. I need something slightly more specific for my database treatment.

I have a similar question to:

copying files from multiple directory to another multiple directory

but how can I copy these files (they all have exactly the same name) to a new directory and add them the parent directory name, so that I can recognize from which directory they were copied ?

Yes I could copy and rename each of name one after the other, if I hadn't 100 of them :)

I tried loops with find . etc, but still, I cannot add the copied files the parents directory name.

I am still junior in this field, and so I am starting with regex. I guess there is something to do here with them, but I cannot figure out how exactly.

So, this is my question graphically. How to do that with a simple command, or a loop, on my unix terminal:

DIR_a1 --> output_file.bam
DIR_a2 --> output_file.bam                              
DIR_a3 --> output_file.bam  

Copy all output_files.bam in NEWDIR_output so that they appear with parent directory name:

output_file_a1.bam
output_file_a2.bam                              
output_file_a3.bam

In advance, very thankful for any input

  • Could be done easily, but not with a "simple command" as far as I can see. Is this on Ubuntu? It matters. If so, what version? – Jacob Vlijm Jul 6 '16 at 7:07
  • yes this would be done in Ubuntu terminal, the latest version 16.04 – Marylaure Jul 6 '16 at 7:28
2

Copying files from a source- into a target directory, adding the containg folder's name to the file

The script below will copy files recursively from a source directory into a flat directory, adding the name of the (direct) superior directory to the file's name. The directory's name will be inserted before possible extension on the file, exacvtly like your example.

The script

#!/usr/bin/env python3
import os
import sys
import shutil

for root, dirs, files in os.walk(sys.argv[1]):
    for f in files:
        # split off the superior directory's name
        dirname = root.split("/")[-1]
        # define file+path, possible extension position
        src = os.path.join(root, f); spl = f.rfind(".")
        # defining new filename
        newname = f[:spl]+"_"+dirname + f[spl:] if spl != -1 else f+"_"+dirname
        # copy the file into the new directory
        shutil.copyfile(src, os.path.join(sys.argv[2], newname))

How to use

  1. Copy the script intyo an empty file, save it as copyandrename.py
  2. Run it with the source- and target directories as arguments:

    python3 /path/to/copyandrename.py <source_directory> <target_directory>
    

    My test command was e.g.

    python3 '/home/jacob/Bureaublad/pscript_4.py' '/home/jacob/Bureaublad/testmap' '/home/jacob/Bureaublad/target'
    

As always, please test on a sample first!

  • Thank you very much for this very complete response! I will try it in a while and let you know. This looks like a super script. .. However, I don't want to be impertinent and apologizes for not having point it out before, but for the moment all my script are in bash, and I would like, whenever possible, to make this step also in bash so that I can run a single script on my data processing at once. But I keep your answer as priceless for it's precision. Bests, – Marylaure Jul 6 '16 at 9:09
  • @Marylaure just curious, but did you try? – Jacob Vlijm Jul 7 '16 at 10:42
2

You should be able to do this using the shell's parameter substitution features e.g.

$ for d in DIR_a*; do echo cp -- "$d/output_file.bam" "NEWDIR_OUTPUT/output_file${d#DIR}.bam"; done
cp DIR_a1/output_file.bam NEWDIR_OUTPUT/output_file_a1.bam
cp DIR_a2/output_file.bam NEWDIR_OUTPUT/output_file_a2.bam
cp DIR_a3/output_file.bam NEWDIR_OUTPUT/output_file_a3.bam

Remove the echo once you have checked that it is doing the right thing.

  • OK! thank you it works superb with my data. I will however change the mv command with cp or scp, because this has removed my entire files (that I deleted after the test... sig) For chance, I kept a copy of them on another computer :) Well, thank for this, of great help at my level! – Marylaure Jul 6 '16 at 14:47
  • Oh! sorry about that, I have corrected the answer. Shouldn't have posted before coffee :( – steeldriver Jul 6 '16 at 14:51
  • no worries, this was a good focus test for me, eheh Murphy is sad today :) – Marylaure Jul 6 '16 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.