Copy all subdirectory names and files with a particular extension to a new directory

Question

I have the following directory structure:

dir1
    py1.py
    py2.py
    subdir1
        file1.py
        anotherfile.txt
    subdir2
        file2.py
        animage.png

I would like to copy the folder structure and the *.py files (except those belonging directly to dir1) to a new directory at the same level as dir1. That is, I'm looking for this:

dir2
    subdir1
        file1.py
    subdir2
        file2.py

I tried (from the layer above dir1):

mkdir dir2
cp -r *.py ../dir2

But this only copied the *.py files from dir 1 into dir2, and ignored the subdirectories.

Answer 1

You could use a shell glob, with the --parents option of cp

Ex. given

$ tree dir1 dir2
dir1
├── py1.py
├── py2.py
├── subdir1
│   ├── anotherfile.txt
│   └── file1.py
└── subdir2
    ├── animage.png
    └── file2.py
dir2

2 directories, 6 files

(note that dir2 already exists) then

$ cd dir1
$ cp --parents -t ../dir2 **/*.py
$ cd ..

gives

$ tree dir1 dir2
dir1
├── py1.py
├── py2.py
├── subdir1
│   ├── anotherfile.txt
│   └── file1.py
└── subdir2
    ├── animage.png
    └── file2.py
dir2
├── subdir1
│   └── file1.py
└── subdir2
    └── file2.py

4 directories, 8 files

I used the globstar pattern ** but if you only need to descend one level you could use a basic * wildcard for the subdirectories.

Answer 2

You can do it with find and cpio, just cd into dir1 and run this:

find . -mindepth 2 -type f -name "*.py" -print | cpio -pdm ../dir2

• find . will start in the current directory ie. dir1
• -mindepth 2 will ignore the contents of the current directory ie. dir1 and include the contents of its sub-directories.
• -type f -name "*.py" -print will include only files with .py extension.

• cpio -pdm will copy. -p creates a list of files, -d creates directories as needed and -m sets the correct modification times on directories.

• ../dir2 will create a directory ( if it does not exist ) named dir2 one level up next to dir1 and copy everything to it.

For reference man find and man cpio.

Best of luck

Answer 3

Use rsync:

mkdir dir2
rsync -amv --include '*/' --include '*/*.py' --exclude '*' dir1/ dir2

This will copy all .py files starting from the second level.

• -m avoid creating empty directories.
• --include '*/' Include all directories
• --include '*/*.py' Include all .py files starting at second level
  • Change to '/*/*.py' if you want to include only second level .py files
• --exclude '*' Exclude everything that is not included.

Answer 4

Okay, I couldn't find a way to make cp or rsync by themselves do what you want, so I've come up with:

for n in dir1/* ; do ls ${n}/*.py >/dev/null 2>&1 ; if [[ $? == 0 ]]; then mkdir dir2/${n/dir1\//} ; cp ${n}/*.py dir2/${n/dir1\//} ; fi ; done

Indenting it properly makes it more readable:

for n in dir1/* 
do
  ls ${n}/*.py >/dev/null 2>&1 
  if [[ $? == 0 ]]
  then
    mkdir dir2/${n/dir1\//} 
    cp ${n}/*.py dir2/${n/dir1\//} 
  fi
done

So, what I'm doing is looking through dir1, assuming each file is a directory and trying to list any *.py files (with output and error disappearing into /dev/null). Obviously, trying this for py1.py and py2.py will fail because they are not directories. It should also fail for any directories that do not have any *.py files in them.

That means that when we are in the if, the result of the previous command ($?) should be something other than 0 and we do nothing. For the subdirs in dir1, the listing should succeed, and so $? will be 0, so we perform the bit inside the loop. That merely creates the subdir in dir2 (remembering to replace "dir1/" with nothing), and copies any *.py files across.

It's certainly not the only way to do this, and while it works for the example you gave, it doesn't work if you have extra levels of subdirs inside subdirs. (e.g. dir1/subdir1/subsubdir1/*.py will not be copied across)