4

I've been trying to get this working for a while now. I want to search for a list of files and then copy them all to a certain path.

If I run the command separately, then it works like so:

find <path> -name 'file1' -exec cp '{}' <path2> \;

But I've been unable to run it inside a for loop.

#this works
for f in file1 file2 file3...; do find <path> -name "$f" \; done;
#none of these work (this code tries to find and copy the files named file and list from the path)
for f in file1 file2 file3...; do find <path> -name "$f" -exec cp '{}' <path2> \; done;
for f in file1 file2 file3...; do find <path> -name "$f" -exec cp {} <path2> \; done;

I've tried a few other stuff that weren't likely to work. The first line in the code quote just gets stuck and the others don't copy anything even though they don't get stuck.

I haven't been able to run anything with exec inside a for loop after a search and at this point I'm not sure what to do.

I have solved the immediate problem by searching the files and logging the results to another file and then running a copy inside a for loop separately but I'd still like to know how to do this.

  • 2
    are you really using single quotes for your variable? The shell has to do parameter expansion for the for loop to work... and I don't understand why you would need to do this! If you pass the file list, why do you need to find the files too? why not, for f in file list; do cp "$f" <path2>; done? – Zanna Aug 25 '17 at 6:19
  • @Zanna I'm not really proficient in bash. Just trying different combinations to see what works and you're partially right. Although I have to search for the files because they're not in the same place on all the PCs. So, if I give the absolute paths, the code works but I want to search for the files. – John Hamilton Aug 25 '17 at 8:35
  • @Zanna There. I've edited it just for you. – John Hamilton Aug 25 '17 at 8:46
  • Can you try like this for f in file1 file2 file3; do find /path -name "$f" -exec cp -vt /path/to/test/dir -- {} +; done? – Zanna Aug 25 '17 at 9:01
  • @Zanna yup that works. Make that into an answer, and explain if you can, and I'll pick it as soon as possible. Thank you :) – John Hamilton Aug 25 '17 at 11:05
4

While the other answers are correct I want to offer a different approach that doesn't require multiple invocations of find to scan the same directory structure repeatedly. The basic idea is to use find to

  1. generate a list of files that match the common criteria,
  2. apply a custom filter to that list, and
  3. perform some action, e. g. cp, on the entries of the filtered list.

Implementation 1

(requires Bash to read null-byte-delimited records)

find /some/path -type f -print0 |
while read -rd '' f; do
  case "${f##*/}" in
    file1|file2|file3)
      printf '%s\0' "$f";;
  esac
done |
xargs -r0 -- cp -vt /some/other/path --

Each pipe corresponds to the beginning of the next step of the three steps described above.

The case statement has the advantage of allowing globbing matches. Alternatively you could use Bash's conditional expressions:

if [[ "${f##*/}" == file1 || "${f##*/}" == file2 || "${f##*/}" == file3 ]]; then
  printf '%s\0' "$f"
fi

Implementation 2

If the list of file names to match is a bit longer and cannot be replaced with a small set of globbing patterns, or if the list of file names to match is not known at the time of writing, you can resort to an array that holds the list of file names of interest:

FILE_NAMES=( "file1" "file2" "file3" ... )

find /some/path -type f -print0 |
while read -rd '' f; do
  for needle in "${FILE_NAMES[@]}"; do
    if [ "${f##*/}" = "$needle" ]; then
      printf '%s\0' "$f"
    fi
  done
done |
xargs -r0 -- cp -vt /some/other/path --

Implementation 3

As a variation we can use an associative array which hopefully has faster look-up times than plain "list" arrays:

declare -A FILE_NAMES=( ["file1"]= ["file2"]= ["file3"]= ... )  # Note the superscripts with []=

find /some/path -type f -print0 |
while read -rd '' f; do
  if [ -v FILE_NAMES["${f##*/}"] ]; then
    printf '%s\0' "$f"
  fi
done |
xargs -r0 -- cp -vt /some/other/path --
7

Two issues:

  1. for f in some_file does not iterate over the content of some_file, just iterate over or take the literal string some_file. To get over this forget about for looping for iterating over file content, use a properly implemented while construct.

  2. Variables won't be expanded when put inside single quotes, '$f' in this case. To get your original idea working, use double quotes.

Putting these together, assuming the filenames are newline separated inside file_list file:

while IFS= read -r f; do 
    find /path1 -name "$f" -exec cp '{}' /path2 \;
done <file_list

Or if you know the files would be in the /path1 directory, not under any subdirectory of it, you can just use an array to get the filenames, and use (GNU) cp directly, again assuming newline separated filenames inside file_list:

( 
 IFS=$'\n'
 files=( $(<file_list) )
 cp -t /path2 "${files[@]}"
)

In case of huge number of files, you would be better off iterating over and cp individually rather than dumping into an array:

while IFS= read -r f; do cp -- "$f" /path2; done <file_list

If you have file list like e.g. file1 file2 file3 ... directly in the for construct, then just using double quotes would do:

for f in file1 file2 file3; do 
    find /path1 -name "$f" -exec cp '{}' /path2 \;
done

Now, you can use cp directly here too if all files would not be in any subdirectory:

cp -t /path2 file1 file2 file3

Or you can give static absolute or relative paths in case you want to use cp only to deal with any files under any subdirectory.

  • I need to find the files in the subfolders. I've tried your suggestions and they don't work (they get stuck and don't move on). I have the file list in the for construct like you have suggested in your last example of the for loop. Like so: for f in file1 file2 file3; do find /path1 -name "$f" -exec cp '{}' /path2 \; done P.S. the directory is not big so searching for one file is done instantlt and I'm trying to do this with just one file as well (for f in file1; etc.) and that's not working either. – John Hamilton Aug 25 '17 at 8:28
  • @JohnHamilton Please define don't work, and also show the exact command you have run. – heemayl Aug 25 '17 at 8:49
  • It doesn't work as in the previous cases. It does nothing just hangs there with a new line forever. – John Hamilton Aug 25 '17 at 11:04
  • @JohnHamilton You definitely didn't use what I've used. Check the commands in my answer again... – heemayl Aug 25 '17 at 12:05
5

You clarified that your file list is not a text file, but a series of filenames you want to find with find. You mentioned that this works for you:

for f in file1 file2 file3 ... ; do find <path> -name "$f" \; done;

presumably you mean you get the expected list of files from that command.

You then say this does not work:

for f in file1 file2 file3 ... ; do find <path> -name "$f" -exec cp '{}' <path2> \; done

presumably you mean the files listed before aren't copied to <path2>. I'm not sure what error you're getting, but as it's written, I'd expect that command to hang like this:

$for f in file1 file2 file3 ... ; do find <path> -name "$f" -exec cp '{}' <path2> \; done
>

waiting for the missing ;. Assuming you corrected that problem, I can't think of any other obvious reason why your command would definitely fail. You should be able to manage with

for f in file1 file2 file3 ... ; do find <path> -name "$f" -exec cp '{}' <path2> \; ; done

However I suggest using the -t flag to specify the directory. I have to thank David Foerster for this comment which I think shows the best form for a cp or mv invocation using find. It will refuse to overwrite duplicate files too.

for f in file1 file2 file3; do find /path/to/search -name "$f" -exec cp -vt /path/to/destination -- {} + ; done

Notes

  • -v be verbose - tell us what is being done
  • -t use the specified directory as destination
  • -- don't accept any further options
  • + if there are multiple matches (in your case this is unlikely since for will run once for each item in the file list, but there may be multiple matching files for each name), then construct an argument list for cp instead of running cp multiple times
  • ; separates commands in the shell. The form of a for loop is

    for var in things; do command "$var"; done
    

    If it doesn't see the ; before done, bash will wait for ; or an interrupt signal.

  • Very detailed and thorough answer, thank you. I've tried that other command with the extra semicolon, it ran the command but without copying anything to the directory. I now think that it was trying to copy to the name of the file instead of the directory specified. – John Hamilton Aug 25 '17 at 11:41
  • After a few more trials, this one started working and I'm not sure why it didn't work before: for f in file1 file2 file3 ... ; do find <path> -name "$f" -exec cp '{}' <path2> \; ; done – John Hamilton Aug 25 '17 at 11:49

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.