Problem: 245,000 plus files residing in multiple recup_dir folders (output from photo_rec run) needing to be moved to a centralized folder for further recovery operations. All files are *.jpg files.

I have a text file named newlist1.txt that I have formatted in the following 4 ways. And to change to each variety, a good find/replace editor is essential. these script files are in a folder /disks/Keep/new-s1-recov. I know, I could have used just . to indicate where to mv the files, but initially, the scripts were in another folder, and the period would not have sufficed.

  1. echo !(recov1/recup_dir.1033/f340010.jpg)| xargs mv -t /disks/Keep/new-s1-recov

  2. echo !recov1/recup_dir.1034/f3442050jpg| xargs mv -t /disks/Keep/new-s1-recov

  3. echo recov1/recup_dir.1034/f344208.jpg| xargs mv -t /disks/Keep/new-s1-recov

  4. ls recov1/recup_dir.1034/f344208.jpg| xargs mv -t /disks/Keep/new-s1-recov

the last two examples, 3 & 4, work correctly when executed from the command line. None of the examples work when I type

sh newlist1.txt


bash newlist1.txt

The thing is, I have over 245000 files that need to be moved. Why don't I use the file manager and do groups of several hundred at a time? Because the file manager is possibly the worst piece of software in Ubuntu 16.04.3. So, the command line is the preferable way of accomplishing this task. If I spend several days figuring out how to get it done like this, I'll consider it time well spent. I have wasted more than that on the file manager, waiting for the screen to come out of its seeming daze and finally show that the movement of several files has finally been accomplished.

The formatting above was taken from other examples I found at various times. I tried to adapt my problem to someone else's solution.

So, the question is: How do I format the 245000 lines (using nano texteditor which has a find/replace without having to add some additional software to gedit) and get it into a file which can be invoked by whatever command so that Ubuntu doesn't give me "TOO MANY ARGUMENTS" and die. It will submit each line as a command, and when it's done, it will execute the next line and so on until it's finished.

  • I'm skipping what you are trying to do (why not mv with globbing? (f3442*), but with your scripts, did you make them executable (chmod +x) and you might want to try giving a path bash ./newlist.txt (so it finds it in your current directory; assuming you $PWD isn't in path) – guiverc Feb 28 '18 at 0:38

Since 245,000 filenames probably won't fit on a command line, use find and xargs (man find;man xargs), and do something like:

find recov1/ -type f -iname '*.jpg' -print0 | \
    xargs -0 --no-run-if-empty mv -t /disks/Keep/new-s1-recov

I always use find ... -print0 and xargs -0 in case filenames contain blanks. You may have to use the xarg switches --max-args= or --max-chars= if xargs fails. You could also replace mv with echo mv for testing.

| improve this answer | |
  • Thanks to @waltinator. My dumb!, forgot the +x. But Waltinator, I used your example and only needed to type it on the command line once. None of my file names had spaces in them and I was only searching for .jpg files. So the line was pretty long, but one line did the trick. Problem solved. Thanks again. – Hal Hackney Mar 1 '18 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.