Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have a problem which I am unable to solve: I need to rename a great dump of files using patterns. I tried using this, but I always get an error.

I have a folder, inside with a lot of files. Running ls -1 | wc -l, it returns that I have like 160000 files inside. The problem is, that I wish to move these files to a Windows system, but most of them have characters like : and ? in them, which makes the file unaccessible on said Windows-based systems.

(As a "do not solve but deal with" method, I tried booting up a LiveCD on the Windows system and moving the files using the live OS. Under that Ubuntu, the files were readable and writable on the mounted NTFS partition, but when I booted back on Windows, it showed that the file is there but Windows was unable to access it in any fashion: rename, delete or open.)

I tried running rename 's/\:/_' * inside the folder, but I got Argument list too long error. Some search revealed that it happens because I have so many files, and then I arrived here. The problem is that I don't know how to alter the command to suit my needs, as I always end up having various errors like

  • Trying find -name '*:*' | xargs rename : _, it gives xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option [\n] syntax error at (eval 1) line 1, near ":" [\n] xargs: rename: exited with status 255; aborting
  • Adding the -0 after xargs turns the error message to xargs: argument line too long

These files are archive files generated by various PHP scripts. The best solution would be having a chance to rename them before they are moved to Windows, but if there is no way to do it, we might have a way to rename the files while they are moved to Windows. I use samba and proftpd to move the files.

Unfortunately, graphical software are out of the question as the server containing the files is what it is, a server, with only command-line interface.

share|improve this question

2 Answers 2

up vote 2 down vote accepted

Using find's -exec should work, though it'll recurse through subdirectories, which in turn will fail if any of the subdirectories happen to contain : or ?.

find . -name "*[:?]*" -exec rename 'y/:?/__/' {} +

The argument list too long error you get because the * gets expanded to all 160k files in the directory, which exceeds the maximum argument length on your system.

$ getconf ARG_MAX
2097152

160k filenames may easily exceed 2MiB. You can work past this by either renaming one file at a time

for file in ./*[:?]*; do 
    mv "$file" "${file//[:?]/_}"
done

or fill an array and process them in chunks. How large the chunk could be depends on the average length of the filenames and the ARG_MAX value; or just set a low enough value to be safe, e.g. 1000:

files=( ./*[:?]* ) n=${#files[@]}
size=1000
for (( i=0; i<n; i+=size )); do
    rename 'y/:?/__/' "${files[@]:i:size}"
done

Also see BashFAQ 30.

share|improve this answer
1  
find can be asked to NOT recurse by using -maxdepth 1. This will operate only on files on the current directory. Also, for the files containing ?, it will have to be escaped, e.g. find -name "*\?*". The original poster didn't mention if he cared about subdirectories; if not, the find solution seems like the best one. –  roadmr Jun 19 '12 at 13:49
1  
@roadmr, sure, though I'd rather use shell globs or find's -prune than the non-standard -maxdepth, but that's mainly because I don't work with only GNU/linux systems. And right, I forgot about ?, I'll update the answer to include replacing ?. –  geirha Jun 19 '12 at 14:44
    
@roadmr Negative, the whole banch of files are in the same subfolder. I will look into that FAQ article later, for now, the code in the middle (for file in...) is started to run, we will see how it turns out. Thank you for the answer. –  Whisperity Jun 19 '12 at 19:39

I use this for mass renaming: vidir. It uses more or less the same syntax us vi does!

apt-get install moreutils

Go to the directory with the 160k files

cd dir

Launch vidir

vidir

Next replace all : in the filename with e.g -

: % s/:/-/g

Do the same with other strange characters like ?

: % s/?/-/g

Now save the files, with :w

vidir allows editing of the contents of a directory in a text editor.

http://joeyh.name/code/moreutils/

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.