How to determine the name of the current working file in a regex operation?

Question

If I try to run a regex operation on files like so (example):

$ mv *.ext *.new.ext

That does not work.

How do I ensure that when I batch rename or perform an operation on multiple files that they keep their current name (not the extension, just the name)?

Note: I am not looking for something like the rename command because this has to work for everything, not just renaming files.

Note 2: Okay, these answers are good and explains things to me that I didn't properly understand before. Thanks!

The command in question I am trying to use is called firmtool, and is used in the 3DS homebrew scene.

It packs .bin files into .firm files and I would like it to run on all .bin files in a directory, packing them while preserving the original name.

Here is the syntax I need for the firmtool command:

firmtool build test.firm -n 0x23F00000 -e 0 -D arm9loaderhax.bin -A 0x23F00000 -C NDMA

where test.firm is the output and arm9loaderhax.bin is the input.

I want to run this command that if I have a directory with test1.bin and test2.bin that it produces test1.firm and test2.firm. Please let me know if I can use regular expression or a loop to complete this command. Thank you for any help!

Answer 1

Hmm. You can't¹, not universally.

* is a wildcard. The shell expands it. When you type

*.ext

The shell expands it to a list of all the matching files, like:

1.ext 2.ext whatever.ext

That is what is passed to the command, in your case mv. If mv receives more than two arguments, the last must be a directory, so

mv *.ext *.new

fails with *.new is not a directory, because mv is receiving a long list of arguments.

The shell itself does not support regex, but Linux has many utilities that do, like perl rename...

rename -n 's/(.*)\.ext/$1.new/' *

And remove -n after testing to really rename...

But you can capture the varying names in a variable and use it to loop over the files...

for f in *.ext; do echo mv -v -- "$f" "${f/.ext/.new}"; done

and remove echo after testing to complete the renaming... You can use almost any command with for, not only mv. Here's a slightly different example based on your question:

for f in *.bin; do
  firmtool build "${f%.bin}.firm" -n 0x23F00000 -e 0 -D "$f" -A 0x23F00000 -C NDMA
done

In this case ${f%.bin} will expand to the value of the variable f with the suffix .bin stripped off (if it has such a suffix; otherwise it expands to the value of f).

See the manual on Shell Parameter Expansion for other expansion types.

¹_{waits to be proved wrong}

Answer 2

What happens and why does mv command fail

Let's actually examine what happens under the hood using strace.

$ touch file{1,2}.ext
$ strace -e trace=execve mv *.ext *.new.ext            
execve("/bin/mv", ["mv", "file1.ext", "file2.ext", "*.new.ext"], [/* 80 vars */]) = 0
mv: target '*.new.ext' is not a directory
+++ exited with 1 +++

As Zanna have explained in her answer *.ext will be expanded by shell to all files in current working directory that end in .ext string. What you see in that example is that *.new.ext is not expanded, exactly because there is no file that matches such pattern,i.e. there is no file foo.new.ext anywhere, hence shell passes that directly as argument.

The mv command in turn treats that as the following syntax:

mv [OPTION]... SOURCE... DIRECTORY

If you actually had an existing directory in the last argument, this would work

$ mv *.ext test_dir

$ tree       
.
└── test_dir
    ├── file1.ext
    └── file2.ext

What you actually want to do

Your original goal, as I understand it, is to rename every *.ext file to corresponding file with .new.ext ending. This can be done either with rename, as Zanna shows, or via looping in shell:

$ for f in ./*.ext ; do mv "$f" "$(basename "$f")".new.ext; done                         

$ ls
file1.ext.new.ext  file2.ext.new.ext

Notice the use of basename to extract the part before .ext and use of ./*.ext. Use if ./* is meant to prevent issues with filenames that may have leading - and is generally recommended approach instead of bare *

Additional notes

In your question you stated:

If I try to run a regex operation on files

That's part of the problem. * in shell is Pathname Expansion, which expands to list of files in current working directory. It doesn't work in the same sense of regular expression abc.* where you match string starting with abc with any last character zero or more times.

How do I ensure that when I batch rename or perform an operation on multiple files that they keep their current name (not the extension, just the name)?

You can iterate over the list of files using a for loop, and actively extract the filename using basename command, like I showed above. prename can be used, if you properly define substitutions in the regex expression. What's nice about prename is that it has dry run functionality (the -n switch), where no actual renaming is performed. Same idea can be used with iteration, where you use echo instead of mv first.

I am not looking for something like the rename command because this has to work for everything, not just renaming files.

Again, * as far as shell is concerned deals with files in current directory. This is not the same as regular expressions. Your approach has to be tailored according to the tools you use.