Curly braces in a bash variable expands correctly, but the rsync isn't properly executing

Question

Why won't these braces work in a variable?

I have bash script with rsync with a --exclude option that calls a variable named EXCLUDE. That variable include literal curly braces.

#!/bin/bash

set -eB

EXCLUDE="{'.env','.git*','*.log','config/ssh','/dev','node_modules','/web/app/uploads'}"

rsync -av --dry-run --exclude=$EXCLUDE web/ ${DEPLOY_USER}@${DEPLOY_HOSTNAME}:/sites/${DEPLOY_DOMAIN}/files/web

If I echo $EXCLUDE I get the correct string:

echo $EXCLUDE
{'.env','.git*','*.log','config/ssh','/dev','node_modules','/web/app/uploads'}

If I run the script without the EXCLUDE variable I get the expected results:

rsync -av --dry-run --exclude={'.env','.git*','*.log','config/ssh','/dev','node_modules','/web/app/uploads'} web/ [email protected]:/sites/xyz.org/files/web

building file list ... done
./
test.txt

However if I run the command with the EXCLUDE variable present the results are not correct:

rsync -av --dry-run --exclude=${EXCLUDE} web/ ${DEPLOY_USER}@${DEPLOY_HOSTNAME}:/sites/${DEPLOY_DOMAIN}/files/web

building file list ... done
./
node_modules
test.log
test.txt

Note that if I use echo !! to echo the last command that was run, the results are literally identical to the version without the $EXCLUDE variable:

echo !!
echo rsync -av --dry-run --exclude=$EXCLUDE web/ ${DEPLOY_USER}@${DEPLOY_HOSTNAME}:/sites/${DEPLOY_DOMAIN}/files/web
rsync -av --dry-run --exclude={'.env','.git*','*.log','config/ssh','/dev','node_modules','/web/app/uploads'} web/ [email protected]:/sites/xyz.org/files/web

I can even take the command that was returned by echo !!, run it, and get the expected results!

I've tried enclosing $EXCLUDE in curly braces, I've tried escaping the braces in the variable, I've tried set -B and with and without -e. Even though bash seems to be sending literally curly braces to rsync rsync is not evaluating the contents to the braces.

Why won't these braces work in a variable?

Answer 1

The command with curly braces works because the shell runs the brace expansion on it.

--exclude={'.env','.git*','*.log','config/ssh','/dev','node_modules','/web/app/uploads'}

becomes

--exclude=.env --exclude=.git* --exclude=*.log --exclude=config/ssh --exclude=/dev --exclude=node_modules --exclude=/web/app/uploads

Variable expansion happens later than brace expansion, so storing the braces in a variable doesn't work.

You can use an array instead:

excludes=(.env '.git*' '*.log' config/ssh /dev node_modules /web/app/uploads)
rsync -av --dry-run "${excludes[@]/#/--exclude=}" ...

The ${array[@]/#/PREPEND} syntax is described in man bash under Parameter Expansion, it prepends PREPEND to the beginning of each member of the array.

Answer 2

AFAIK rsync itself only understands simple globs in the --exclude argument. So you're relying on the interactive shell to process the brace expansion.

That means {'.env','.git*','*.log','config/ssh','/dev','node_modules','/web/app/uploads'} isn't "the correct string" - you want the rsync command line to be passed with --exclude='.env' --exclude='.git*' ... already expanded.

You could use brace expansion within an array, and then expand the array within the rsync command:

EXCLUDE=(--exclude={'.env','.git*','*.log','config/ssh','/dev','node_modules',/web/app/uploads'})

rsync -av --dry-run "${EXCLUDE[@]}" web/ [email protected]:/sites/xyz.org/files/web

See also How can we run a command stored in a variable?