So I'm currently trying to set up a shell script utilizing wget to download a huge amount of files from a server. I'm using the -A 'pattern*.extension' option to download only files with a specific pattern and extension that I'm interested in. (Besides, there are loads of files that I don't want wasting space on my PC.) If I execute wget via command line, everything works just fine and it downloads exactly all the files matching the pattern. Now if I try to run the script with the exact same wget command as before, it suddenly says: "wget: No match". (I should mention that the shell script is being executed by a python script that is provided with a list of IDs to download files from specific subfolders.) But: If I print the exact wget line the shell script executes and copy&paste it into the command line, it works again. That just doesn't make sense to me.
Here is how my wget looks:
wget -r -c -nH -np -nd -e robots=off -P PATH -A 'pattern*.extension' -a logfile.log --progress=bar:force --no-check-certificate https://.../ID/
And here is how my shell script in principle looks like:
#!/usr/bin/tcsh set ID=$1 #just an ID for subfolders set OPT2=$3 #additional options that can be passed to wget set OPT="-r -c -nH -np -nd -e robots=off -P PATH/$ID -A 'pattern*.extension' -a $ID.log" set OOP="--progress=bar:force --no-check-certificate" while ($1 != '') echo "wget $OPT $OOP $OPT2 https://.../$ID/" wget $OPT $OOP $OPT2 https://.../$ID/ shift end
The output then is as follows:
wget -r -c -nH -np -nd -e robots=off -P PATH/ID -A 'pattern*.extension' -a ID.log --progress=bar:force --no-check-certificate https://.../ID/ wget: No match.
But now, if I copy that exact line that my script echoes, it works! Please, enlighten me. I was trying so hard to fix this, but, obviously, I just couldn't manage to do so.
Also: If I replace -A 'pattern*.extension' by -A.extenstion, it downloads all the files with that extension. For some reason it just doesn't work when using a pattern.
As I mentioned above, the shell script is called by a python script:
for ID in IDs: cmd = 'csh PATH/script.csh %s' % (ID) sub.call( cmd, shell=True )
Maybe that also helps and maybe I should also mention that I'm not really an advanced programmer.
Thanks in advance.