Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I would like to create a short but sweet script for wget to use a .list file. The catch is that I'd like to set directories that these files go into.

Example:

file: url.list
[group 1]
http://www.somehost.com/files/tool.7z
http://www.someotherhost.com/files/icon36.png

[group 2]
http://www.idunno.net/other-tool.tar.gz
http://265.265.265.265/config.ini
http://www.myownsite.com/tools/script-to-run-tool.cmd
eof

([group 1] and [group 2] are just here for readability, they are NOT in my real list file) (yeah I know 265 isn't real, that's why it's an example)

command (currently using which cannot parse folders) wget --continue --timestamping --content-disposition -i url.list

Of course, this currently downloads all 5 files to the same directory. my question is, is there a way to tell wget to use a different folder for group 1, and for group 2, in my case i'd like this to grab several tools that I use at work, i have a separate script in windows that creates a winpe usb key and injects all tools in these directories to the key.

So my ultimate question, can this be done super easy, or does it require me to use a full bash script to grab them and create the folders for them to go in and move them there? (using -o in wget screws with my timestamping, and timestamps are mission critical)

in theory when this finishes, i would like to have a fresh copy of (psuedo names):

tools/cool-tool/tool.7z
tools/cool-tool/icon36.png

tools/special-tool/other-tool.tar.gz
tools/special-tool/config.ini
tools/special-tool/script-to-run-tool.cmd
share|improve this question
add comment

2 Answers

You should be able to do what you want with the wget -P or --directory-prefix option

   -P prefix
   --directory-prefix=prefix
       Set directory prefix to prefix.  The directory prefix is the
       directory where all other files and subdirectories will be saved
       to, i.e. the top of the retrieval tree.  The default is . (the
       current directory).

There are a number of other options for controlling how any source directory components are handled, which may also be useful depending whether you are downloading individual files or are using recursive wgets - see the Directory Options section of the wget manpages.

share|improve this answer
add comment

Create a list that has the URL and target directory on the same line:

http://www.somehost.com/files/tool.7z tools/cool-tool/
http://www.someotherhost.com/files/icon36.png tools/cool-tool/
http://www.idunno.net/other-tool.tar.gz tools/special-tool/
http://265.265.265.265/config.ini tools/special-tool/
http://www.myownsite.com/tools/script-to-run-tool.cmd tools/special-tool/

Then, use a bash loop to read the file and feed it to wget:

while read url target; do 
  wget "$url" -P "$target";
done < url.list
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.