Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How can I download files (that are listed in a text file) using wget or some other automatic way?

Sample file list:
share|improve this question

wget has a built-in flag for this: wget -i your_list You can find this kind of thing by reading man wget

share|improve this answer

Get them in parallel with

cat urlfile | parallel --gnu "wget {}"

By default it will run as many processes as you have cores, you can probably ramp this up another 10x if you really want to pull them down quickly by adding "-j 20" after parallel.

share|improve this answer
I get as many up votes as down for this. No comments, no love! – meawoppl Apr 16 '14 at 1:29
I for one just can't get it to work. I don't see any proc spawned, switching echo for wget doesn't output anything – Jakub Bochenski May 9 '14 at 19:10
Some earlier ubuntus have a dumb defect in this space:… – meawoppl May 11 '14 at 2:30
Note with the 'it will run as many processes as you have cores' - network bandwidth is likely going to be more of a limiting factor. – Wilf Jun 21 '14 at 17:10
It really depends. For a large number of small files this can be almost an order of magnitude faster, as most of the transfer time is the handshake/TCP round trip's. Also in the situation where you are downloading from a number of smaller hosts, sometime the per connection bandwidth is limited, so this will bump things up. – meawoppl Jun 23 '14 at 17:22
xargs -i wget 'http://{}'  < your_list
share|improve this answer
awk '{print "http://" $0;}' list.txt | xargs -l1 wget

where list.txt is your list file

share|improve this answer

I saw Florian Diesch's answer.

I got it to work by including the parameter bqc in the command.

xargs -i wget -bqc 'http://{}' < download.txt

All downloads started in parallel in the background.

  • -b: Background. Go to background immediately after start
  • -q: Quiet. Turn off wget's output
  • -c: Continue. Continue getting a partially-downloaded file
share|improve this answer

Link file links.txt

Command for down load all links file

    cat links.txt | wget -i
share|improve this answer

parallel has a built-in flag --arg-file (-a) that will use an input-file as the source, so you can avoid cat |. You can use

parallel --gnu -a urlfile wget

Or simply parallel --gnu wget < urlfile

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.