1

I'm having some difficulty downloading a zip file from a server URL using wget and curl. Neither worked the way they're supposed to work. For example, typing

wget https://www.example.com/home/all_tools/tools_123.zip

Note: Website above is just an example and may not lead anywhere.

It should download the zip to the path where I ran the command. Instead, it downloads an empty file that's much smaller (same filename). Correct zip file is around 6MB, the downloaded file is 7KB. I also tried putting "quote marks" around the URL and also got the same result.

However, manually going to the website UI via browser, I'm able to download the correct zip file and unzip it.

Similarly, if I use curl command, it outputs a bunch of HTML text and says "This document has moved". For this, I used the following command.

curl https://www.example.com/home/all_tools/tools_123.zip

Adding -o /home outputs an empty file called home. Using either wget or curl, how do I correctly download the entire zip file with all its contents, and not empty files with the same filename?

Improve this question
1
  • Hi @jcchong9, I'm facing the same issue, did you find any fix for this?
    – Rahul
    Dec 14, 2020 at 22:40

1 Answer 1

Reset to default
4

Have you examined the file you downloaded? A lot of sites redirect to some HTML page - which may fit the size you report. Both wget and curl work exactly the way they are supposed to.

Check the downloaded file with file filename.zip. It will probably tell you that it's HTML.

Improve this answer
4
  • 1
    Thanks for the clarification. I ran "file filename.zip" and it says it's an HTML file. Since that's the case, how do I download the actual zip file via command line using wget or curl and not some HTML file? Going directly to the URL in the browser downloads the correct zip file.
    – RCube123
    Jul 27, 2020 at 19:52
  • 1
    @jcchong9 Does the target URL needs authentication or tokens?
    – Kulfy
    Jul 27, 2020 at 20:05
  • Yes, the site usually asks for 2FA authentication every 24 hours or so. If I already performed the 2FA auth for the day, and I'm able to go to the web server via browser, it should work with wget and curl? The only other way I can think of to automate this is to use xdg-open command, which opens the URL in browser and downloads the correct zip file. Just thought there would be an easier way to do it using wget or curl.
    – RCube123
    Jul 27, 2020 at 20:25
  • @jcchong9 authentication works via cookies stored in your browser, which wget/curl have no idea about. you can try finding the exact request your browser sends (if you go Inspect Element > Network Requests or similar and download the file from the browser, it'll give you an option to export that request as a curl command, which will have all the cookies necessary)
    – ashvatthama
    Jul 28, 2020 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.