0

I'm having some difficulty downloading a zip file from a server URL using wget and curl. Neither worked the way they're supposed to work. For example, typing

wget https://www.example.com/home/all_tools/tools_123.zip

Note: Website above is just an example and may not lead anywhere.

It should download the zip to the path where I ran the command. Instead, it downloads an empty file that's much smaller (same filename). Correct zip file is around 6MB, the downloaded file is 7KB. I also tried putting "quote marks" around the URL and also got the same result.

However, manually going to the website UI via browser, I'm able to download the correct zip file and unzip it.

Similarly, if I use curl command, it outputs a bunch of HTML text and says "This document has moved". For this, I used the following command.

curl https://www.example.com/home/all_tools/tools_123.zip

Adding -o /home outputs an empty file called home. Using either wget or curl, how do I correctly download the entire zip file with all its contents, and not empty files with the same filename?

1
  • Hi @jcchong9, I'm facing the same issue, did you find any fix for this?
    – Rahul
    Dec 14 '20 at 22:40
3

Have you examined the file you downloaded? A lot of sites redirect to some HTML page - which may fit the size you report. Both wget and curl work exactly the way they are supposed to.

Check the downloaded file with file filename.zip. It will probably tell you that it's HTML.

4
  • Thanks for the clarification. I ran "file filename.zip" and it says it's an HTML file. Since that's the case, how do I download the actual zip file via command line using wget or curl and not some HTML file? Going directly to the URL in the browser downloads the correct zip file.
    – jcchong9
    Jul 27 '20 at 19:52
  • 1
    @jcchong9 Does the target URL needs authentication or tokens?
    – Kulfy
    Jul 27 '20 at 20:05
  • Yes, the site usually asks for 2FA authentication every 24 hours or so. If I already performed the 2FA auth for the day, and I'm able to go to the web server via browser, it should work with wget and curl? The only other way I can think of to automate this is to use xdg-open command, which opens the URL in browser and downloads the correct zip file. Just thought there would be an easier way to do it using wget or curl.
    – jcchong9
    Jul 27 '20 at 20:25
  • @jcchong9 authentication works via cookies stored in your browser, which wget/curl have no idea about. you can try finding the exact request your browser sends (if you go Inspect Element > Network Requests or similar and download the file from the browser, it'll give you an option to export that request as a curl command, which will have all the cookies necessary) Jul 28 '20 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.