3

I'd like to download a file using the wget utility. I followed the instruction on "How do I use wget/curl to download from a site I am logged into?" and the download process works; however, the saved cookies expires after a certain time, so I'm unable to continue downloading the file the following day.

Here's the URL I'm trying to download the a file from:

https://frbservices.org/EPaymentsDirectory/FedACHdir.txt

The download page requires that I click the "Agree" button before I can proceed with the download.

Is there a way to include the "Agree" submission with the wget utility?

Thank you.

4

You can submit a form using wget, using the --post-data option. First, look at the form that page uses:

<form name="acceptedForm" id="acceptedForm" action="submitAgreement" method="post">
...
        <button id="agree_terms_use" name="agreementValue" type="submit" value="Agree">Agree</button>  &nbsp; 
        <button id="disagree_terms_use" name="agreementValue" type="submit" value="Do Not Agree">Do Not Agree</button>

Typically, the value of the action attribute is used to get the target URL, which becomes https://frbservices.org/EPaymentsDirectory/submitAgreement. The names of the form elements become the parameters. Then you need to save the cookies that you get from accepting the agreement. From this SO post, we can construct the command needed:

wget --post-data="agreementValue=Agree" https://frbservices.org/EPaymentsDirectory/submitAgreement --save-cookies cookie.txt --keep-session-cookies --delete-after

Then, we can use these cookies to download the file:

wget --load-cookies=cookie.txt 'https://frbservices.org/EPaymentsDirectory/FedACHdir.txt?AgreementSessionObject=Agree'
  • +1 And is this also possible with aria2c? ;) – A.B. Apr 28 '15 at 8:20
  • @A.B. I don't think aria2c has command line options to submit POST data. That will have to be done by wget or curl. Once that is done, you can use the cookie file with aria2c. – muru Apr 28 '15 at 8:30
  • Thank you Muru! The steps you provided answered my question. – Adam Apr 28 '15 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.