0

Manually I enter an URL on a browser that calls a page that has a GET function that directly prompts me to save a file. I would like to automate with a cron job on a script this actions:

  • Call the URL
  • Save the file in a specific folder

Anyone can help to write the script? I can privately provide the URL so to test the behaviour.

5

One way to download an URL without having to use an user interface (text or GUI) is to use wget with all needed parameters (as you wish) as a cron job.

About wget

An example for a single URL (with output name merged with a timestamp, to enrich the answer, could be a static name):

 wget URL -O "/home/uname/dirn/`date +%y%m%d-%H%M%S`-n.htm" -b
 wget URL -O "/home/uname/dirn/`date +\%y\%m\%d-\%H\%M\%S`-n.htm" -b #in crontab

note: in cron you have to use "\%" (as in the second line above) instead of just "%" (as in the first line above), because cron interprets the "percent"-character.

where: -O "/home..../date...-theRestOfTheName" breaks up into:

  • absolute path (cron may have its own path configured, so from practical point of view, it is often favourable to have your path of interest specified in each command)

  • on-the-fly execution of date with output configured as a human-readable kind-of a time-stamp

  • the rest of the target file name

-b asks wget do work in the background, with logging into syslog.

You can optionally use a list of URLs from a file (-i filename). You could also play around with wget and its manpage (man wget) - it is relatively short and can give you lots of ideas.

About cron

Note, cron assusmes your machine is on at all times: cron does not catch up with jobs for the time the machine was off. Editing cron as a current user

crontab -e

Add a line of configuration that specifies the frequency of execution of the command (simple instruction gets displayed as a comment in the topmost lines the crontab editor, please read it). For running every 5 minutes on Sunday you would write:

# m  h  dom mon dow   command
*/5  *   *   *   0    wget google.com -O "/home/uname/dirn/`date +\%y\%m\%d-\%H\%M\%S`-ind3.htm" -b
  • @steeldriver - you are absolutely right about the need for escaping of "%" in the crontab. I made the corrections. – Patryk Mazurkiewicz Jun 16 '18 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.