4

First of all, sorry my bad English. Im new to Crontab and i was making some schedules. I was scraping some data from the internet with Python and i have this website that when you scrape some data, you have to wait exactly 24 hours to get the next part of data, so if my cronjob starts at 00.00 every day and it needs some seconds to run Selenium and scrape the data. The next day have to run with some seconds offset to make sure passet 24 hours from the moment i made the scraping and not 24 hours from the moment the job started. So, is there any way to run the job every day with 5 minutes delay from the day before or i need to make something in the python script that will sleep longer every day.

Sorry for the long post

3

or i need to make something in the python script that will sleep longer every day.

Probably the easiest. cron can not handle variables in the time. The at command is a lot more flexible when it is about the time you need to start a script. You need is a script that executes a

at midnight + 1 minute < {script}

The 1 is a variable you increase based on the day you started (day 1 is +1 min, day 2 is +2 min etc). Mind though you will run into trouble at some point: in 60 * 24 = 1440 days you would have you run past the next midnight. Add that script to your cron.

There is also at -t where you set a time that would make it normal commands where you do not need cron:

at -t 201903300000
at -t 201903310001    
at -t 201903010002
...
at -t 201912310410 

(at 0:00 on 30.3.2019, 1 min past 0:00 on 31.3, 2 mins past 0:00 on 1.4 etc)

  • Best answer ever got in all StackExchange sites, thanks a lot. – djsony90 Mar 31 at 9:43
3

Another possible way to achieve your goal is to use systemd timer units and service units instead of a cronjob.

File /etc/systemd/system/my-script.timer:

[Unit]
Description=Timer for MyScript

[Timer]
OnBootSec=2min
OnUnitInactiveSec=1day 5min

[Install]
WantedBy=multi-user.target

File /etc/systemd/system/my-script.service:

[Unit]
Description=MyScript

[Service]
Type=oneshot
ExecStart=/path/to/my-script.sh

Then run the following commands:

sudo systemctl daemon-reload
sudo systemctl enable --now my-script.timer

This will enable (=autostart at boot time) the timer unit and start it right now.

The timer unit controls the service unit, that is: it starts the service unit of the same name (my-script) 2 minutes after the system has booted and from then on it re-starts the service unit 1 day and 5 minutes after the service unit became inactive (=has stopped).

If the boot time is more than 2 minutes in the past, the timer is activated immediately.

Note that the service unit will run as user root. To change that, add the User= attribute to the service unit:

[Unit]
Description=MyScript

[Service]
Type=oneshot
User=djsony90
ExecStart=/path/to/my-script.sh

To check the status, issue:

systemctl status my-script.{timer,service}
● my-script.timer - Timer for MyScript
   Loaded: loaded (/etc/systemd/system/my-script.timer; enabled; vendor preset: enabled)
   Active: active (waiting) since Sun 2019-03-31 13:47:29 CEST; 19min ago
  Trigger: Mon 2019-04-01 13:52:59 CEST; 23h left

Mar 31 13:47:29 host systemd[1]: Started Timer for MyScript.

● my-script.service - MyScript
   Loaded: loaded (/etc/systemd/system/my-script.service; static; vendor preset: enabled)
   Active: inactive (dead) since Sun 2019-03-31 13:47:59 CEST; 18min ago
  Process: 22833 ExecStart=/home/pduck/my-script.sh (code=exited, status=0/SUCCESS)
 Main PID: 22833 (code=exited, status=0/SUCCESS)

Mar 31 13:47:29 host systemd[1]: Starting MyScript...
Mar 31 13:47:59 host systemd[1]: Started MyScript.

Here we can see that the timer unit is enabled (meaning it will start at boot time) and currently waiting for 19 minutes. The service unit will be activated at Mon 2019-04-01 13:52:59 in approx. 23 hours. The service unit is currently inactive. (My test script, my-script.sh, simply does a sleep 30.) We can also see that the service unit is inactive since Sun 2019-03-31 13:47:59. Adding 1 day and 5 minutes gives exactly Mon 2019-04-01 13:52:59 which is the trigger time of the timer unit.

Further readings:

  • You sure know what you're doing man, I will try this out – djsony90 Mar 31 at 12:33
  • 1
    @djsony90 You are welcome. systemd timer units are more flexible than cronjobs, but -- as you can see here -- they are also more complicated to write. In addition to your script you always need two further files: the service unit that starts the script and the timer unit that starts the service unit. Usually I stick to cronjobs, but when it comes to more complicated things I sometimes use systemd. – PerlDuck Mar 31 at 12:48
  • 1
    I like this one even more than the old at. – Rinzwind Mar 31 at 13:29
  • 1
    @Rinzwind Oh, thank you very much. But as said: it's more complicated. BTW, instead of good old at I sometimes use e.g. systemd-run --on-active=10m /path/to/executable or even systemd-run --on-calendar=23:30 systemctl isolate poweroff.target. See systemd-run. And yes, admittedly I'm a systemd junkie ;-) – PerlDuck Mar 31 at 13:46
1

Rather than using cron, you could simply run a persistent bash script that sleeps for a day and ten seconds after each time it invokes your program:

#/bin/bash

while true
do
    ### invoke your program here ###
    sleep 1d 10s
done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.