9

When I run a python script in a terminal it runs as expected; downloads file and saves it in the desired spot.

sudo python script.py    

I've added the python script to the root crontab, but then it runs as it is supposed to except it does not write the file.

$ sudo crontab -l
> * * * * * python /home/test/script.py >> /var/log/test.log 2>&1

Below is a simplified script that still has the problem:

#!/usr/bin/python

scheduleUrl = 'http://test.com/schedule.xml'
schedule = '/var/test/schedule.xml'

# Download url and save as filename
def wget(url, filename):
    import urllib2
    try:
        response = urllib2.urlopen(url)
    except Exception:
        import traceback
        logging.exception('generic exception: ' + traceback.format_exc())
    else:
        print('writing:'+filename+';')
        output = open(filename,'wb')
        output.write(response.read())
        output.close()

# Download the schedule
wget(scheduleUrl, schedule)

I do get the message "writing:name of file;" inside the log, to which the cron entry outputs. But the actual file is nowhere to be found...

The dir /var/test is chmodded to 777 and using whatever user, I am allowed to add and change files as I please.

6
  • It is a rights issue. I figured out that when I have the files I want to write present and I chmod those to something like 666 then everything works fine. So how to get python to be able to write as well? I thought by adding the script to the sudo crontab it would run with some priviledges I'd usually have when sudoing myself...
    – Ruud
    Nov 28, 2011 at 22:07
  • 1
    You can't sudo when you are not there. This command gives you temporary elevated rights, predicated on entry of the required password.
    – david6
    Nov 29, 2011 at 1:52
  • david6, I thought that by adding it using sudo crontab filename.cron that the scripts would be executed as if executed using sudo. So with elevated rights. This is not correct?
    – Ruud
    Nov 29, 2011 at 11:14
  • The command sudo crontab -l will report your current crontab settings. The 'sudo' would NOT effect anything. You would need to try something like using 'cron' (for root user) [Not Recommended] - you should instead be looking at: chroot.
    – david6
    Nov 29, 2011 at 22:43
  • david6; you are wrong. sudo crontab -l and crontab -l display different entries. And so far I'm not getting permission denieds for tasks in the sudo crontab on tasks that do display this message in the current user crontab.
    – Ruud
    Dec 2, 2011 at 7:33

4 Answers 4

5
  • Check log files grep -i cron /var/log/syslog
  • Add an empty line to the end of the crontab, This has been a known bug for ages, not sure if it is solved.
  • Remove the 2>&1 from the command line until it works as designed. Any usefull errors are redirected to a file that is not created; effectively lost.
  • Check if root received mail (eg. using mutt or in /var/spool/mail). Error messages from cron are sent to system email by default.

Also:

  • Reconsider the 777 permissions as soon as possible. When running from root, 755 root:root should be sufficient to be able to check the logs from unprivileged user)
  • Reconsider running the script from root, it is bad practise.
1

What works for me

Crontab

#Borrowed from anacron
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
#End borrowed from anacron

* * * * *   python /home/username/somedir/test.py

Python script

scheduleUrl = 'http://example.com/index.html'
schedule = '/tmp/test.html'

# Download url and save as filename
def wget(url, filename):
    import urllib2
    try:
        response = urllib2.urlopen(url)
    except Exception:
        import logging
        logging.exception('error')
    else:
        print('writing:'+filename+';')
        output = open(filename,'wb')
        output.write(response.read())
        output.close()

# Download the schedule
wget(scheduleUrl, schedule)

Added environment variables. Used tmp instead of var to verify there weren't permissions issues.

1

I had a similar problem:

f = open('./my_file.txt', 'w')
f.close()

Was not opening and writing the file when run from cron. This solved it

f = open('<full_path_of_file>/my_file.txt', 'w')
f.close()
1
  • 1
    Thanks - would have never figured that out! Nov 10, 2021 at 17:01
0

For me the solution was as simple as changing the file access mode. Instead of:

output = open(filename,'wb')

Try:

output = open(filename,'rb+')

I used this to scrape craigslist for job postings (for myself) and to model into a database. All done on a raspberry pi.

I found this and it's very similar to the issue I was having, though I didn't quite get my answer from here. The cron job was executing but the python script would not write files when executed via a cron job. The script would write web scraped text files if executed from the command line.

Solution for me was simply the wb to rb+ rb+ Opens a file for both reading and writing in binary format. The file pointer placed at the beginning of the file.

1
  • Actually, my mistake. I changed to wb+ but the most import think is to use the absolute file path when saving the file. file = open('absolute/path/to/file.txt', 'wb+') Now things are working Mar 6, 2016 at 5:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .