Cron doesnt output anything from python script if selenium is run headless

Question

I've got a WEIRD problem, I've been banging my head against this issue for the last 2 weeks, googling around and drinking to wash away the pain of my failures.

So here's the deal, when I run my alarm script from cron, nothing at all is redirected to my log file, and the script doesn't seem to function. HOWEVER, when I comment out a seemingly meaningless option in selenium, the logging works. The script functions perfectly when run outside of cron. I'm using non-sudo cron by the way, as that's where my python environment is set up.

The script basically gets some information and types it into a very nice TTS engine (natural reader) and plays it in the morning. I don't want to pay for a TTS or natural reader which is why I'm using selenium

---

CODE

---

Here is my cron script, I've ommited my other cron entries as they are not important:

@reboot sh scripts/YakScripts.sh -alarm > scripts/logs/alarm.log 2>&1

Here is my YakScripts.sh file, again, I've omitted the meat of it:

#!/usr/bin/env bash

TZ=":America/New_York" date

export PYTHONPATH=${PYTHONPATH}:"scripts"


case "$1" in
  -alarm) python3 scripts/Alarm/alarm.py ;;
  *) echo "Thats not a function, dumbass."
    ;;
esac

Here is my alarm script:

import Alarm.Weather.getDailyForecast as weather
import Alarm.SunCycles.suncycles as suncycles
import TTS.speak as voice

from datetime import date

today = date.today()

# dd/mm/YY
naturalDate = today.strftime("%B %d, %Y")
print(naturalDate)

dayOfWeek = today.strftime("%A")
print(dayOfWeek)


alarmText = "Good Morning. \n"
alarmText += "Today is " + dayOfWeek + ". " + naturalDate + ".\n"



alarmText += weather.getMostRecentForecast() + "\n\n"

alarmText += "Today, the sun rises at " + suncycles.getSunrise() + ".\n"
alarmText += "and sets at " + suncycles.getSunset() + ".\n"


print(alarmText)
voice.speak(alarmText)

And lastly (and the most important bit) here is my speak script, I've removed most of this as I dont think naturalReader would like me posting this. But I tested and it functions the same:

import time

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

import TTS.locators as locators
import globalData

selectedVoice = globalData.tts["voice"]
max_stale_tries = globalData.tts["max_retries"]


def speak(message):
    options = Options()
    #options.add_argument("--headless")
    driver = webdriver.Firefox(options=options)

    driver.get("https://www.naturalreaders.com/online/")

    print("Loaded the web page")
    time.sleep(3)

---

ISSUES

---

So with that, here's the problem, when I don't comment out the headless line in there (line 15 of speak.py) this is what my alarm.log file looks like:

Tue Feb  9 21:08:03 EST 2021

and when I comment out that line, this is what It looks like:

Tue Feb  9 21:27:43 EST 2021
February 10, 2021
Wednesday
Good Morning. 
Today is Wednesday. February 10, 2021.
Tonight the forecast is Snow showers before 10pm, then snow showers and patchy freezing drizzle. Cloudy, with a low around 8. Wind chill values as low as -6. Northeast wind 8 to 12 mph.
Wednesday the forecast is Snow showers before 7am. Cloudy, with a high near 16. Wind chill values as low as -7. Northeast wind 10 to 14 mph, with gusts as high as 18 mph.

Today, the sun rises at 07:29 AM.
and sets at 06:12 PM.

Traceback (most recent call last):
  File "../../../../../../../../scripts/Alarm/alarm.py", line 35, in <module>
    voice.speak(alarmText)
  File "/scripts/TTS/speak.py", line 16, in speak
    driver = webdriver.Firefox(options=options)
  File "/home/appa/.local/lib/python3.8/site-packages/selenium/webdriver/firefox/webdriver.py", line 170, in __init__
    RemoteWebDriver.__init__(
  File "/home/appa/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/appa/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/appa/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/appa/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1

I know this error has to do with the webdriver not being able to connect to the display, which is why I need to be running this headless.

One more thing, I can see both the geckodriver/python3 processes in htop, but again, get no output from the alarm.py script.

I'm really just wondering why in the world I get output when that line is uncommented, and DONT get output when run normally (headless).

Answer 1

Jobs run through cron, crontab, aren't run in the same runtime environment that you have on your desktop. None of your PATH changes, or other environment variable settings are automatically propagated to your cron job. For example, there's no $DISPLAY, so GUI programs need special treatment (read man xhost).

One can set environment variables for all one's cron jobs in the crontab file Read man 5 crontab.

Look at the results of echo "=== set ===";set;echo "=== env ===";env | sort;echo "=== alias ===";alias in each of your environments.

Since the command part of the crontab line is, by default, interpreted by /bin/sh, which has a simpler syntax than /bin/bash, I recommend having command be a call to a bash script (executable, mounted, starts with #!/bin/bash) which sets up the environment, then calls the desired program.