0

Here is my crontab command:

*/15 * * * * cd /home/ec2-user/SageMaker && /bin/bash /home/ec2-user/SageMaker/churn_end_to_end_pipeling_actionable.sh

In my mail, here are the errors I'm getting:

Message-Id: <202002290600.01T608DE020009@ip-172-16-124-135.ec2.internal>
X-Authentication-Warning: ip-172-16-124-135.ec2.internal: ec2-user set sender to root using -f
From: root@ip-172-16-124-135.ec2.internal (Cron Daemon)
To: ec2-user@ip-172-16-124-135.ec2.internal
Subject: Cron <ec2-user@ip-172-16-124-135> cd /home/ec2-user/SageMaker && /bin/bash /home/ec2-user/SageMaker/churn_end_to_end_pipeling_actionable.sh
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/ec2-user>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=ec2-user>
X-Cron-Env: <USER=ec2-user>

Traceback (most recent call last):
  File "feature_engineering.py", line 9, in <module>
    import seaborn as sns
ImportError: No module named seaborn
Traceback (most recent call last):
  File "generate_churn_outputs_actionable.py", line 5, in <module>
    from xgboost import XGBClassifier
ImportError: No module named xgboost

I'm not sure why I'm running into this error since I'm have sudo -H pip install <package> before running the cron job.

Also, here are is the content of my bash script and the python scripts that it references:

Bash:

#!/bin/bash

python feature_engineering.py 
python generate_churn_outputs_actionable.py 1

Script 1 and 2:

#! /home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin/python

Any help would be much appreciated. Also, if I'm missing any details, let me know and I'll add it.

4
  • Is python Python v2 or Python v3? A classical error is to use "pip" for the wrong Python version. – xenoid Feb 29 '20 at 18:35
  • It is Python 3.6 – Madhav Thaker Feb 29 '20 at 23:06
  • pip is for V2, try pip3 install ... – xenoid Feb 29 '20 at 23:15
  • I checked and all of my packages are installed on pythong 3.6 – Madhav Thaker Feb 29 '20 at 23:17
0

I had the same issue on Ubuntu 18.04 and using the explicit path in the crontab did not work. For some reason, cron did not work and sudo from my account did work. I would have assumed they would be the same but alas, they were not. I came to the conclusion that it was an environment variable issue.

You can simulate the cron issue in sudo by running the following:

sudo bash -c "HOME=FOO;python3 -c \"import xgboost\""

For me the fix was to have crontab call a shell script which then executes the python script. Also had to set HOME and PYTHONPATH environment variables:

crontab:

# Execute shell script and pipe stdout and stderr to a log file
# Which will enable you to see what's going on in real time
* * * * * <your_path>/yourscript.sh  >> <your_path>yourscript_cron.log 2>&1

yourscript.sh:

#!/bin/bash
echo yourscript.sh called: `date`
HOME=<your_home_dir>
PYTHONPATH=<path_to_dist_packages>
cd <path_to_your_python_script>
<python_executable> ./<your_python_script> 2>&1 1>/dev/null

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.