Airflow scheduler server is not working

Question

I run airflow scheduler command, it is working. However, I am not able to set up airflow scheduler service.

Following is my airflow scheduler service code.

[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service rabbitmq-server.service
Wants=postgresql.service rabbitmq-server.service

[Service]
EnvironmentFile=/home/ubuntu/airflow
User=ubuntu
Group=ubuntu
Type=simple
ExecStart=/bin/airflow scheduler -n ${SCHEDULER_RUNS}
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

However, it is not running, when i start airflow-scheduler. I get following error:

May 25 21:11:37 ip-11-210-11-108 systemd[1]: airflow-scheduler.service: Service hold-off time over, scheduling restart.
May 25 21:11:37 ip-11-210-11-108 systemd[1]: Stopped Airflow scheduler daemon.
-- Subject: Unit airflow-scheduler.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit airflow-scheduler.service has finished shutting down.
May 25 21:11:37 ip-11-210-11-108 systemd[1]: airflow-scheduler.service: Failed to load environment files: No such file or directory
May 25 21:11:37 ip-11-210-11-108 systemd[1]: airflow-scheduler.service: Failed to run 'start' task: No such file or directory
May 25 21:11:37 ip-11-210-11-108 systemd[1]: Failed to start Airflow scheduler daemon.
-- Subject: Unit airflow-scheduler.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit airflow-scheduler.service has failed.
--
-- The result is failed.
May 25 21:11:37 ip-11-210-11-108 systemd[1]: airflow-scheduler.service: Unit entered failed state.
May 25 21:11:37 ip-11-210-11-108 systemd[1]: airflow-scheduler.service: Failed with result 'resources'.

I don't have any experience of systemd. What is SCHEDULER_RUNS? How to set it up ?

Answer 1

In your systemd file, there is this line

EnvironmentFile=/home/ubuntu/airflow

That is supposed to point to an EnvironmentFile, which is used by systemd to hold some variables. It looks like you have it pointing to the airflow home directory.

It should point to a file called airflow, inside which there should be a variable for the SCHEDULER RUNS. See the airflow file here: https://github.com/apache/incubator-airflow/tree/master/scripts/systemd

You also have this error: Failed to run 'start' task: No such file or directory

This relates to this line: ExecStart=/bin/airflow scheduler -n ${SCHEDULER_RUNS}

That line is saying to run /bin/airflow , which I am assuming doesn't exist on your machine. It is probably in /usr/local/bin/airflow or somewhere like that. You can run which airflow to see your actual installed location. You might be able to add a symlink for it so /bin/airflow points to your actual location, but no need to over-complicate it, at least get it working first. Just checked, apparently symlinks do indeed work in systemd https://bugzilla.redhat.com/show_bug.cgi?id=955379

Answer 2

I know I'm digging up a dated post, but I too was trying to figure out why I could not get the scheduler to run automatically when the server is running.

I did find a solution that works for me on Ubuntu 18.04 and 18.10

I provided a full write-up of how to install Airflow and PostgreSQL on the back-end on the link here.

from the later part of my article:

Essentially it comes down to making a specific change to the airflow-scheduler.system file.

This is one of the ‘gotchas’ for an implementation on Ubuntu. The dev team that created Airflow designed it to run on a different distribution of linux and therefore there is a small (but critical) change that needs to be made so that Airflow will automatically run when the server is on. The default systemd service files initially look like this:

[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
EnvironmentFile=/etc/sysconfig/airflow
User=airflow
Group=airflow
Type=simple
ExecStart=/bin/airflow scheduler
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

However, this will not work as the ‘EnvironmentFile’ protocol doesn’t fly on Ubuntu 18. Instead, comment out that line and add in :

Environment="PATH=/home/ubuntu/anaconda3/envs/airflow/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

You will likely want to create a systemd service file at least for the Airflow Scheduler and also probably the Webserver if you want the UI to launch automatically as well. Indeed we do want both in this implementation, so we will be creating two files, airflow-scheduler.service & airflow-webserver.service. Both of which will be copied to the /etc/systemd/system folder. These are as follows:

---

airflow-scheduler.service

[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
#EnvironmentFile=/etc/default/airflow
Environment="PATH=/home/ubuntu/anaconda3/envs/airflow/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
User=airflow
Group=airflow
Type=simple
ExecStart=/home/ubuntu/anaconda3/envs/airflow/bin/airflow scheduler
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target
#airflow-webserver.service

---

airflow-webserver.service

[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
#EnvironmentFile=/etc/default/airflow
Environment="PATH=/home/ubuntu/anaconda3/envs/airflow/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
User=airflow
Group=airflow
Type=simple
ExecStart=/home/ubuntu/anaconda3/envs/airflow/bin/airflow webserver -p 8085 --pid /home/ubuntu/airflow/airflow-webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Finally, with both of those files copied to the /etc/systemd/systemd folder by way of a superuser copy command sudo cp it is time to hit the ignition:

sudo systemctl enable airflow-scheduler
sudo systemctl start airflow-scheduler
sudo systemctl enable airflow-webserver
sudo systemctl start airflow-webserver