7

I am using Ubuntu 16.04 and systemd to run celery as a daemon. I have created the unit file also but i am not able to run celery as a service. Why is this error happening?

Here are the contents of /etc/systemd/system/celery.service:

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=celery
Group=celery
EnvironmentFile=-/etc/default/celery
WorkingDirectory=/srv/weaver/src
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target

This is the /etc/default/celery file:

ENABLED="true"
CELERYD_NODES="worker1"
#CELERYD_NODES="worker1 worker2 worker3"
CELERY_BIN="/usr/local/bin/celery"
CELERY_APP="main:celery_app"
CELERYD_CHDIR="/srv/weaver/src"
CELERYD_OPTS=" --queue=weaver  --time-limit=100000 --concurrency=2"
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERY_D_LOG_LEVEL="DEBUG"
CELERYD_PID_FILE="/var/run/celery2/%N.pid"
CELERYD_USER="celery"
CELERYD_GROUP="celery"
CELERY_CREATE_DIRS=1
# Change Celery Beat
CELERYBEAT_CHDIR="/srv/weaver/src"
# Log files
CELERYBEAT_LOG_FILE="/var/log/celery/celerybeat.log"
# Celery Beat Log files
CELERYBEAT_PID_FILE="/var/run/celery/celerybeat.pid"
# Scheduler for celery
CELERYBEAT_OPTS=" --pidfile=/var/run/celery/celerybeat.pid  --schedule=/var/run/celery/celerybeat-schedule"

This is the output of the running service:

● celery.service - Celery Service
   Loaded: loaded (/etc/systemd/system/celery.service; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2017-01-12 17:12:32 IST; 2min 17s ago
  Process: 18561 ExecStop=/bin/sh -c ${CELERY_BIN} multi stopwait ${CELERYD_NODES}    --pidfile=${CELERYD_PID_FILE} (code=exited, status=0/SUCCESS)
  Process: 18540 ExecStart=/bin/sh -c ${CELERY_BIN} multi start ${CELERYD_NODES}    -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE}    --logfile=${CELERYD_LOG_FIL
 Main PID: 18555 (code=exited, status=1/FAILURE)

Jan 12 17:12:30 fb01 systemd[1]: Starting Celery Service...
Jan 12 17:12:31 fb01 sh[18540]: celery multi v3.1.23 (Cipater)
Jan 12 17:12:31 fb01 sh[18540]: > Starting nodes...
Jan 12 17:12:31 fb01 sh[18540]:         > worker1@fb01: OK
Jan 12 17:12:31 fb01 systemd[1]: Started Celery Service.
Jan 12 17:12:32 fb01 systemd[1]: celery.service: Main process exited, code=exited, status=1/FAILURE
Jan 12 17:12:32 fb01 sh[18561]: celery multi v3.1.23 (Cipater)
Jan 12 17:12:32 fb01 sh[18561]: > worker1@fb01: DOWN
Jan 12 17:12:32 fb01 systemd[1]: celery.service: Unit entered failed state.
Jan 12 17:12:32 fb01 systemd[1]: celery.service: Failed with result 'exit-code'.
Improve this question

2 Answers 2

Reset to default
1

I've had the same problem not long ago.

  • Using CELERY_CONFIG_MODULE in /etc/default/celery file was helpful for me.
  • Also check if you have the user named celery. You can run celery under another unprivileged user. I launch celery under my own user.

Below is my file /etc/default/celery:

ENABLED="true"
CELERYD_NODES="celery_worker"

# Where to chdir at start.
CELERYD_CHDIR="/path/to/my/project"

CELERYD_OPTS="--time-limit=300 --concurrency=8"

CELERY_CONFIG_MODULE="celeryd"
CELERY_APP="celeryd:celery"

# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"

# Workers should run as an unprivileged user.
CELERYD_USER="my_user_name"
CELERYD_GROUP="my_user_name"

CELERY_CREATE_DIRS=1

CELERY_BIN="/usr/local/bin/celery"

Here is my /etc/systemd/system/celery.service file word by word:

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=user
Group=group
EnvironmentFile=-/etc/default/celeryd
WorkingDirectory=/opt/celery
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target
Improve this answer
2
  • Note that, based on your celery.service script (the ExecStart part) your environment settings such as CELERYD_CHDIR, CELERYD_USER, CELERYD_GROUP may be ignored by celery, and should define directly in .service file.
    – mrdaliri
    Apr 2, 2017 at 12:53
  • @mrdaliri is that because celery ignores environment variables? Or, put another way, celery doesn't support those configuration options as environment variables?
    – ostergaard
    Oct 2, 2018 at 16:56
0

I had this same symptom, turned out to be permissions.

In my case something like:

chmod o+x /srv/weaver/src

sorted it.

Note: That this is not the best way to enable the required permission but that's not pertinent to this answer.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .