Systemd doesn't restart monit after kill

Question

I enabled systemd to load monit at startup

sudo systemctl enable monit
sudo systemctl restart monit
ps aux | grep monit
root      6843  0.0  0.2 112492  2948 ?        Sl   18:19   0:00 /usr/bin/monit -c /etc/monit/monitrc
lookpla+  6857  0.0  0.0  14212  1008 pts/0    S+   18:21   0:00 grep --color=auto monit

But when I kill monit it doesn't start again:

 sudo kill 6843
 ps aux | grep monit
 lookpla+  6862  0.0  0.1  14212  1028 pts/0    S+   18:23   0:00 grep --color=auto monit

Systemctl shows that monit is active but exited

sudo systemctl status monit.service

● monit.service - LSB: service and resource monitoring daemon
   Loaded: loaded (/etc/init.d/monit; generated; vendor preset: enabled)
  Drop-In: /etc/systemd/system/monit.service.d
           └─override.conf
   Active: active (exited) since Wed 2017-09-06 18:19:17 UTC; 4min 29s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 6830 ExecStop=/etc/init.d/monit stop (code=exited, status=0/SUCCESS)
  Process: 6835 ExecStart=/etc/init.d/monit start (code=exited, status=0/SUCCESS)
 Main PID: 6843 (code=exited, status=0/SUCCESS)

Sep 06 18:19:17 localhost systemd[1]: Stopped LSB: service and resource monitoring daemon.
Sep 06 18:19:17 localhost systemd[1]: Starting LSB: service and resource monitoring daemon...
Sep 06 18:19:17 localhost monit[6835]:  * Starting daemon monitor monit
Sep 06 18:19:17 localhost monit[6835]:    ...done.
Sep 06 18:19:17 localhost systemd[1]: monit.service: PID file /run/monit.pid not readable (yet?) after start: No such file or directory
Sep 06 18:19:17 localhost systemd[1]: Started LSB: service and resource monitoring daemon.

Could you help me to set up systemd so it will restart monit after it has been killed?

I use ubuntu 17.04

Answer 1

I believe the addition of the Restart option to your systemd service file would ensure it gets start when not stopped by the systemctl stop command:

Restart=[no | on-success | on-failure | on-abnormal | on-watchdog | on-abort | always ]

Info:

no: the default, the service will not be restarted

on-success: will be restarted only when the service process exits cleanly

on-failure: will be restarted when the process exits with a non-zero exit code

on-abort: will be restarted only if the service process exits due to an uncaught signal not specified as a clean exit status

on-watchdog: will be restarted only if the watchdog timeout for the service expires

always: will be restarted regardless of whether it exited cleanly or not, got terminated abnormally by a signal, or hit a timeout

Restart= Configures whether the service shall be restarted when the service process exits, is killed, or a timeout is reached. The service process may be the main service process, but it may also be one of the processes specified with ExecStartPre=, ExecStartPost=, ExecStop=, ExecStopPost=, or ExecReload=. When the death of the process is a result of systemd operation (e.g. service stop or restart), the service will not be restarted. Timeouts include missing the watchdog "keep-alive ping" deadline and a service start, reload, and stop operation timeouts.

Takes one of no, on-success, on-failure, on-abnormal, on-watchdog, on-abort, or always. If set to no (the default), the service will not be restarted. If set to on-success, it will be restarted only when the service process exits cleanly. In this context, a clean exit means an exit code of 0, or one of the signals SIGHUP, SIGINT, SIGTERM or SIGPIPE, and additionally, exit statuses and signals specified in SuccessExitStatus=. If set to on-failure, the service will be restarted when the process exits with a non-zero exit code, is terminated by a signal (including on core dump, but excluding the aforementioned four signals), when an operation (such as service reload) times out, and when the configured watchdog timeout is triggered. If set to on-abnormal, the service will be restarted when the process is terminated by a signal (including on core dump, excluding the aforementioned four signals), when an operation times out, or when the watchdog timeout is triggered. If set to on-abort, the service will be restarted only if the service process exits due to an uncaught signal not specified as a clean exit status. If set to on-watchdog, the service will be restarted only if the watchdog timeout for the service expires. If set to always, the service will be restarted regardless of whether it exited cleanly or not, got terminated abnormally by a signal, or hit a timeout.

Image source

Source:

https://www.freedesktop.org/software/systemd/man/systemd.service.html

Answer 2

RemainAfterExit configuration option did the trick

After hours of experimenting with systemd configuration I found out the solution which works:

sudo systemctl edit monit

Then fill in the following configuration:

[Service]
PIDFile=/var/run/monit.pid
Restart=always
RemainAfterExit=no

And reload the daemon:

sudo systemctl daemon-reload
sudo systemctl restart monit

Now systemd restarts my monit service when I kill it:

ps aux | grep monit
root      7822  0.0  0.2 112476  2872 ?        Sl   19:09   0:00 /usr/bin/monit -c /etc/monit/monitrc
lookpla+  7855  0.0  0.0  14212  1008 pts/0    S+   19:12   0:00 grep --color=auto monit

sudo kill 7822
root      7872  0.0  0.2 112476  2964 ?        Sl   19:13   0:00 /usr/bin/monit -c /etc/monit/monitrc
lookpla+  7875  0.0  0.0  14212   984 pts/0    S+   19:13   0:00 grep --color=auto monit