Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

Following on from this question, I have written a simple upstart service (/etc/init/pms.conf) for my headless Ubuntu Server 11.04 box as follows:

start on filesystem and net-device-up IFACE=eth0
stop on runlevel [016]
respawn

exec /home/administrator/pms-current/PMS.sh

I can start (or stop) this service at will from the command line:

service pms start

And I can see that it is indeed running.

However, when I first boot my machine the service does not start. If I SSH into the box and check the service status I get:

$ service pms status
pms stop/waiting

My question is why is this happening? Why isn't my service starting on boot?

UPDATE 1: unsure whether my service was being started and subsequently dying or just wasn't being start at all, I added the following to PMS.sh:

echo "STARTED" > $STARTLOG

This obviously just gives me something to look for. I tested this by starting the service myself and then checking start.log. I then deleted the start.log and rebooted. It wasn't there after the restart, so it seems as though upstart definitely isn't starting my service. I suppose it could be dying at an earlier point in the process, but that seems rather unlikely given the simplicity of it all.

UPDATE 2: I've just upgraded to 11.10 which includes an upstart upgrade, but this problem still occurs.

UPDATE 3: As requested, I've booted with --debug. The output of cat /var/log/syslog | grep init is too long to place in the question, but you view it here.

UPDATE 4: More logs, this time the upstart conf is included at the top. Run 1 and run 2.

share|improve this question
    
Are you sure the script was not executed? The pms stop/waiting message means that the upstart job was run and that all the commands terminated normally. –  arrange Sep 24 '11 at 15:17
    
Add the output of cat /var/log/syslog | grep init after enabling boot loggging for upstart using the instructions at Upstart Debugging –  Anarci Oct 26 '11 at 2:20
    
@Anarci: please see update 3 in my question. –  Kent Boogaart Oct 26 '11 at 12:33
    
Most users won't give out there email address like that, rather provide a link to a pastebin site like Ubuntu pastebin –  Anarci Oct 26 '11 at 12:56
    
@Anarci: done - please see my question. –  Kent Boogaart Oct 26 '11 at 13:00

9 Answers 9

I would recommend increasing the verbosity of the job, e.g. by using pre-start/post-start entries.

pre-start script
  logger "pre-start for myprog"
end script

post-start script
  logger "post-start for myprog"
end script

# and for PMS itself:
script
  exec logger "just before executing PMS"
  exec /home/administrator/pms-current/PMS.sh
end script

Further information at http://upstart.ubuntu.com/cookbook/

Also have a look at http://upstart.ubuntu.com/wiki/Debugging

share|improve this answer
    
This really is doing my head in. I've tried a dozen different things off the back of your post. All have failed with varying obscure messages in the logs. My latest attempt resulted in init: pms main process (1329) terminated with status 143, which means naught to me. I can see that PMS.sh isn't even being started because the first thing it does is write to its own log, and that log entry isn't present. I can see my pre-start output, which tells me that the target file exists and is executable. I will pick this up again tomorrow, but if you have any ideas I'd love to hear them. Thanks. –  Kent Boogaart Oct 24 '11 at 18:50
    
hi @KentBoogaart, I seem to have your same problem. Have you found a solution? –  Daniele B May 22 at 19:50

What's probably happening here is that pms is starting before your network adapters come up, and probably before even the loopback adapter (lo). Assuming we're talking about PS3 Media Server, it's a networked service and it probably doesn't like starting up with no interfaces available.

Try changing your start on criteria to:

start on filesystem and net-device-up IFACE!=lo

Meaning, start after any "real" network interface is up. However, that might not be ideal, if eth0 is the next interface up, PMS starts, but you really want PMS to use wlan0, that won't do. The service will start but it might not have been able to pick the interface you wanted it to listen on. Assuming you know the interface you're going to stream over and it won't be changing, I would hardcode it into the job, e.g.:

start on filesystem and net-device-up IFACE=wlan0

On Oneiric (11.10), you can use the event static-network-up to wait for all statically configured devices. Which is nice because it allows you to write network-dependent jobs without hardcoding an interface. [Note: by "all statically configured devices", I'm referring to using /etc/network/interfaces instead of NetworkManager. It does not mean static in the sense of static IP vs. DHCP.]

share|improve this answer
    
This sounded like the trick, but it didn't work. I only have lo and eth0 but I used your second suggestion: start on filesystem and net-device-up IFACE=eth0. Still no go after a reboot. I've just noticed something in the PMS log that may be a lead. I'll investigate and get back... –  Kent Boogaart Sep 24 '11 at 17:29
    
It's interesting. One thing I didn't mention is that I tried your original script and it worked on boot for my machine. I attributed that to just the luck of the draw (i.e. in my race condition, the good car won, and in yours the bad car won). I really can't see what other dependency we're missing here. Weird. –  Mark Russell Sep 24 '11 at 17:49
1  
Since you can start it after booting up, we must be missing another service dependency. One dirty hack that might work (but won't illuminate us at all) is to just drop in a sleep 10 -- or higher -- in a "pre-start script" before exec'ing the shell script. –  Mark Russell Sep 24 '11 at 17:51
    
Sorry Mark - but we're on the same page. I tried the sleep 10 thing already in a pre-start script. No go. Then I tried deleting the debug.log altogether and rebooting. After boot I had the same service status, and no debug.log file, so I'm not convinced PMS is actually run at all. Is there an easy way to diagnose this? If I alter the PMS.sh to spit out some output, where will it go? I suppose I could always direct it to my own file - might give that a shot next. –  Kent Boogaart Sep 24 '11 at 17:57
    
I just updated my question with more info. –  Kent Boogaart Sep 24 '11 at 18:13

From examining your syslog the pms process starts with no errors but then after a short while its goal is changed from start to stop meaning it is killed.

This is slightly strange because you have added the repsawn clause so it should attempt to start again after it is stopped but it never does. So I'm guessing you removed the respawn clause.

Between the pms service starting and stopping only 2 services are started ufw and network-interface (eth0), and 1 is started udev-fallback-graphics.

It seems that you process pms is being started in parallel. Unfortunately the upstart documentation is a little bit hazy on the exact differences between start on ... vanilla and start on starting ... and start on started ....

Try changing your startup stanza to

start on started networking

or just too

start on net-device-up IFACE=eth0

The log output is slightly strange as the net-device-up event comes much later but pms starts before it.

This should ensure that your process only starts once all networking set up is finished i.e. the job has not only started but finished.

Also do not trust log output completely, early in the boot process logging output to any file does not always work. See the answer in Debugging Upstart

share|improve this answer

I came across chkconfig during my RHCSA/CE training:

sudo apt-get install chkconfig
sudo chkconfig pms on

You can check it's Oneiric man page for more details on it's capabilities.

share|improve this answer

I've found a solution for this but I don't understand it. If I move PMS out of /home/administrator and into /bin/pms with root as the owner, it all works fine.

If I leave it under /home/administrator/ but make sure root is the owner of everything bar the /home/administrator/ directory itself, it still doesn't work.

If I set administrator as owner of everything and change the pertinent part of my script to:

sudo su administrator -c '/home/administrator/pms-current/PMS.sh'

It still doesn't work.

I suppose for now I'll make a /home/root/ directory and move everything there, though I'd really like to fully understand this.

share|improve this answer
    
So the chkconfig didn't work either? Did you try giving the directory PMS.sh was in to root? If only your solution works, then go to the Upstart's Launchpad page and contact the devs directly. –  Oxwivi Oct 26 '11 at 17:08
    
And if it's enough to just move the .sh then just leave everything there and edit the script to point to that directory (or maybe even change directory?). –  Oxwivi Oct 26 '11 at 17:10
    
Yes, I tried making the whole PMS directory owned by root. Presumably it didn't work because /home/administrator/ is not owned by root. –  Kent Boogaart Oct 26 '11 at 17:18
    
This doesn't make sense anyway, I regularly run scripts in my /home directory via upstart without problems, strange. –  arrange Oct 26 '11 at 17:26
    
Even stranger: I just tried everything under /home/root/ which is obviously owned by root. Didn't work. I moved everything back under /bin/pms and it worked again. So it seems trying to upstart PMS from under /home isn't working on my system. –  Kent Boogaart Oct 26 '11 at 17:49

Is your home directory on NFS? Sometimes root can't access NFS.

For the record, in my little test just now on 12.04:

  • start on started networking and start on network-interface-up INTERFACE=eth0 don't work, but

  • start on started network-interface INTERFACE=eth0 does.

Thanks to http://os4.org/wiki/upstart.html for pointing out that initctl list always shows job networking as stopped.

share|improve this answer

Managed to fix similar problem by using start on runlevel instead:

start on runlevel [2345]
share|improve this answer

I had a similar "no start" problem when I have realized that my script depended on a file that was in my home, and the home was not accessible because was crypted with the standard ubuntu mechanism ( .Private).

start on local-filesystems event is (probably) emitted before the decryption process is ended.

share|improve this answer

I had the same problem and eventually I solved it simply with:

start on runlevel [2345]

without any net-device-up or started networking stuff

This is the complete upstart script, and it works perfectly:

# MyApp

description     "MyApp"
author          "me"

start on runlevel [2345]
stop on runlevel [016]

respawn

exec /usr/bin/myapp 2>> /var/logs/myapp.log
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.