67

For some reason I'm getting an error during apport upgrades, the cause of which is

% sudo service apport start
start: Job failed to start

Under sysvinit, I could debug this kind of problem by running eg

sudo sh -x /etc/init.d/whatever start

but that doesn't seem to map over to Upstart. What should I try next?

It turns out there is a workaround that will let the install proceed. But I'm still interested in the general question of how one would trace the script.

  • 1
    I'm working on a more thorough reply, but thanks for the link to the workaround, same thing happened to me updating natty today! – Mark Russell Apr 21 '11 at 0:38
  • Haha, and they released the fix 12 minutes ago. Isn't running Beta fun? (no really!) – Mark Russell Apr 21 '11 at 0:55
  • +Muhammad posted the correct answer below. – Travis Reeder Feb 5 '14 at 19:06
  • I found useful debugging info in /var/log/syslog. – Matt Mar 15 '16 at 2:46
29

All of the following information (and quite a lot more useful Upstart help) is from The Upstart Cookbook. Section 18 covers debugging. http://upstart.ubuntu.com/cookbook/#debugging

In this specific case of tracing a "script" stanza of an Upstart job you should add the following lines right below the word "script":

exec 2>>/dev/.initramfs/myjob.log
set -x

The reason for the odd location is that /dev/.initramfs/ is available in very early boot, before the root file system has been loaded, and continues to be available after boot. I'm guessing with apport, however, you probably don't need to use that path. Still, it's nice to know the option.

It should also be noted that all scripts are run with set -e so any command that fails will exit the script entirely. Which makes sense, as one should be very careful when running scripts as root.

I highly recommend consulting the Upstart Cookbook linked above in general for anyone working with Upstart jobs.

  • Thanks for that, if I hadn't seen the bug was resolved, I probably would have modified the script to turn on set -x as you suggest. I was mostly wondering if there was any cleaner way to do it. – poolie Apr 21 '11 at 0:55
  • Yeah, not that I know of (and if it's not in the cookbook, that means there probably just isn't). Aside from that idea, most of the debugging techniques mentioned are more about troubleshooting Upstart and Upstart events themselves, which is where I usually find myself. :) – Mark Russell Apr 21 '11 at 1:02
  • And yet this method will fail if setuid and setgid are used in jobs. – 0xC0000022L Jul 27 '15 at 7:27
  • /dev/.initramfs is not always present. I have just come across a machine which doesn't have this path. – Sarge Borsch May 9 '16 at 17:53
110

Upstart logs your service execution in a log file by the same name in /var/log/upstart/your-service-name.log. It should be helpful.

  • 2
    Right, it looks like this includes the stderr of the jobs, so if I turned on set -x it would probably show up there. – poolie Sep 3 '12 at 1:34
  • 1
    I put in an echo based log statement in my /etc/init.d/Xservice script but it doesn't show up in this location! OS: Ubuntu 14.04 Trusty – Champ Aug 6 '15 at 20:39
  • Why not to syslog? May I suggest a general feature, maybe rsyslog itself, that upstart and any component would use to write to syslog if it's not available, or to a temporary place that would be transfered to syslog asap. It took me half a day to find this. Please forward this remark TIA. – Papou Jan 9 '16 at 18:07
  • 4
    This seems no longer to be true in 16.04, apparently since 15.10 at least: the last files in that directory are dated from right before the 15.10 release. – FGM Apr 29 '16 at 12:41
  • 1
    @FGM this is because 16.04 no longer uses upstart, it uses systemd. Logs are accessed with journalctl – Jeremy Nov 13 '16 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.