What made “smbd” stop running on boot-up?

Question

Recently I was faced with a weird problem on my samba server (Ubuntu 10.04 LTS am64) when all shares were "suddenly" inaccessible to the clients. Tracking down the error revealed that smbd was not started on boot-up. From the syslog:

[...]
Dec  4 12:28:45 takkat-home dhclient: 
Dec  4 12:28:45 takkat-home NetworkManager: <info>  DHCP: device eth0 state changed (null) -> preinit
Dec  4 12:28:45 takkat-home init: smbd main process (1027) terminated with status 255
Dec  4 12:28:45 takkat-home init: smbd main process ended, respawning
Dec  4 12:28:45 takkat-home dhclient: Listening on LPF/eth0/00:24:1d:25:40:e4
Dec  4 12:28:45 takkat-home dhclient: Sending on   LPF/eth0/00:24:1d:25:40:e4
Dec  4 12:28:45 takkat-home dhclient: Sending on   Socket/fallback
Dec  4 12:28:45 takkat-home avahi-daemon[917]: No service file found in /etc/avahi/services.
Dec  4 12:28:45 takkat-home init: smbd main process (1045) terminated with status 255
Dec  4 12:28:45 takkat-home init: smbd main process ended, respawning
Dec  4 12:28:45 takkat-home init: smbd main process (1050) terminated with status 255
Dec  4 12:28:45 takkat-home init: smbd main process ended, respawning
[...]

and smbd.log:

[...]
[2010/12/04 12:28:45,  0] smbd/server.c:1069(main)
  smbd version 3.4.7 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2009
[2010/12/04 12:28:45,  0] printing/print_cups.c:103(cups_connect)
  Unable to connect to CUPS server localhost:631 - Connection refused
[2010/12/04 12:28:45,  0] printing/print_cups.c:103(cups_connect)
  Unable to connect to CUPS server localhost:631 - Connection refused
[2010/12/04 12:28:45,  0] smbd/server.c:1115(main)
  standard input is not a socket, assuming -D option
[2010/12/04 12:28:45,  0] passdb/pdb_interface.c:253(guest_user_info)
  guest_user_info: Unable to locate guest account [guest]!
[2010/12/04 12:28:45,  0] smbd/server.c:1201(main)
  ERROR: failed to setup guest info.
[2010/12/04 12:28:45,  0] smbd/server.c:1069(main)
  smbd version 3.4.7 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2009
[...]

it was clear that smbd crashed.

Before that I did not change anything to the samba configuration, did not install anything but security updates and did not create or delete any new users. Today, I got everything repaired and running smooth as usual by simply running

sudo smbd restart

The crashes on boot are gone and shares are available to clients again.

Still, I have no clue what process was responsible for this behaviour and I am worried on how to avoid this in the future. Any ideas of what process could be made responsible for smbd crashing would be much appreciated. Also I'd be interested to hear what restarting smbd could possibly have repaired and if this was a good idea to do at all.

Answer 1

If indeed it is a timing issue, it probably means something (I'm guessing a network service) which is required pre smbd startup isn't happening in time. The fact you can start it no problem after startup means you can probably just set it at a really low startup priority (i.e right at the end as it doesn't appear to be a dependency of any other service).

You do this by using the update-rc.d script.

For a bit of background, checkout /etc/rc2.d (or any of /etc/rc2.d through /etc/rc5.d) which are the services that are set to automatically start when the system enters run level 2 - 5 (for this exercise let's assume theyre the same). You'll probably notice that there's a S20smbd which means auto start smbd (links to startup script /etc/init.d/smbd) with a priority of 20, where things with a low priority are started first until everything in the run level directory has been processed.

Typically a user level startup would use a low priority (starts last) of 99, so from the update-rc.d docs, we'll use 98:

• update-rc.d smbd defaults 98 02

This simply means update the auto start for smbd for default run levels (2-5) to priority 98, so its pretty much going to be the last thing that starts (dependant on how your system is set up).

Answer 2

I have had a quick look around the web and it looks like either a permissions problem or timing. For instance I run docky and usually it pops up with a notification after login saying that compiz is not enabled and some of the functions wont be enabled however by the time it actually appears compiz has had time to start and docky is fine. Docky has the ability to fall back to a more basic form if compiz hasn't started yet. However AWN (avant window navigator) doesn't do this and will just not load if compiz isn't present even if it manages to get it lazy butt in gear at a later "date". Although all my ramblings are about visual gui elements the theory should apply to service programs as well. One possible solution is a script to check for the successful running of the smbd service and if its not running attempt to start it again.

Hope this helps or at least gives you an idea.

Answer 3

There's a further problem on Ubuntu 11.04 (natty, amd64). From the bootlog:

Rather than invoking init scripts through /etc/init.d, use the service(8) utility, e.g. service S20smbd start

Since the script you are attempting to invoke has been converted to an Upstart job, you may also use the start(8) utility, e.g. start S20smbd start: Unknown job: S20smbd

So, it seems smbd tries to be started as "S20smbd", thus it's not recognized by "service". Again, restarting smbd once the system is up fixes it.

I cannot figure out where the problem is in the upstart scripts. Any help would be appreciated, thanks.

Answer 4

I was having the same problem. I figured out the solution while reading a howto on installing sendmail on my company server.

Install the packages sysv-rc-conf and sysvinit-utils by typing the following in a terminal.

sudo apt-get install sysv-rc-conf sysvinit-utils

Then you type this to configure your system (Ubuntu 10.04) to load nmbd on boot:

sudo sysv-rc-conf nmbd on

Answer 5

I think this issue is due to a race between dhclient, which brings up the network interface asynchronously, and other services which need the network interface.

I've had Samba sporadically not starting on my Debian system for a long time, and now essentially the same happened with mysqld (needed by the MythTV back end), which finally motivated me to dig deeper into it.

If the DHCPACK arrives in time and dhclient brings up the interface before Samba / mysql start up everything is OK, otherwise the services fail to start:

mysqld: 120520 11:38:02 [ERROR] Can't start server: Bind on TCP/IP port: Cannot assign requested address

May 20 12:50:38 piano smbd[1763]:   WARNING: no network interfaces found 

I've fixed it by the ugly hack of adding a 15 second delay after the start of network-manager in its init script, and moving the two init scripts after network-manager in the boot order.

A better way would be to add a proper dependency for the network interface, but that doesn't seem to be possible with sysvinit.

I considered other solutions too risky (e.g. using upstart).