1

You can start a nodemanager instance by running yarn-daemon.sh start nodemanager. If you make an upstart job out if this, you have two problems:

  1. The yarn-daemon.sh script will be executed constantly since respawn checks if the process died, and the yarn-daemon.sh script dies after starting yarn.
  2. service nodemanager stop command does nothing because upstart thinks the process is already stopped.

How can I make an upstart service that restarts yarn (or another hadoop daemon) after a crash?

1

The code below shows the upstart service configuration file /etc/init/nodemanager.conf. You execute the yarn-daemon.sh start nodemanager script as pre-start hook, and yarn-daemon.sh stop nodemanager as post-stop hook. This starts the actual nodemanager instance.

The script checks if nodemanager is up. If nodemanager is down, the script exits. This signals upstart that the service has gone down and has to be restarted.

description "nodemanager"

start on startup
stop on shutdown

setuid hduser

respawn

# actually start nodemanager
pre-start exec opt/hadoop/current/sbin/yarn-daemon.sh start nodemanager

#
# upstart executes this script. If this script exits, upstart respawns the service
# cannot just excecute *-daemon.sh here, because the actual daemon script excecutes after start
#   and upstart thinks service is stopped so respawns *-daemon.sh constantly.
#
script
  while jps | grep -q NodeManager; do
    sleep 5
  done
end script

post-stop exec /opt/hadoop/current/sbin/yarn-daemon.sh stop nodemanager

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.