Add manual "wait" in upstart script

Question

I have an upstart script like below:

start on stopped hw_boot
script
/usr/bin/python_program
end script

This works fine. But to optimize, I would like to hide the delay of slow python's library imports (Currently, it takes 5-10 sec). One obvious way is to start the python's library imports even during hw boot.

So, what I want to do is something like below:

start on some_boot_event (this event is earlier than stopped hw_boot)
script
    //import bunch of python libraries that are needed later
    wait for "stopped hw_boot"
    /usr/bin/python_program
end_script

How can we write a logic to wait for "stopped hw_boot" in the upstart script?

Answer 1

I won't go into detail and line out entire Upstart scripts for you since you appear to know how to write them already but I'm going to show you the crucial parts.

From your question I'll use

• some_boot_event as the "earlier" boot event that's supposed to trigger the initialisation of the Python program and
• stopped hw_boot as the even that's supposed to trigger the execution of the Python program.

This is how it can work:

1. Refactor your Python program to include a callable entry point that can be called later at an arbitrary time from other modules. (Ideally your Python module/program is already written that way.) If the program currently executes any non-initialisation task at module load time, i. e. at global scope, you should wrap them inside a method.

   E. g. if your module currently looks like this,

   #!shebang
   import foo, bar
   # ... various constant, class and method definitions ...
   print("Hello World")
   

   you should refactor it to this:

   #!shebang
   import foo, bar
   # ... various constant, class and method definitions ...
   
   def main():
       print("Hello World!")
   
   if __name__ == "__main__":
       main()
   

2. Write a Python module that imports the main module of your Python program, waits for a signal and then invokes the main method of the main module:

   #!/usr/bin/env python3
   import signal, MyMainModule
   # Perform other initialisation tasks if necessary
   signal.sigwaitinfo((signal.SIGCONT,))
   MyMainModule.main()
   

   If you can’t use Python 3 you can refer to Python 3's Signal.sigwaitinfo equivalent in Python 2.7? for something equivalent in Python 2.

3. Start an Upstart “service” task with the previous Python program at some_boot_event. Let’s call it my_service_task.

4. Start a second Upstart “one-shot” task at stopped hw_boot that sends a CONT signal to the previous task:

   set -e
   kill -s CONT -- "$(initctl status my_service_task | grep -oEe '[0-9]+$')"
   

If you need to report status information from my_service_task to the task in step 4 you can set up a FIFO before sending the signal in the latter:

#!/usr/bin/env python3
import errno, signal, MyMainModule
# Perform other initialisation tasks if necessary

signal.sigwaitinfo((signal.SIGCONT,))
try:
    return_value = MyMainModule.main()
except Exception as ex:
    return_value = ex
try:
    with open("/var/run/my_service_task.status") as status_fifo:
        print(return_value, file=status_fifo)
except OSError as ex:
    if ex.errno not in (errno.ENOENT, errno.EPIPE):
        raise ex
if isinstance(return_value, Exception):
    raise return_value

On the reading end:

set -e
STATUS_FIFO=/var/run/my_service_task.status
mkfifo -m 0600 "$STATUS_FIFO"
trap 'rm -f "$STATUS_FIFO"' 0 INT QUIT TERM

kill -s CONT -- "$(initctl status my_service_task | grep -oEe '[0-9]+$')"

read return_value < "$STATUS_FIFO"
# Do stuff with $return_value