How to keep processes running after ending ssh session?

Question

Let's say I launch a bunch of processes from a ssh session. Is it possible to terminate the ssh session while keeping those processes running on the remote machine?

Answer 1

You should look for modern alternatives like tmux.

tmux is superior to screen for many reasons, here are just some examples:

• Windows can be moved between session and even linked to multiple sessions
• Windows can be split horizontally and vertically into panes
• Support for UTF-8 and 256 colour terminals
• Sessions can be controlled from the shell without the need to enter a session

Basic Functionality

To get the same functionality as explained in the answer recommending screen, you would need to do the following:

• ssh into the remote machine
• start tmux by typing tmux into the shell
• start the process you want inside the started tmux session
• leave/detach the tmux session by typing Ctrl+b and then d

You can now safely log off from the remote machine, your process will keep running inside tmux. When you come back again and want to check the status of your process you can use tmux attach to attach to your tmux session.

If you want to have multiple sessions running side-by-side, you should name each session using Ctrl+b and $. You can get a list of the currently running sessions using tmux list-sessions or simply tmux ls, now attach to a running session with command tmux attach-session -t <session-name>.

tmux can do much more advanced things than handle a single window in a single session. For more information have a look in man tmux or the tmux GitHub page. In particular, here's an FAQ about the main differences between screen and tmux.

Answer 2

Option 1: nohup

The best way is often the simplest.

nohup long-running-command &

It was made specifically for this, it even logs stdout to nohup.log.

man nohup

Option 2: bg + disown

Ctrl+Z followed by:

bg
disown -h

If you want to "background" already running tasks, then Ctrl+Z then run bg to put your most recent suspended task to background, allowing it to continue running. disown will keep the process running after you log out. The -h flag prevents hangup.

---

screen and others can do it, but that's not what they're for. I recommend nohup for tasks you know you are going to leave behind and bg for tasks you're already running and don't want to re-start.

Keep in mind, both are bash specific. If you're not using bash, then the commands could be different.

Answer 3

You could do that by using screen.

Type man screen to find out more or read this screen man page.

Simple scenario:

• ssh into your remote box. Type screen. Then start the process you want.

• Press Ctrl+A, then Ctrl+D. This will "detach" your screen session, but leave your processes running. You can now log out of the remote box.

• If you want to come back later, log on again and type screen -r This will "resume" your screen session and you can see the output of your process.

Answer 4

Screen and nohup is the better way, but if you have to detach a process already running without screen or nohup you can run disown command.

disown [-ar] [-h] [jobspec… |pid… ]

Without options, remove each jobspec from the table of active jobs. If the -h option is given, the job is not removed from the table, but is marked so that SIGHUP is not sent to the job if the shell receives a SIGHUP. If jobspec is not present, and neither the -a nor the -r option is supplied, the current job is used. If no jobspec is supplied, the -a option means to remove or mark all jobs; the -r option without a jobspec argument restricts operation to running jobs.

With disown you can close the terminal and get the process running on the machine.

Answer 5

I was stuck in a large mv so I wasn't in a position to stop the process, setup screen and then start it again. I managed to exit the SSH session with the process running by essentially doing the following steps:

1. Establish SSH connection: ssh user@host
2. Run the desired command to start the process
3. Press Ctrl+Z to pause the process
4. Run bg to put the paused process in the background and resume it.
5. Run disown [pid] (process ID is optional, defaults to last process) to disown the process. To get a list of jobs simply type jobs before.
6. Exit the SSH session by running logout.

Usage of the disown command:

disown [-ar] [-h] [jobspec ... | pid ... ]
              Without  options,  remove  each jobspec from the table of active
              jobs.  If jobspec is not present, and neither the -a nor the  -r
              option  is  supplied, the current job is used.  If the -h option
              is given, each jobspec is not removed from  the  table,  but  is
              marked  so  that  SIGHUP  is  not  sent  to the job if the shell
              receives a SIGHUP.  If no jobspec is  supplied,  the  -a  option
              means  to  remove or mark all jobs; the -r option without a job‐
              spec argument restricts operation to running jobs.   The  return
              value is 0 unless a jobspec does not specify a valid job.

Answer 6

There are two major programs you can use to maintain programs and terminal state over multiple ssh connections. They are screen (the incumbent, but unfortunately unmaintained. Apparently being actively developed now) and tmux (newer, actively maintained). Byobu is a front end that can run on top of their of these systems and offer additional ubuntu status information. On new installations it will use tmux as a backend, if you have an older installation of byobu and an existing config it will maintain the previous backend, be it screen or tmux.

Byobu

Byobu can be installed on the computer by doing so in a Debian-based machine:

sudo aptitude install byobu

Using yum, you do

su -c 'yum install byobu'

It's also possible to install byobu on other distributions.

Using byobu

You can start byobu by running byobu on the host machine after connecting using ssh. This will give you a shell that looks like this:

You can also use Byobu Terminal on a Ubuntu machine with -X option and easily have a perfectly working byobu.

Usage:

Start byobu by typing byobu.

You can press F2 to create a new window within the current session, F3-F4 to switch between the various windows.

The best part about byobu is, you dont have to actually kill the processes running in the terminal to leave the terminal. You can simply send screen/tmux (the skeleton of byobu) to background and resume the next time you come:

• To leave byobu and keeep it running (detach) press F6.

• The next time you come, just do byobu and you sholud be back right where you were.

  

You can also create various byobu sessions by byobu -S session1 and so on. And you can connect to either of them when you come back.

You can do much more using Byobu. Use it! Some definitive guides are here, or here.

Answer 7

You cannot do this once the process has started, you need to have set things up before you run a long running job.

You can use nohup but modern wisdom suggests you use screen or byobu as your login so you can detach and leave things running.

Screen has the advantage that you can detach from one machine and reattach from another which is handy if you want to check on long running processes that run beyond the end of the working day.

There is a reasonable getting started guide to screen here.

byobu puts an easy to use interface on top of screen with menus etc. It's also the current implementation of screen on newer ubuntu. F2 to start a new terminal F3/F4 to toggle back and forth and F6 to disconnect. Type exit to actually end terminals permanently.

Answer 8

For a single shell script that I have running over a long period of time, I will login, and run the process in the background using '&'.

Example:

/path/to/my/script &

I've logged out and disconnected my SSH session. When I log in some time later, the script is still executing as proven by continuous data collection from the script.

Answer 9

Hey, while I agreed that screen is the most efective option. You can use vncserver and then start the process on it.

Also if your only interes is to have the process running and no need to take control back of it, and utterly most important you were not aware you will need to close the session and you have the process already running, you are not of luck if you used bash as the shell

First you need to send the process to background by typing Ctrl+Z followed by bg %1 (the number depends on the job number, usually it is 1, but you can easily pull the list using the command jobs)

Finally invoke the command disown (followed by the jobid ... same as with bg command)

This will remove the parent-child relationship between your shell and the process in background, preventing it to die when your shell is terminated.

Answer 10

The easiest way is to run your command in the background with &. Then just write:

disown -a

Answer 11

You should check out GNU Screen and see if it helps you. Depending on how you need you application to run in realtime, it may cause more issues than it solves, but at least it will allow you to resume your session as if you never left it.

How to use :

• Use the command screen for the first start, scroll through the introduction messages, you should be handed a terminal.
• C-a C-c opens another terminal
• C-a C-k kills a terminal
• You can use C-a C-Space and C-a C-Backspace to cycle through terminals
• C-a C-a is handy if you're mostly using only two terminals
• C-a C-d detachs the current screen session and exits screens. You can then use screen -r to resume that session. You can have several detached screen sessions at once, in this case you'll be displayed a list of available sessions.

There are many other options, for example split screens, and also all shortcuts are fully customizable.

Answer 12

Simplest answer...

ctrl+z will suspend the running program

"bg" will run it in the background

Answer 13

Instead of :

cmd options; 

Add before nohup :

nohup cmd options & 

Then, you will be able to see the console stdout by :

tail -f nohup.out

Answer 14

While everyone says to use disown (the only option you have after you already started the process), nohup, or even running the command in screen, which is useful if you want to see all the output from the command... I'm a fan of screen.. I still have tried most recent mainstream distributions of Linux and simply putting the job in the background and quitting does not cause all the processes that are running to die. There must be a global setting or something. I'm trying this on some pretty old systems (slackware 12) and my test script keeps running until I manually kill it:

shell$ cat > test.pl

#!/usr/bin/perl
while(1){
     sleep(1);
}
    shell$ perl ./test.pl &
    shell$ exit
    logout
    shell$ ps aux test.pl
    mymom 31337     1  0 13:25 ?        00:00:00 perl ./test.pl
    shell$ 

While I agree that screen would be the best way to run this, even if my script wrote to log files or whatever.. I've never needed to use disown -a or nohup unless it was out of complete paranoia. Maybe someone can shed some light on how bash behaves by default? Maybe some system administrators change the defaults on large shells to keep their users' processes from overloading the system?