647

Let's say I launch a bunch of processes from a ssh session. Is it possible to terminate the ssh session while keeping those processes running on the remote machine?

15 Answers 15

752
+100

You should look for modern alternatives like tmux.

tmux is superior to screen for many reasons, here are just some examples:

  • Windows can be moved between session and even linked to multiple sessions
  • Windows can be split horizontally and vertically into panes
  • Support for UTF-8 and 256 colour terminals
  • Sessions can be controlled from the shell without the need to enter a session

Basic Functionality

To get the same functionality as explained in the answer recommending screen, you would need to do the following:

  • ssh into the remote machine
  • start tmux by typing tmux into the shell
  • start the process you want inside the started tmux session
  • leave/detach the tmux session by typing Ctrl+b and then d

You can now safely log off from the remote machine, your process will keep running inside tmux. When you come back again and want to check the status of your process you can use tmux attach to attach to your tmux session.

If you want to have multiple sessions running side-by-side, you should name each session using Ctrl+b and $. You can get a list of the currently running sessions using tmux list-sessions, now attach to a running session with command tmux attach-session -t 0.

tmux can do much more advanced things than handle a single window in a single session. For more information have a look in man tmux or the tmux GitHub page. In particular, here's an FAQ about the main differences between screen and tmux.

  • 4
    @CraigM Use screen -x -r [screenname] or screen -rx for short if you have only one screen session active. This allows you to attach an existing screen instance. – Lekensteyn Nov 25 '12 at 10:31
  • 5
    This advice helped me with the same problem but I think it includes a typo. I am pretty sure that you need to type Ctrl-b and then d in order to leave/detach the tmux session. Certainly that is the case for the version of tmux on my Ubuntu 12.04. – cxrodgers Dec 25 '12 at 8:31
  • 17
    I made a mistake while following the above instructions. I will share it if anybody might fall into the same mistake: I started tmux in the shell of my own computer instead of the shell of the remote computer. One needs to start tmux in the shell of remote computer. – Mert Nuhoglu Aug 22 '14 at 14:53
  • 5
    Screen is now being developed again: lists.gnu.org/archive/html/screen-devel/2014-04/msg00024.html Could you update your answer? – muru Nov 3 '14 at 3:27
  • 4
    Alternatively, run tmux detach instead of typing ctrl-b d – Andrew Nov 3 '15 at 22:51
308

Option 1: nohup

The best way is often the simplest.

nohup long-running-command &

It was made specifically for this, it even logs stdout to nohup.log.

man nohup

Option 2: bg

If you want to "background" some already running tasks, then your best bet is to Ctrl+Z then run

bg

to put your most recent suspended task to background, allowing it to continue running.

Then a quick disown should keep the process running after you log out.

screen and others can do it, but that's not what they're for. I recommend nohup for tasks you know your going to leave behind and bg for tasks your already running and don't want to re-start.

Keep in mind, both are bash specific. If you're not using bash, then the commands could be different.

  • 4
    I would recommend to do disown -h – Michele Aug 21 '17 at 16:45
  • interestingly screen and tmux doesn't work for miner, bur nohup works – Yasin Okumuş Nov 1 '17 at 19:04
  • 1
    bg + disown didn't work for me. I had a deployment script running I forgot to start inside tmux, and needed to leave early for a meeting. The script continuously outputs progress to the shell. ctrl+z stopped the process, returning me to bash. bg resumed the process, but it also resumed outputting the status to bash, making it impossible to see what I was typing. However, a disown command produced "disown: current: no such job" – BrianHVB Sep 11 '18 at 23:42
  • this didn't work for me with julia myFile.jl for whatever reason. tmux does and is great. – kilgoretrout Oct 25 '18 at 5:11
  • This worked for me, thank you so much – Chad Mar 24 at 3:20
207

You could do that by using screen.

Type man screen to find out more or read this screen man page.

Simple scenario:

  • ssh into your remote box. Type screen Then start the process you want.

  • Press Ctrl-A then Ctrl-D. This will "detach" your screen session but leave your processes running. You can now log out of the remote box.

  • If you want to come back later, log on again and type screen -r This will "resume" your screen session, and you can see the output of your process.

  • 32
    I will usually name my screen sessions using screen -S name to make it easier to connect to the correct one later. – David Oneill Nov 24 '12 at 22:37
  • The link is down – Gab是好人 Sep 16 '16 at 13:50
  • 1
    Personally, I'm working on a box without any package control software. After spending about half an hour building dependencies for TMUX (which I personally have experience with and like) from source, it became clear that screen was the better and simpler solution for me. TL;DR: the optimal solution to this problem depends on the use case, the machine, and how much time you have to get set up. Thanks for this answer :) – Max von Hippel Nov 1 '17 at 3:30
  • Combination with screen -S before leave and screen -r when coming back is amazing! – Meloman Mar 14 '18 at 10:03
  • Thanks for this. This solution worked for me while tmux did not (I use a version of bash for windows while ssh'ing into a linux machine) – Michael Sorensen May 13 '18 at 19:53
83

Screen and nohup is the better way, but if you have to detach a process already running without screen or nohup you can run disown command.

disown [-ar] [-h] [jobspec… |pid… ]

Without options, remove each jobspec from the table of active jobs. If the -h option is given, the job is not removed from the table, but is marked so that SIGHUP is not sent to the job if the shell receives a SIGHUP. If jobspec is not present, and neither the -a nor the -r option is supplied, the current job is used. If no jobspec is supplied, the -a option means to remove or mark all jobs; the -r option without a jobspec argument restricts operation to running jobs.

With disown you can close the terminal and get the process running on the machine.

  • 10
    This is also my favourite way to do it. I frequently use disown -a && exit – Stefano Palazzo Oct 21 '10 at 17:45
  • 1
    Perfect. This is a lovely command and deserves all the upvotes! – Greg Apr 29 '14 at 10:38
  • 8
    one word of caution, I stopped a running process with Ctrl-Z and didn't start it in the background before calling disown and it killed it. – HDave Sep 23 '14 at 21:07
  • 3
    May crash if it wants to write to the terminal. Redirecting output of a running process isn't as easy. Better to start it properly with nohup. – jiggunjer Aug 25 '16 at 12:42
  • 2
    This is a good answer, because it directly answers the question. The author of the question asked what to do after having already executed a bunch of long running commands. Most of these answers let you know what to do before you execute the commands. – q0rban Aug 17 '17 at 0:24
53

I was stuck in a large mv so I wasn't in a position to stop the process, setup screen and then start it again. I managed to exit the ssh session with the process running by essentially doing the following steps:

  1. ssh [server]
  2. command
  3. Ctrl+Z
  4. bg
  5. disown [process pid optional, defaults to last]
  6. exit

Step 3 pauses the current process (e.g. my 'mv' command).
Step 4 puts the paused process in to the background and resumes it.
Step 5 lets you disown the process.** To get a list of jobs simply type jobsbefore.


** Regarding disown (from bash manual):

disown [-ar] [-h] [jobspec ... | pid ... ]
              Without  options,  remove  each jobspec from the table of active
              jobs.  If jobspec is not present, and neither the -a nor the  -r
              option  is  supplied, the current job is used.  If the -h option
              is given, each jobspec is not removed from  the  table,  but  is
              marked  so  that  SIGHUP  is  not  sent  to the job if the shell
              receives a SIGHUP.  If no jobspec is  supplied,  the  -a  option
              means  to  remove or mark all jobs; the -r option without a job‐
              spec argument restricts operation to running jobs.   The  return
              value is 0 unless a jobspec does not specify a valid job.
  • 2
    Built-in's are always my first choice :) THX – ken_oy Apr 2 '18 at 13:30
  • This answer should be marked as right. To install and getting used to screen for solving this problem is exaggerated. – Ulrich-Lorenz Schlüter Apr 5 '18 at 11:41
  • This is very helpful when you have already started any command and which is being running for long. Terminating and starting it in a TMUX Session will take time – Deepali Mittal May 14 '18 at 10:54
  • 1
    @tom-brossman: Please watch out for unnecessary edits like this one! If invoked without job specification the disown built-in command acts on the most recent background job. – David Foerster Oct 29 '18 at 16:13
22

There are two major programs you can use to maintain programs and terminal state over multiple ssh connections. They are screen (the incumbent, but unfortunately unmaintained. Apparently being actively developed now) and tmux (newer, actively maintained). Byobu is a front end that can run on top of their of these systems and offer additional ubuntu status information. On new installations it will use tmux as a backend, if you have an older installation of byobu and an existing config it will maintain the previous backend, be it screen or tmux.

Byobu

Byobu can be installed on the computer by doing so in a Debian-based machine:

sudo aptitude install byobu

Using yum, you do

su -c 'yum install byobu'

It's also possible to install byobu on other distributions.

Using byobu

You can start byobu by running byobu on the host machine after connecting using ssh. This will give you a shell that looks like this:

image-byobu

You can also use Byobu Terminal on a Ubuntu machine with -X option and easily have a perfectly working byobu.

Usage:

Start byobu by typing byobu.

You can press F2 to create a new window within the current session, F3-F4 to switch between the various windows.

The best part about byobu is, you dont have to actually kill the processes running in the terminal to leave the terminal. You can simply send screen/tmux (the skeleton of byobu) to background and resume the next time you come:

  • To leave byobu and keeep it running (detach) press F6.
  • The next time you come, just do byobu and you sholud be back right where you were.

    byobu-detach-attach

You can also create various byobu sessions by byobu -S session1 and so on. And you can connect to either of them when you come back.

You can do much more using Byobu. Use it! Some definitive guides are here, or here.

  • I tried using byobu from a PuTTY-based session to my Ubuntu box, but I get the status line repeated and scrolling on the display. Although it detached properly wen pressing F6, it was not a useable solution in my setup. – jfmessier Nov 6 '13 at 17:47
  • 1
    @jfmessier It's because PuTTY doesn't take ncurses (utf-8 basically) well. It's possible to eliminate this problem by following this thread - stackoverflow.com/questions/10731099/… – SiddharthaRT Nov 7 '13 at 10:40
  • This is awesome! 1. it gives me a colored bash prompt that I cannot seem to enable with bash as my default shell and 2. I get to run 2 bots at the same time and still have another terminal to do work on! @SiddharthaRT you deserve an upvote sir! – Dev Jul 22 '16 at 6:53
18

You cannot do this once the process has started, you need to have set things up before you run a long running job.

You can use nohup but modern wisdom suggests you use screen or byobu as your login so you can detach and leave things running.

Screen has the advantage that you can detach from one machine and reattach from another which is handy if you want to check on long running processes that run beyond the end of the working day.

There is a reasonable getting started guide to screen here.

byobu puts an easy to use interface on top of screen with menus etc. It's also the current implementation of screen on newer ubuntu. F2 to start a new terminal F3/F4 to toggle back and forth and F6 to disconnect. Type exit to actually end terminals permanently.

  • byobu uses tmux these days.. – scottl Nov 28 '12 at 7:04
  • 12
    "You cannot do this once the process has started, you need to have set things up before you run a long running job." -- no, you can use disown to achieve this. See @bassgey's answer – Rich May 23 '13 at 16:11
  • 1
    after struggling to learn screen and tmux....byobu brought tears to my eyes – HDave Sep 23 '14 at 21:06
  • disown and if needed just Ctrl-z, then bg to get it of the active terminal and to the background. Then, disown. – mimoralea Nov 21 '17 at 20:49
8

Hey, while I agreed that screen is the most efective option. You can use vncserver and then start the process on it.

Also if your only interes is to have the process running and no need to take control back of it, and utterly most important you were not aware you will need to close the session and you have the process already running, you are not of luck if you used bash as the shell

First you need to send the process to background by typing Ctrl+Z followed by bg %1 (the number depends on the job number, usually it is 1, but you can easily pull the list using the command jobs)

Finally invoke the command disown (followed by the jobid ... same as with bg command)

This will remove the parent-child relationship between your shell and the process in background, preventing it to die when your shell is terminated.

  • 2
    This answer is the best! Why is everyone talking about screen, the question was posed post-login event, how to keep the processes running, now after logging in but before starting them. Great answer Jorge, you've really helped me! :) – jwbensley Aug 10 '12 at 17:13
  • 1
    Simply bg (without %1) is often enough, since the default is the current job – Walter Tross Nov 23 '12 at 21:55
  • It would be nice to have a comprehensive answer: A priori (setting up in advance): ascii terminal screen (old), tmux (neewer); X windows: vnc (old, still maintained), xpra (newer), with xpra being rootless. A posterior (persisting after you started): ^Z, disown, ... But I am too lazy to flesh it out more than this. – Krazy Glew Jul 12 '18 at 23:02
8

For a single shell script that I have running over a long period of time, I will login, and run the process in the background using '&'.

Example:

/path/to/my/script &

I've logged out and disconnected my SSH session. When I log in some time later, the script is still executing as proven by continuous data collection from the script.

  • 3
    Yes, I would like to know how screen / tmux is better than this simple solution. – Mads Skjern Mar 16 '15 at 8:51
  • Yes, I can see this on my ubuntu also, but it should not happen in theory. I don't understand why – Daniel Pinyol Apr 22 '16 at 13:08
  • 1
    @MadsSkjern Because you can't enter any input into the script using this method. – Ken Sharp Feb 10 '17 at 22:18
  • 1
    @MadsSkjern the reason is that if you run processes like this with & if you log out and in to your SSH session, the process will still be running however you will not be able to see the outputof that process (if your script echoed something you won't see it, but if it writes a file the file will be there) – DarkCygnus Aug 24 '17 at 16:20
4

You should check out GNU Screen and see if it helps you. Depending on how you need you application to run in realtime, it may cause more issues than it solves, but at least it will allow you to resume your session as if you never left it.

How to use :

  • Use the command screen for the first start, scroll through the introduction messages, you should be handed a terminal.
  • C-a C-c opens another terminal
  • C-a C-k kills a terminal
  • You can use C-a C-Space and C-a C-Backspace to cycle through terminals
  • C-a C-a is handy if you're mostly using only two terminals
  • C-a C-d detachs the current screen session and exits screens. You can then use screen -r to resume that session. You can have several detached screen sessions at once, in this case you'll be displayed a list of available sessions.

There are many other options, for example split screens, and also all shortcuts are fully customizable.

3

Simplest answer...

ctrl+z will suspend the running program

"bg" will run it in the background

  • 5
    Without disowning the process (with something like disown or nohup), this will not usually keep the process running after the end of the SSH session. – Eliah Kagan Aug 13 '13 at 0:32
  • 3
    On my Ubuntu server, with default setup, it really does keep running! – Mads Skjern Mar 16 '15 at 8:53
3

The easiest way is to run your command in the background with &. Then just write:

disown -a
0

While everyone says to use disown (the only option you have after you already started the process), nohup, or even running the command in screen, which is useful if you want to see all the output from the command... I'm a fan of screen.. I still have tried most recent mainstream distributions of Linux and simply putting the job in the background and quitting does not cause all the processes that are running to die. There must be a global setting or something. I'm trying this on some pretty old systems (slackware 12) and my test script keeps running until I manually kill it:

shell$ cat > test.pl

#!/usr/bin/perl
while(1){
     sleep(1);
}
    shell$ perl ./test.pl &
    shell$ exit
    logout
    shell$ ps aux test.pl
    mymom 31337     1  0 13:25 ?        00:00:00 perl ./test.pl
    shell$ 

While I agree that screen would be the best way to run this, even if my script wrote to log files or whatever.. I've never needed to use disown -a or nohup unless it was out of complete paranoia. Maybe someone can shed some light on how bash behaves by default? Maybe some system administrators change the defaults on large shells to keep their users' processes from overloading the system?

  • If you have any further query please ask it as a new question – heemayl Jul 30 '15 at 2:05
0

Unfortunately, a terminated SSH session can result in tmux or screen being killed off. This is because systemd will terminate any child processes upon logging out of a session.

You can change this setting in your logind.conf (/etc/systemd/logind.conf):

KillUserProcesses=no

Thanks to answer in https://unix.stackexchange.com/questions/490267.

-1

Instead of :

cmd options; 

Add before nohup :

nohup cmd options & 

Then, you will be able to see the console stdout by :

tail -f nohup.out

protected by heemayl Sep 16 '15 at 20:52

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.