998

Let's say I launch a bunch of processes from a ssh session. Is it possible to terminate the ssh session while keeping those processes running on the remote machine?

2

14 Answers 14

1122
+100

You should look for modern alternatives like tmux.

tmux is superior to screen for many reasons, here are just some examples:

  • Windows can be moved between session and even linked to multiple sessions
  • Windows can be split horizontally and vertically into panes
  • Support for UTF-8 and 256 colour terminals
  • Sessions can be controlled from the shell without the need to enter a session

Basic Functionality

To get the same functionality as explained in the answer recommending screen, you would need to do the following:

  • ssh into the remote machine
  • start tmux by typing tmux into the shell
  • start the process you want inside the started tmux session
  • leave/detach the tmux session by typing Ctrl+b and then d

You can now safely log off from the remote machine, your process will keep running inside tmux. When you come back again and want to check the status of your process you can use tmux attach to attach to your tmux session.

If you want to have multiple sessions running side-by-side, you should name each session using Ctrl+b and $. You can get a list of the currently running sessions using tmux list-sessions or simply tmux ls, now attach to a running session with command tmux attach-session -t <session-name>.

tmux can do much more advanced things than handle a single window in a single session. For more information have a look in man tmux or the tmux GitHub page. In particular, here's an FAQ about the main differences between screen and tmux.

18
  • 4
    @CraigM Use screen -x -r [screenname] or screen -rx for short if you have only one screen session active. This allows you to attach an existing screen instance.
    – Lekensteyn
    Nov 25, 2012 at 10:31
  • 8
    This advice helped me with the same problem but I think it includes a typo. I am pretty sure that you need to type Ctrl-b and then d in order to leave/detach the tmux session. Certainly that is the case for the version of tmux on my Ubuntu 12.04.
    – cxrodgers
    Dec 25, 2012 at 8:31
  • 23
    I made a mistake while following the above instructions. I will share it if anybody might fall into the same mistake: I started tmux in the shell of my own computer instead of the shell of the remote computer. One needs to start tmux in the shell of remote computer. Aug 22, 2014 at 14:53
  • 6
    Screen is now being developed again: lists.gnu.org/archive/html/screen-devel/2014-04/msg00024.html Could you update your answer?
    – muru
    Nov 3, 2014 at 3:27
  • 11
    Alternatively, run tmux detach instead of typing ctrl-b d
    – Andrew
    Nov 3, 2015 at 22:51
598

Option 1: nohup

The best way is often the simplest.

nohup long-running-command &

It was made specifically for this, it even logs stdout to nohup.log.

man nohup

Option 2: bg + disown

Ctrl+Z followed by:

bg
disown -h

If you want to "background" already running tasks, then Ctrl+Z then run bg to put your most recent suspended task to background, allowing it to continue running. disown will keep the process running after you log out. The -h flag prevents hangup.


screen and others can do it, but that's not what they're for. I recommend nohup for tasks you know you are going to leave behind and bg for tasks you're already running and don't want to re-start.

Keep in mind, both are bash specific. If you're not using bash, then the commands could be different.

11
  • 12
    I would recommend to do disown -h
    – Michele
    Aug 21, 2017 at 16:45
  • interestingly screen and tmux doesn't work for miner, bur nohup works Nov 1, 2017 at 19:04
  • 5
    bg + disown didn't work for me. I had a deployment script running I forgot to start inside tmux, and needed to leave early for a meeting. The script continuously outputs progress to the shell. ctrl+z stopped the process, returning me to bash. bg resumed the process, but it also resumed outputting the status to bash, making it impossible to see what I was typing. However, a disown command produced "disown: current: no such job"
    – Brian H.
    Sep 11, 2018 at 23:42
  • 2
    Just to clarify, Option 2 is bg + disown. bg by itself isn't enough. bg or & might work if you log off your ssh (because default shopt huponexit=off), but it won't work if your ssh connection gets dropped unless you disown
    – wisbucky
    Apr 28, 2020 at 5:55
  • 6
    I needed to use disown -a instead of disown -h on Ubuntu 18; make sure the output of jobs is empty after disowning to verify that it's truly disowned. Jul 24, 2020 at 23:08
278

You could do that by using screen.

Type man screen to find out more or read this screen man page.

Simple scenario:

  • ssh into your remote box. Type screen. Then start the process you want.

  • Press Ctrl+A, then Ctrl+D. This will "detach" your screen session, but leave your processes running. You can now log out of the remote box.

  • If you want to come back later, log on again and type screen -r This will "resume" your screen session and you can see the output of your process.

7
  • 45
    I will usually name my screen sessions using screen -S name to make it easier to connect to the correct one later. Nov 24, 2012 at 22:37
  • 3
    Personally, I'm working on a box without any package control software. After spending about half an hour building dependencies for TMUX (which I personally have experience with and like) from source, it became clear that screen was the better and simpler solution for me. TL;DR: the optimal solution to this problem depends on the use case, the machine, and how much time you have to get set up. Thanks for this answer :) Nov 1, 2017 at 3:30
  • 3
    Combination with screen -S before leave and screen -r when coming back is amazing!
    – Meloman
    Mar 14, 2018 at 10:03
  • Thanks for this. This solution worked for me while tmux did not (I use a version of bash for windows while ssh'ing into a linux machine) May 13, 2018 at 19:53
  • 5
    It is also helpful to use screen -list to list all active screens when you'd like to resume a screen in the future. You can use screen -r XXXXX to resume screen with id XXXXX.
    – Dale Kube
    Jan 4, 2019 at 21:16
106

Screen and nohup is the better way, but if you have to detach a process already running without screen or nohup you can run disown command.

disown [-ar] [-h] [jobspec… |pid… ]

Without options, remove each jobspec from the table of active jobs. If the -h option is given, the job is not removed from the table, but is marked so that SIGHUP is not sent to the job if the shell receives a SIGHUP. If jobspec is not present, and neither the -a nor the -r option is supplied, the current job is used. If no jobspec is supplied, the -a option means to remove or mark all jobs; the -r option without a jobspec argument restricts operation to running jobs.

With disown you can close the terminal and get the process running on the machine.

5
  • 13
    This is also my favourite way to do it. I frequently use disown -a && exit Oct 21, 2010 at 17:45
  • 1
    Perfect. This is a lovely command and deserves all the upvotes!
    – Greg
    Apr 29, 2014 at 10:38
  • 8
    one word of caution, I stopped a running process with Ctrl-Z and didn't start it in the background before calling disown and it killed it.
    – Dave
    Sep 23, 2014 at 21:07
  • 3
    May crash if it wants to write to the terminal. Redirecting output of a running process isn't as easy. Better to start it properly with nohup.
    – jiggunjer
    Aug 25, 2016 at 12:42
  • 4
    This is a good answer, because it directly answers the question. The author of the question asked what to do after having already executed a bunch of long running commands. Most of these answers let you know what to do before you execute the commands.
    – q0rban
    Aug 17, 2017 at 0:24
91

I was stuck in a large mv so I wasn't in a position to stop the process, setup screen and then start it again. I managed to exit the SSH session with the process running by essentially doing the following steps:

  1. Establish SSH connection: ssh user@host
  2. Run the desired command to start the process
  3. Press Ctrl+Z to pause the process
  4. Run bg to put the paused process in the background and resume it.
  5. Run disown [pid] (process ID is optional, defaults to last process) to disown the process. To get a list of jobs simply type jobs before.
  6. Exit the SSH session by running logout.

Usage of the disown command:

disown [-ar] [-h] [jobspec ... | pid ... ]
              Without  options,  remove  each jobspec from the table of active
              jobs.  If jobspec is not present, and neither the -a nor the  -r
              option  is  supplied, the current job is used.  If the -h option
              is given, each jobspec is not removed from  the  table,  but  is
              marked  so  that  SIGHUP  is  not  sent  to the job if the shell
              receives a SIGHUP.  If no jobspec is  supplied,  the  -a  option
              means  to  remove or mark all jobs; the -r option without a job‐
              spec argument restricts operation to running jobs.   The  return
              value is 0 unless a jobspec does not specify a valid job.
4
  • 4
    Built-in's are always my first choice :) THX
    – ken_oy
    Apr 2, 2018 at 13:30
  • This answer should be marked as right. To install and getting used to screen for solving this problem is exaggerated. Apr 5, 2018 at 11:41
  • This is very helpful when you have already started any command and which is being running for long. Terminating and starting it in a TMUX Session will take time May 14, 2018 at 10:54
  • 1
    @tom-brossman: Please watch out for unnecessary edits like this one! If invoked without job specification the disown built-in command acts on the most recent background job. Oct 29, 2018 at 16:13
28

There are two major programs you can use to maintain programs and terminal state over multiple ssh connections. They are screen (the incumbent, but unfortunately unmaintained. Apparently being actively developed now) and tmux (newer, actively maintained). Byobu is a front end that can run on top of their of these systems and offer additional ubuntu status information. On new installations it will use tmux as a backend, if you have an older installation of byobu and an existing config it will maintain the previous backend, be it screen or tmux.

Byobu

Byobu can be installed on the computer by doing so in a Debian-based machine:

sudo aptitude install byobu

Using yum, you do

su -c 'yum install byobu'

It's also possible to install byobu on other distributions.

Using byobu

You can start byobu by running byobu on the host machine after connecting using ssh. This will give you a shell that looks like this:

image-byobu

You can also use Byobu Terminal on a Ubuntu machine with -X option and easily have a perfectly working byobu.

Usage:

Start byobu by typing byobu.

You can press F2 to create a new window within the current session, F3-F4 to switch between the various windows.

The best part about byobu is, you dont have to actually kill the processes running in the terminal to leave the terminal. You can simply send screen/tmux (the skeleton of byobu) to background and resume the next time you come:

  • To leave byobu and keeep it running (detach) press F6.

  • The next time you come, just do byobu and you sholud be back right where you were.

    byobu-detach-attach

You can also create various byobu sessions by byobu -S session1 and so on. And you can connect to either of them when you come back.

You can do much more using Byobu. Use it! Some definitive guides are here, or here.

3
  • I tried using byobu from a PuTTY-based session to my Ubuntu box, but I get the status line repeated and scrolling on the display. Although it detached properly wen pressing F6, it was not a useable solution in my setup.
    – jfmessier
    Nov 6, 2013 at 17:47
  • 1
    @jfmessier It's because PuTTY doesn't take ncurses (utf-8 basically) well. It's possible to eliminate this problem by following this thread - stackoverflow.com/questions/10731099/… Nov 7, 2013 at 10:40
  • This is awesome! 1. it gives me a colored bash prompt that I cannot seem to enable with bash as my default shell and 2. I get to run 2 bots at the same time and still have another terminal to do work on! @SiddharthaRT you deserve an upvote sir!
    – Dev
    Jul 22, 2016 at 6:53
20

You cannot do this once the process has started, you need to have set things up before you run a long running job.

You can use nohup but modern wisdom suggests you use screen or byobu as your login so you can detach and leave things running.

Screen has the advantage that you can detach from one machine and reattach from another which is handy if you want to check on long running processes that run beyond the end of the working day.

There is a reasonable getting started guide to screen here.

byobu puts an easy to use interface on top of screen with menus etc. It's also the current implementation of screen on newer ubuntu. F2 to start a new terminal F3/F4 to toggle back and forth and F6 to disconnect. Type exit to actually end terminals permanently.

4
  • 1
    byobu uses tmux these days..
    – scottl
    Nov 28, 2012 at 7:04
  • 20
    "You cannot do this once the process has started, you need to have set things up before you run a long running job." -- no, you can use disown to achieve this. See @bassgey's answer
    – Rich
    May 23, 2013 at 16:11
  • 1
    after struggling to learn screen and tmux....byobu brought tears to my eyes
    – Dave
    Sep 23, 2014 at 21:06
  • disown and if needed just Ctrl-z, then bg to get it of the active terminal and to the background. Then, disown.
    – mimoralea
    Nov 21, 2017 at 20:49
11

For a single shell script that I have running over a long period of time, I will login, and run the process in the background using '&'.

Example:

/path/to/my/script &

I've logged out and disconnected my SSH session. When I log in some time later, the script is still executing as proven by continuous data collection from the script.

6
  • 3
    Yes, I would like to know how screen / tmux is better than this simple solution. Mar 16, 2015 at 8:51
  • 1
    @MadsSkjern Because you can't enter any input into the script using this method.
    – Ken Sharp
    Feb 10, 2017 at 22:18
  • 3
    @MadsSkjern the reason is that if you run processes like this with & if you log out and in to your SSH session, the process will still be running however you will not be able to see the outputof that process (if your script echoed something you won't see it, but if it writes a file the file will be there)
    – DarkCygnus
    Aug 24, 2017 at 16:20
  • 1
    How to output to some file while keep it running? Should I put the > before or after &?
    – JP Zhang
    Mar 24, 2020 at 3:13
  • 1
    @AugustinNewton it's some_cmd > some_file 2>&1 & see this question here: unix.stackexchange.com/a/106641/351732
    – N4ppeL
    Sep 23, 2020 at 22:22
8

Hey, while I agreed that screen is the most efective option. You can use vncserver and then start the process on it.

Also if your only interes is to have the process running and no need to take control back of it, and utterly most important you were not aware you will need to close the session and you have the process already running, you are not of luck if you used bash as the shell

First you need to send the process to background by typing Ctrl+Z followed by bg %1 (the number depends on the job number, usually it is 1, but you can easily pull the list using the command jobs)

Finally invoke the command disown (followed by the jobid ... same as with bg command)

This will remove the parent-child relationship between your shell and the process in background, preventing it to die when your shell is terminated.

3
  • 2
    This answer is the best! Why is everyone talking about screen, the question was posed post-login event, how to keep the processes running, now after logging in but before starting them. Great answer Jorge, you've really helped me! :)
    – Baldrick
    Aug 10, 2012 at 17:13
  • 1
    Simply bg (without %1) is often enough, since the default is the current job Nov 23, 2012 at 21:55
  • It would be nice to have a comprehensive answer: A priori (setting up in advance): ascii terminal screen (old), tmux (neewer); X windows: vnc (old, still maintained), xpra (newer), with xpra being rootless. A posterior (persisting after you started): ^Z, disown, ... But I am too lazy to flesh it out more than this.
    – Krazy Glew
    Jul 12, 2018 at 23:02
5

The easiest way is to run your command in the background with &. Then just write:

disown -a
0
4

You should check out GNU Screen and see if it helps you. Depending on how you need you application to run in realtime, it may cause more issues than it solves, but at least it will allow you to resume your session as if you never left it.

How to use :

  • Use the command screen for the first start, scroll through the introduction messages, you should be handed a terminal.
  • C-a C-c opens another terminal
  • C-a C-k kills a terminal
  • You can use C-a C-Space and C-a C-Backspace to cycle through terminals
  • C-a C-a is handy if you're mostly using only two terminals
  • C-a C-d detachs the current screen session and exits screens. You can then use screen -r to resume that session. You can have several detached screen sessions at once, in this case you'll be displayed a list of available sessions.

There are many other options, for example split screens, and also all shortcuts are fully customizable.

3

Simplest answer...

ctrl+z will suspend the running program

"bg" will run it in the background

2
  • 7
    Without disowning the process (with something like disown or nohup), this will not usually keep the process running after the end of the SSH session. Aug 13, 2013 at 0:32
  • 3
    On my Ubuntu server, with default setup, it really does keep running! Mar 16, 2015 at 8:53
1

Instead of :

cmd options; 

Add before nohup :

nohup cmd options & 

Then, you will be able to see the console stdout by :

tail -f nohup.out
0

While everyone says to use disown (the only option you have after you already started the process), nohup, or even running the command in screen, which is useful if you want to see all the output from the command... I'm a fan of screen.. I still have tried most recent mainstream distributions of Linux and simply putting the job in the background and quitting does not cause all the processes that are running to die. There must be a global setting or something. I'm trying this on some pretty old systems (slackware 12) and my test script keeps running until I manually kill it:

shell$ cat > test.pl

#!/usr/bin/perl
while(1){
     sleep(1);
}
    shell$ perl ./test.pl &
    shell$ exit
    logout
    shell$ ps aux test.pl
    mymom 31337     1  0 13:25 ?        00:00:00 perl ./test.pl
    shell$ 

While I agree that screen would be the best way to run this, even if my script wrote to log files or whatever.. I've never needed to use disown -a or nohup unless it was out of complete paranoia. Maybe someone can shed some light on how bash behaves by default? Maybe some system administrators change the defaults on large shells to keep their users' processes from overloading the system?

1
  • If you have any further query please ask it as a new question
    – heemayl
    Jul 30, 2015 at 2:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .