20

I have a server that runs Debian and sshd on it, and in case I need to reboot the server my SSH session hangs at client side until TCP timeout. I assume this is because when sshd is being terminated it does not explicitly close open SSH sessions to the host. What should I do to make sshd first disconnect everyone, then terminate itself as normal? So far I don't see a parameter in man sshd_config that's related to shutsown behavior.

1
  • 1
    When all else fails, you can kill an SSH client at any time by pressing [Enter] [~] [.]. Explanation on how to actually solve your problem coming up as well.
    – n.st
    Commented Jul 17, 2015 at 9:05

6 Answers 6

38

When you shutdown or reboot your system, systemd tries to stop all services as fast as it can. That involves bringing down the network and terminating all processes that are still alive -- usually in that order. So when systemd kills the forked SSH processes that are handling your SSH sessions, the network connection is already disabled and they have no way of closing the client connection gracefully.

Your first thought might be to just kill all SSH processes as the first step during shutdown, and there are quite a few systemd service files out there that do just that.

But there is of course a neater solution (how it's "supposed" to be done): systemd-logind.
systemd-logind keeps track of active user sessions (local and SSH ones) and assigns all processes spawned within them to so-called "slices". That way, when the system is shut down, systemd can just SIGTERM everything inside the user slices (which includes the forked SSH process that's handing a particular session) and then continue shutting down services and the network.

systemd-logind requires a PAM module to get notified of new user sessions and you'll need dbus to use loginctl to check its status, so install both of those:

apt-get install libpam-systemd dbus

Be sure your /etc/ssh/sshd_config is actually going to use the module with UsePAM yes.

5
  • Okay, goal achieved, and thanks for the explanation. (Although I'd like some way of dependency control for shutdown, so that everything is first SIGTERM'd while being able to finish their work, this can do as a layered solution.)
    – Vesper
    Commented Aug 3, 2015 at 15:09
  • For some reason I wasn't notified about the answer while visiting only StackOverflow. Weird.
    – Vesper
    Commented Aug 3, 2015 at 15:10
  • 1
    So, for short, just to shave few seconds at shutdown, we have to install a whole load of crap just to have our connections terminated properly ? Seriously ? This is getting awry. We're doomed.
    – leucos
    Commented Jun 29, 2016 at 9:31
  • 3
    Note that tjhe first reboot after installing libpam-systemd and dbus would still leave your SSH session hanging. To avoid that, instead of reboot, do a shutdown -r which defaults to a 1 minute delay, leaving you time to close the SSH session.
    – mivk
    Commented Oct 30, 2016 at 19:20
  • 1
    We are still having this problem in 2020 (fresh installation of Debian buster). The solution shown here did not work for me; however, Rfraile's answer did.
    – Binarus
    Commented Mar 31, 2020 at 11:20
11

This is something you need to set on the client side, not the server side. Edit your ~/.ssh/config to contain

ServerAliveInterval 15
ServerAliveCountMax 5

This means that after 15 seconds of inactivity, your client will send a message to the server. If it doesn't get any response, it will try again up to 5 times, and when it still doesn't get an answer, it'll close the session.

3
  • 3
    This will result in 75s delay before ssh client reports server dead. Right?
    – Vesper
    Commented Jul 17, 2015 at 8:39
  • 1
    Yes, and you can adjust the parameters to get shorter timeout. Commented Jul 17, 2015 at 9:12
  • This solved the problem for me. Such an irritating problem. Commented Jun 28, 2016 at 1:09
11

This behaviour is reported on this Debian Bug, you only need to setup correctly the shutdown scripts shiped with the package because, automatically, they aren't copied by default:

cp /usr/share/doc/openssh-client/examples/ssh-session-cleanup.service /etc/systemd/system/
systemctl  enable ssh-session-cleanup.service
2
  • 1
    Great job. This problem has been a pain in my ... for several years now. However, it was not essential and life-threatening enough to put a lot of time and research into it. Now it's 2020, and obviously, Linux distributors are still not able to solve it (fresh installation of Debian buster). By accident, researching something else, I came across this answer, quickly tried it, not expecting that it would work (all the other solutions I saw didn't), but hey - Success! Upvoted ... And by the way, the accepted answer did not work for me (probably because it is three years later now).
    – Binarus
    Commented Mar 31, 2020 at 11:25
  • works as advertised with debian 11, simple and much appreciated. this really should be the accepted answer.
    – cloudxix
    Commented May 30, 2022 at 1:26
2

You can specify the options that Jenny D talked about in her answer just for one ssh command, such as

ssh -t -o ServerAliveInterval=1 -o ServerAliveCountMax=1 user@host sudo poweroff

if you do that often, you can script it.

1

sadly serverfault didn't let me answer in thread cause by too few points since years. But I don't need to spam in other blogs to get the unlocking ^^... so as dedicated answer:

As Rfraile mentioned

cp /usr/share/doc/openssh-client/examples/ssh-session-cleanup.service /etc/systemd/system/
systemctl  enable ssh-session-cleanup.service

works. To be using it without reboot the instance/server you should do additional tasks:

systemctl daemon-reload
systemctl start ssh-session-cleanup.service

so the service is registered and started and systemd needs to stop it for reboot/shutdown purposes.

2
  • So are you just repeating another answer?
    – RalfFriedl
    Commented May 7, 2019 at 18:45
  • No? I used the 2 lines only as reference for the upper reply because I can't reply yet as written. I added the necessary steps to use it without reboot. This may be given also in another question thread but that I haven't checked out.
    – Reiner030
    Commented May 8, 2019 at 19:24
0

Works for me with lshd. So the solution would be

apt install lsh-server
apt remove openssh-server

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .