Cannot login to Linux server via SSH/Console

Question

I got two computers with the exact same symptoms today: Both seem to be alive. They answer pings and they host virtual machines which are working.

However, I cannot login via SSH. I get

 $ ssh -v obelix
OpenSSH_5.3p1 Debian-3ubuntu6, OpenSSL 0.9.8k 25 Mar 2009
debug1: Reading configuration data /home/mike/.ssh/config
debug1: Applying options for obelix
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to obelix [10.10.11.238] port 22.
debug1: Connection established.
debug1: identity file /home/mike/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
ssh_exchange_identification: Connection closed by remote host

On the console I can switch the ttys and enter my login credentials. After that I see the GNU/Debian Linux welcome message an no prompt. The system seems to hang. I'm able to switch the tty again and login with the same result. (I tried that three times now and I start running out of ttys.)

A search on the Internet suggested some ressoucre shortage.
How could I release some resoucres than?

How could I further investigate the situation before just trying a hard power off?
(Currently I don't even know how to reboot the system gracefully.)

Answer 1

This does sound very much like it has run out of one resource or another.

How long have you tried leaving the console login? It may succeed after a (considerably long) time.

Unless you can log in there is no way to shut the system down gracefully, although pressing the power button may send an ACPI PowerDown signal which should shut the system down gracefully. You may find even this fails to operate though, so a hard power-off (press and hold the power button for 4 seconds) is all that you can do really.

The symptoms are very reminiscent of a 'fork/malloc' bomb - a rather nasty little trick in unprotected Linux systems whereby a basic user can cause the machine to grind to a halt by consuming all the free processes and using up all the memory.

You should investigate limiting the number of processes and amount of memory a basic user can use to ensure that this doesn't happen again.