How do you fix the issue with a SSH server which freezes after authenticating?
$ ssh [email protected]
The authenticity of host ‘xx.xx.xxx.xxx (xx.xx.xxx.xxx)’ can’t be established.
RSA key fingerprint is 56:c2:ec:dd:64:ba:4d:3b:d0:fa:fd:f2:b2:7f:bf:b3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘xx.xx.xxx.xxx’ (RSA) to the list of known hosts.
PTY allocation request failed on channel 0
In the above scenario, you can see that I logged in but I’m not able to get into the console. Now, to trouble shoot this issue, I must run the ssh in debug more using -vv option.
Once you’re in debug more, you will see lots of messages elaborating the SSH access steps in detail. These messages might point you to the root cause or the stage at which you’re being locked.
In my case, I got freezed after authenticating myself.
debug1: Next authentication method: keyboard-interactive
debug1: Authentication succeeded (keyboard-interactive).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
So, I was not greeted with any hints to resolve the issue. Now, I should go ahead and check the logs in the server to which I’m unable to log in. This is possible only if you’re inside a console of the same server via KVM or from some other remote access method. Other wise you got to be a DC tech to directly login to the server.
Now, check the SSH configuration to confirm that SSH daemon is logging the messages under /var/log/secure or /var/log/messages. Following line must be uncommented for the same in /etc/ssh/sshd_config.
Restart the SSH service once you’re done with the editing and look at /var/log/secure.
This pointed me to the root cause:
Oct 13 07:03:56 server1 sshd: reverse mapping checking getaddrinfo for sdpgsdcds2.us.infsd.com failed – POSSIBLE BREAKIN ATTEMPT!
Oct 13 07:03:57 server1 sshd: Accepted publickey for root from ::ffff:xxx.xxx.xxx.xxx port 39047 ssh2
Oct 13 07:03:59 server1 sshd: error: openpty: No such file or directory
Oct 13 07:03:59 server1 sshd: error: session_pty_req: session 0 alloc failed
I found that its due to a device file which was missing after googling for some time. I remembered working with such files couple of weeks back. That made me think of creating the device file using the old technique.
Now, finally I created the missing device file ptmx and confirmed that its back in place.
sbin/MAKEDEV -d /dev ptmx
ls -l /dev/ptmx
crw-rw-rw- 1 root root 5, 2 Oct 13 07:07 /dev/ptmx
Once this was over, I just restarted SSH and found that all are able to login to SSH server without any issue.
ptmx issue was due to udev package which was recently installed on the server during the CentOS upgrade from 3.9 to 4.6. This is common in some of the VPS os_templates which we use on OpenVZ servers.