For a long time I had a problem with ssh login on a Redhat 6 server – it was taking too long to connect to it, around 30 seconds. Normally it hasn’t been a big issue – after all, you connect once and work for all day as long as you enable server keepalive packets to avoid session timeout.
However when it comes to work with SFTP o GIT it might become annoying. Everytime you sFTP upload or git push you have to wait 30 seconds again.
This kind of problems are often related to DNS issues but this is not always the case. Following are the most common solutions:
1. Disable reverse IP resolution on SSH server
It turns out there is a setting in OpenSSH that controls whether SSHd should not only resolve remote host names but also check whether the resolved host names map back to remote IPs. Apparently, that setting is enabled by default in OpenSSH. The directive UseDNS controls this particular behaviour of OpenSSH, and while it is commented in sshd_config (which is the default configuration file for the OpenSSH daemon in most enviornments), as per the man page for sshd_config, the default for UseDNS is set to enabled. Add the following line:
2. DNS resolver fix for IPv4/IPv6 enabled stacks
It’s a known issue on the Red Hat knowledgebase article DOC-58626, but since it’s closed without login, I’ll share the solution below:
The resolver uses the same socket for the A and AAAA requests. Some hardware mistakenly only sends back one reply. When that happens the client sytem will sit and wait for the second reply. Turning this option on changes this behavior so that if two requests from the same port are not handled correctly it will close the socket and open a new one before sending the second request.
The solution is to add the following line to your /etc/resolv.conf. Just add it all the way at the bottom, as the last line.
3. Disable GSSAPI authentication method
OpenSSH server enables by default the GSSAPI key exchange which allows you to leverage an existing key management infrastructure such as Kerberos or GSI, instead of having to distribute ssh host keys throughout your organisation. With GSSAPI key exchange servers do not need ssh host keys when being accessed by clients with valid credentials.
If you are not using GSSAPI as a authentication mecanism, it might be causing this connection delay.
In my particular case, I ran ssh -v myserver to find out that it was hanging whilst attempting to authenticate with GSSAPI, with the slow section looking like:
.... .... debug2: key: /home/user/.ssh/id_rsa (0xb961d7a8) debug2: key: /home/user/.ssh/id_dsa ((nil)) debug2: key: /home/user/.ssh/id_ecdsa ((nil)) debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-keyex debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-keyex debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug2: we did not send a packet, disable method debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_1000' not found debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_1000' not found debug1: Unspecified GSS failure. Minor code may provide more information
Turned out that it was stalling after trying gssapi-with-mic authentication method. Had several “Unspecified GSS failure” messages with several seconds delay between them, therefore it was definitely the root cause of long delays.
The fix is simple – disable attempts to use GSS-API by adding the following to /etc/sshd_config (server side) or yout ~/.ssh/ssh_config (client side).
There is an easy way to check beforehand whether this solution will work. Try to ssh into your server by disabling GSSAPI authentication:
ssh -o GSSAPIAuthentication=no user@yourserver