OpenSSH Troubleshooting

Internal

• OpenSSH

Troubleshooting sshd Server

Execute the sshd Server in Foreground

One method to troubleshoot a sshd server in an attempt to figure out why a specific ssh connection does not work as intended, is to stop the current sshd server (stopping the server won't drop the existing connections) and start it in foreground:

systemctl stop sshd

/usr/sbin/sshd -d [-D] [-d] [-d]

sshd running in foreground behaves differently than the same binary running in background - for example, a foreground server will allow root logging with a public key without password, while the background server won't, so this method has limited usefulness in some cases.

Turn On Debug Logging

Stop the current sshd server (stopping the server won't drop the existing connections), increase logging verbosity as described here: sshd Logging Verbosity, and restart the server.

systemctl stop sshd
# update logging verbosity
systemctl start sshd

Increased log output will be available in /var/log/secure.

Scenarios

Connection Takes a Long Time

In most cases this is caused by the server attempting to use DNS to get the IP name of the client. It is fixed by turning 'UseDNS' off. For more details, see:

Turn Off Client Name DNS Verification

Read from socket failed: Connection reset by peer

Symtoms:

ssh user@ip fails with:

NOMBP2:~ ovidiu$ ssh openshift@172.20.2.1
Read from socket failed: Connection reset by peer

/var/log/secure shows:

Sep 11 12:33:40 openshift-master1 sshd[1442]: fatal: No supported key exchange algorithms [preauth]
Sep 11 12:35:36 openshift-master1 sshd[1445]: fatal: No supported key exchange algorithms [preauth]

Root cause:

The host key files had zero length. Not sure why is that, probably the installation procedure did not complete correctly.

To fix:

ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key

with empty passwords.