SSH Connection Issues: Complete Diagnostic and Troubleshooting Guide

Introduction / Why This Is Needed

Connecting to a remote server via SSH is the foundation of daily work for system administrators, DevOps engineers, and developers. When a connection fails, work stops. SSH errors often have similar root causes: network unavailability, incorrect daemon settings, authentication problems, or firewall blocking. This guide offers a structured, step-by-step diagnostic approach to help you quickly find and fix the root cause, rather than just "restart the service and hope."

After completing this guide, you will be able to independently diagnose any SSH connection issue in Linux, from checking the network path to analyzing logs.

Prerequisites / Preparation

Before starting diagnostics, ensure you have:

1. Access to the server console (KVM, IPMI, virtual console in your hosting provider's control panel). This is critical if the problem has blocked your primary SSH access.
2. Superuser privileges (sudo) on the target server to check logs, configuration, and manage the firewall.
3. A client machine with the OpenSSH client (ssh, scp, ssh-keygen) installed.
4. Basic knowledge of network commands (ping, nc, ss) and Linux file permission structure.

Step 1: Checking Basic Network Accessibility

First, you need to understand whether your network packet reaches the server and if the server is listening on the correct port (default 22).

1. Check host availability:

   ping <server_IP_or_domain>
   

   
   If ping fails, the issue is at the network level (routing, firewall on an intermediate node, server is down).
2. Check if the SSH daemon is listening on the port and if it's open externally: On the server (via console), run:

   # See which port sshd is listening on and on which interface
   sudo ss -tlnp | grep :22
   # Or
   sudo netstat -tlnp | grep :22
   

   
   Expected result: 0.0.0.0:22 or :::22 (listening on all interfaces). If it's 127.0.0.1:22 — the daemon is only listening on localhost, and you need to fix sshd_config.
   From the client, try checking port openness with nc (netcat):

   nc -zv <server_IP> 22
   

   
   Successful result: Connection to <IP> 22 port [tcp/ssh] succeeded!. If Connection refused — the port is closed (daemon not listening or blocked by firewall). If Operation timed out — packets are being dropped (firewall is resetting the connection or there's a routing issue).

⚠️ Important: Some cloud providers (AWS, GCP, DigitalOcean) use Security Groups / Firewall Rules at the hypervisor level. Ensure those rules allow inbound traffic on port 22/TCP from your IP address.

Step 2: Analyzing SSH Daemon Logs on the Server

Logs are the most valuable source of information. Connect to the server console and check the system log.

• For Debian/Ubuntu:

  sudo tail -f /var/log/auth.log
  

• For RHEL/CentOS/Rocky/AlmaLinux:

  sudo tail -f /var/log/secure
  

Now, from the client, try the failed connection. Watch the logs in real time. Typical entries and their meanings:

• Failed password for <user> from <client_ip> port <port> ssh2 — incorrect password (if password authentication is enabled).
• Invalid user <user> from <client_ip> — login attempt for a non-existent user.
• Connection closed by <client_ip> port <port> [preauth] — client disconnected before authentication (often due to key mismatch or timeout).
• Authentication failed. — general authentication failure (key, password, GSSAPI).
• POSSIBLE BREAK-IN ATTEMPT! — too many failed attempts; the daemon may temporarily block the IP (if MaxAuthTries or fail2ban is configured).

What to look for: exact attempt time, client IP, username, and the specific error line.

Step 3: Validating SSH Daemon Configuration and Access Permissions

Incorrect settings in /etc/ssh/sshd_config or file permissions are a common cause of access denial.

1. Check the main config:

   sudo cat /etc/ssh/sshd_config | grep -E "^(PermitRootLogin|PasswordAuthentication|PubkeyAuthentication|AllowUsers|DenyUsers)"
   

   
   Key parameters:
   • PasswordAuthentication — yes or no. If no, a working public key is required.
   • PubkeyAuthentication — should be yes for key-based auth.
   • PermitRootLogin — prohibit-password (key-only), yes, or no.
   • AllowUsers / AllowGroups — if set, only listed users/groups can connect.
   • Port — non-standard port (if changed, connect via it: ssh -p <port> user@host).
   
   After any changes, restart the daemon:

   sudo systemctl restart sshd
   # Or for old sysvinit:
   # sudo service ssh restart
   

2. Check permissions on user files on the server: The SSH daemon is very strict about permissions. Connect to the console and check for the target user (e.g., ubuntu):

   # Permissions on home directory
   ls -ld /home/ubuntu
   # Expected: drwxr-xr-x (755) or drwx------ (700). Should NOT be writable by group/others (e.g., 777).
   
   # Permissions on .ssh folder
   ls -ld /home/ubuntu/.ssh
   # Expected: drwx------ (700)
   
   # Permissions on authorized_keys file
   ls -l /home/ubuntu/.ssh/authorized_keys
   # Expected: -rw------- (600)
   

   
   If permissions are too broad, fix them:

   sudo chmod 700 /home/ubuntu/.ssh
   sudo chmod 600 /home/ubuntu/.ssh/authorized_keys
   sudo chown -R ubuntu:ubuntu /home/ubuntu/.ssh
   

Step 4: Checking Firewall Rules and SELinux/AppArmor

The server's firewall can block incoming connections on port 22.

1. Check active rules:
   • firewalld (RHEL/CentOS 7+, Fedora):

     sudo firewall-cmd --list-all
     

     In the output, the services: or ports: section should include ssh or 22/tcp. If not, add it:

     sudo firewall-cmd --permanent --add-service=ssh
     sudo firewall-cmd --reload
     

   • ufw (Ubuntu/Debian default):

     sudo ufw status verbose
     

     Should show 22/tcp (SSH) ALLOW IN. If not:

     sudo ufw allow ssh
     sudo ufw reload
     

   • iptables (legacy):

     sudo iptables -L INPUT -n -v | grep 22
     

     Look for an ACCEPT rule for port 22. If none exists or it's set to DROP, add:

     sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
     # Save rules depending on distro (iptables-save, netfilter-persistent)
     

2. Check SELinux (RHEL/CentOS/Fedora) or AppArmor (Ubuntu/Debian):
   • SELinux:

     getenforce
     

     If Enforcing, check the security context of the .ssh folder:

     ls -laZ /home/ubuntu/.ssh
     

     Correct context for a user's home directory is unconfined_u:object_r:user_home_t:s0. For authorized_keys it's ssh_home_t. If the context is wrong, restore it:

     sudo restorecon -Rv /home/ubuntu/.ssh
     

   • AppArmor: Usually doesn't interfere with SSH, but a profile might be in enforce mode. Check:

     sudo aa-status | grep ssh
     

     If there are issues, try setting the profile to complain mode for testing:

     sudo aa-complain /etc/apparmor.d/usr.sbin.sshd
     

Step 5: Client-Side and Key Diagnostics

If the server is fine, the problem might be on the client side.

1. Ensure you are using the correct user and host:

   # Basic command
   ssh user@server_ip
   # If using a non-standard port
   ssh -p 2222 user@server_ip
   

2. Enable verbose logging on the client:

   ssh -vvv user@server_ip
   

   
   The -v flags (up to three) will show the entire chain: key-based auth attempt, password, GSSAPI. Look for lines like debug1: Authentications that can continue: and debug1: Offering public key: .... If your key isn't offered, the problem is its location or permissions. If it is offered but the server rejects it — check the server logs (Step 2).
3. Check permissions on local key files:

   ls -la ~/.ssh/
   

   
   Permissions on the private key (e.g., id_rsa) must be 600 (-rw-------). The .ssh folder should be 700. The public key (id_rsa.pub) can be 644, but 600 is better.
4. Ensure the public key is added on the server: On the client, display the public key contents:

   cat ~/.ssh/id_rsa.pub
   

   
   Copy the output and compare it with the contents of ~/.ssh/authorized_keys on the server. The line must match exactly (no extra spaces or line breaks). It's best to rewrite the key on the server:

   # On client (copy key to clipboard)
   cat ~/.ssh/id_rsa.pub | xclip -selection clipboard  # for Linux with xclip
   # Or just open the file and copy manually.
   
   # On server (via console)
   echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..." >> /home/ubuntu/.ssh/authorized_keys
   # Replace ... with the actual key string. Ensure the key is appended on a new line.
   

5. Check if fail2ban or similar is blocking you: If you repeatedly tried to connect with a wrong password/key, your IP might be blocked.

   sudo fail2ban-client status sshd
   sudo fail2ban-client set sshd unbanip <your_client_IP>
   

Verifying the Result

After applying all necessary fixes, try connecting again with the standard command:

ssh user@server_ip

If using a non-standard port, don't forget the -p flag. A successful connection will end at the server's command prompt (user@server_ip:~$).

Success criteria:

• Connection is established.
• You see the remote server's shell prompt (bash, zsh).
• No Permission denied, Connection refused, or Connection timed out errors.

Common Issues

• ssh: connect to host ... port 22: Connection timed out
  • Cause: Network packet isn't reaching. Check ping, routing (traceroute), cloud Security Group rules, and local/intermediate firewalls.
• ssh: connect to host ... port 22: Connection refused
  • Cause: SSH daemon isn't listening on port 22 on an external interface. Check sshd_config (ListenAddress setting) and ensure the daemon is running (sudo systemctl status sshd).
• Permission denied (publickey,password).
  • Cause: Authentication failed. Check server logs (/var/log/auth.log). Verify the key is correct, permissions on ~/.ssh and authorized_keys are proper, and that sshd_config allows the intended method (PubkeyAuthentication yes or PasswordAuthentication yes).
• Received disconnect from ...: 2: Too many authentication failures
  • Cause: SSH client offered too many keys (from ssh-agent or defaults), and the server refused. Solution: explicitly specify the key (ssh -i ~/.ssh/id_rsa user@host) or disable the agent for this host in ~/.ssh/config:

    Host your-server
        IdentitiesOnly yes
        IdentityFile ~/.ssh/id_rsa
    

• WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
  • Cause: The host's fingerprint changed (server reinstalled, IP reassigned). Do not ignore this warning! Remove the old record from ~/.ssh/known_hosts (find the line with the IP/domain and delete it) and reconnect.

Additional Resources

• man sshd_config — official documentation for the daemon's configuration file.
• man ssh — documentation for the client and its parameters (-v, -i, -p).
• man ssh-keygen — managing SSH keys (generation, conversion).
• OpenSSH Documentation: https://www.openssh.com/manual.html
• SSH Hardening Guide: /guides/linux/openssh-hardening
• Linux Firewall Troubleshooting: /guides/linux/firewall-configuration