Solving SSH Connection Issues in Linux: A Complete Guide

Introduction / Why This Is Needed

SSH (Secure Shell) is the primary tool for remote management of Linux servers. Connection issues can paralyze operations: you won't be able to deploy an application, update the system, or even regain access after a failure. This guide systematizes troubleshooting, starting from basic network checks and ending with log analysis. You'll get a clear action plan that works on most distributions (Ubuntu, Debian, CentOS, Arch).

Prerequisites / Preparation

1. Console access on the target server (via KVM, iLO, hosting panel console) or access from another working node on the network.
2. sudo or root privileges on the server to restart services and view logs.
3. The openssh-client package installed on your local machine.
4. Basic understanding of the Linux command line.

Step 1: Checking Basic Network Accessibility

Before diving into SSH configuration, ensure the server is reachable over the network and the SSH port is open.

# 1. Check if the host responds (replace server_ip with the IP or domain)
ping -c 4 server_ip

# 2. Check if the SSH port (default 22) is open using netcat (nc)
# Install nc if missing: sudo apt install netcat (Debian/Ubuntu) or sudo yum install nc (RHEL/CentOS)
nc -zv server_ip 22

# Alternative with nmap (if installed)
nmap -p 22 server_ip

• nc result: Connection to server_ip 22 port [tcp/ssh] succeeded! — port is open.
• nc result: Connection refused — port is closed or service is not listening.
• nc result: No route to host or timeout — issues at the network level (firewall, routing, host is down).

⚠️ Important: If ping fails, the problem is not with SSH but with the network. Check IP settings, gateway, firewall on the local machine, and intermediate devices.

Step 2: Diagnosing the SSH Service on the Server

If the port is open but the connection isn't established, check if the sshd daemon is running.

# 1. Check service status (for systemd)
sudo systemctl status sshd

# 2. Find out which port(s) the service is listening on
sudo ss -tlnp | grep sshd
# Or
sudo netstat -tlnp | grep sshd

Common scenarios:

• Service is inactive (inactive): Start it: sudo systemctl start sshd and enable autostart: sudo systemctl enable sshd.
• Service listens only on 127.0.0.1:22 instead of 0.0.0.0:22 (or :::22): This means sshd is configured to accept connections only locally. Check the ListenAddress parameter in the config (see Step 3).
• Service listens on a non-standard port (e.g., 2222): Either change the port on the client (ssh -p 2222 user@host) or revert to the standard port in the config.

Step 3: Analyzing SSH Server Configuration

The main configuration file is /etc/ssh/sshd_config. Make changes cautiously and create a backup: sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup.

# View current configuration, ignoring comments
sudo cat /etc/ssh/sshd_config | grep -v "^#" | grep -v "^$"

Critical parameters to check:

After any config change, restart the service:

sudo systemctl restart sshd

Step 4: Checking SSH File and Directory Permissions (for Key-Based Authentication)

The SSH server is very strict about file permissions. Incorrect permissions are a frequent cause of Permission denied (publickey).

On the server (as the user you're trying to log in as):

# 1. Check permissions on the user's home directory
ls -ld ~

# 2. Check permissions on the .ssh directory
ls -ld ~/.ssh

# 3. Check permissions on the authorized_keys file
ls -l ~/.ssh/authorized_keys

Correct permissions:

• Home directory (/home/username): drwxr-xr-x (755) or stricter.
• .ssh directory: drwx------ (700).
• authorized_keys file: -rw------- (600).
• Owner of all these objects should be the user logging in.

Fixing permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
# Sometimes helps:
chown -R username:username ~/.ssh

💡 Tip: If you just generated a key and copied it to the server, ensure you copied the public key (file id_ed25519.pub or id_rsa.pub), not the private one, and that it was added to authorized_keys as a single line.

Step 5: Analyzing SSH Logs

Logs are the most reliable source of truth. Where to look depends on the distribution.

# For RHEL/CentOS/Fedora/AlmaLinux/Rocky Linux and derivatives:
sudo tail -f /var/log/secure

# For Debian/Ubuntu and derivatives:
sudo tail -f /var/log/auth.log

# Universal method via systemd journal (works everywhere):
sudo journalctl -u sshd -f

What to look for in logs:

• Failed password for ... — password error.
• Connection closed by ... — client disconnected, often due to key issues.
• Authentication refused: bad ownership or modes for directory /home/... — incorrect permissions (see Step 4).
• User ... not allowed because not listed in AllowUsers — user is denied in config.
• Received disconnect from ...: 11: Bye Bye — may indicate a session or key problem.
• Invalid user ... — attempt to log in as a non-existent user.

Example for quick filtering by IP or user:

sudo journalctl -u sshd | grep "192.168.1.100"
sudo journalctl -u sshd | grep "user_login"

Step 6: Testing Connection with Debugging (Client Side)

Run the connection with maximum verbosity. This will show exactly where the failure occurs.

# Maximum detailed logs (3 levels of -v)
ssh -vvv user@server_ip

What to notice in the output:

1. debug1: Connecting to server_ip [server_ip] port 22. — client found host and port.
2. debug1: Connection established. — TCP connection succeeded.
3. debug1: Authenticating to server_ip:22 as 'user' — start of authentication.
4. debug1: Offering public key: /home/you/.ssh/id_ed25519 ... — client offers a key. If no key is offered, maybe ssh-agent isn't running or the key isn't added (ssh-add -l).
5. debug1: Authentications that can continue: publickey,password — server lists accepted authentication methods.
6. debug1: Offering public key: ... repeats, then debug1: Authentications failed. — server rejected the key (server-side issue: key not in authorized_keys, wrong permissions, SELinux).
7. If followed by debug1: Next authentication method: password, the server accepted a fallback to password (if PasswordAuthentication yes).

Client options for testing:

# Disable key usage (test password authentication)
ssh -o PubkeyAuthentication=no user@server_ip

# Specify a particular key
ssh -i ~/.ssh/my_specific_key user@server_ip

# Disable known_hosts check (dangerous, for testing only!)
ssh -o StrictHostKeyChecking=no user@server_ip

Verifying Success

A successful connection will end with the remote server's command prompt. In the client logs (-vvv), you'll see:

debug1: Authentication succeeded (publickey).
Authenticated to server_ip ([server_ip]:22).

On the server in the logs, an entry will appear:

Accepted publickey for user from client_ip port client_port ssh2: ED25519 SHA256:...

Common Issues

Additional Diagnostic Methods

If standard steps didn't help, check the following:

1. Checking the server firewall (UFW example):

   sudo ufw status numbered
   # If a rule for 22/tcp is missing, add it:
   sudo ufw allow 22/tcp
   

2. Checking SELinux/AppArmor:

   # For SELinux (RHEL/CentOS/Fedora)
   sudo sestatus
   sudo ausearch -m avc -ts recent | grep sshd
   # For AppArmor (Debian/Ubuntu)
   sudo aa-status | grep sshd
   

3. Checking /etc/hosts.allow and /etc/hosts.deny: These files can block access. Ensure sshd: ALL or your IP is in hosts.allow, and sshd: ALL is not in hosts.deny.
4. Checking server resources: If the server is overloaded (out of memory, disk space), sshd may not accept new connections. Check: free -h, df -h, top.
5. Test from another client: Try connecting from a different machine. If it works — the problem is with the original client (its keys, ~/.ssh/config, firewall).
6. Temporarily increase logging level in sshd_config: Uncomment or add LogLevel DEBUG and restart sshd. Remember to revert to INFO after diagnostics!

sudo sed -i 's/^#LogLevel.*/LogLevel DEBUG/' /etc/ssh/sshd_config
sudo systemctl restart sshd
# ... perform connection attempt ...
# Revert:
sudo sed -i 's/^LogLevel.*/LogLevel INFO/' /etc/ssh/sshd_config
sudo systemctl restart sshd

Conclusion

SSH troubleshooting is a sequential process of elimination: network -> service -> config -> permissions -> logs. Start with simple checks (ping, nc, systemctl status sshd), then delve into configuration and permissions. Logs (journalctl -u sshd or /var/log/auth.log) almost always contain the specific reason for failure. Use the client-side debug mode (ssh -vvv) to see the process from the other side. After resolving the issue, don't forget to restore security settings (e.g., LogLevel and PermitRootLogin) to their original state.