Setting Up SSH on Linux: Key Generation and Usage

Introduction / Why This Is Needed

Connecting to a remote Linux server via SSH (Secure Shell) is the de facto standard for administration. Using cryptographic keys instead of a password significantly enhances security, protecting against brute-force attacks and enabling the configuration of automated tasks (e.g., via cron or CI/CD pipelines). After completing this guide, you will be able to connect to the server by entering only the command ssh user@host, without being prompted for a password.

Prerequisites / Preparation

Before you begin, ensure that:

1. You have SSH access to the target server (currently via password or another method).
2. The openssh-client package is installed on your local computer (client). It is usually present by default in Linux and macOS. In Windows 10/11, you can use the built-in OpenSSH client or WSL.
3. The sshd daemon (typically from the openssh-server package) is running and configured on the target server. You can check this with the command:

   sudo systemctl status sshd
   

4. You know the username and IP address or domain name of the server.

Step-by-Step Guide

Step 1: Generate an SSH Key Pair on the Client

Open a terminal on your local machine. It is recommended to use the modern and secure Ed25519 algorithm. If you have an older system, you can use RSA (minimum 3072 bits).

ssh-keygen -t ed25519 -C "your@email.com"

• -t ed25519 — specifies the algorithm type.
• -C "comment" — an arbitrary comment (often an email), added to the end of the public key for identification.

You will be prompted to:

1. Specify a path to save the keys. The default is ~/.ssh/id_ed25519 (private) and ~/.ssh/id_ed25519.pub (public). Press Enter to use the default path.
2. Enter a passphrase. This is optional, but highly recommended for additional protection of the private key. If you are setting up the key for automated tasks, leave the field empty (press Enter twice).

After execution, two files will appear in ~/.ssh/. Never share your private key (id_ed25519) with anyone!

Step 2: Copy the Public Key to the Server

There are several ways to do this. The simplest is to use the ssh-copy-id utility.

ssh-copy-id username@server_ip_address

• Replace username with the username on the server (e.g., ubuntu, root, debian).
• Replace server_ip_address with the server's IP address or domain.

You will be asked to enter the password for that user on the server. The utility will automatically:

1. Create the ~/.ssh directory on the server (if it doesn't exist) with permissions 700.
2. Append the contents of your local ~/.ssh/id_ed25519.pub file to the end of the ~/.ssh/authorized_keys file on the server.
3. Set the correct permissions on the files (authorized_keys — 600).

Alternative manual method (if ssh-copy-id is unavailable):

# 1. Output the contents of the public key on the local machine
cat ~/.ssh/id_ed25519.pub

# 2. Copy the output (starting with "ssh-ed25519 ...") to the clipboard.
# 3. On the server (connected via password), run:
mkdir -p ~/.ssh
echo "copied_key_string" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Step 3: Check Server Configuration (Optional but Important)

For the server to accept key-based authentication, the correct parameters must be set in its main configuration file /etc/ssh/sshd_config. Connect to the server (if not already connected) and check:

sudo nano /etc/ssh/sshd_config

Ensure the lines look roughly like this (value yes):

PubkeyAuthentication yes
PasswordAuthentication no   # <-- Recommended to disable after successfully setting up keys!

Important: Do not disable PasswordAuthentication until you are sure key-based login works for all necessary users! Otherwise, you may lose access to the server.

After making changes, restart the SSH daemon:

sudo systemctl restart sshd

Step 4: Test the Key-Based Connection

Now try connecting again:

ssh username@server_ip_address

If everything is configured correctly:

• You will not be prompted for the username user's password on the server.
• If you set a passphrase when generating the key, the system will ask for it (this is local, on your machine). This is normal.
• You will reach the server's command line.

Step 5: (Optional) Client Configuration for Convenience

You can create or edit the ~/.ssh/config configuration file on your local computer to avoid entering the IP address and username each time.

Example contents:

Host myserver
    HostName 192.168.1.100
    User admin
    IdentityFile ~/.ssh/id_ed25519
    Port 22

Now you can connect with just:

ssh myserver

Verification

1. Successful connection: You see the remote server's welcome message (e.g., Welcome to Ubuntu 22.04...) and a command prompt.
2. Check authentication on the server: On the server, run:

   sudo tail -f /var/log/auth.log
   

   Then, in another window, try to connect. In the log, you will see a line with Accepted publickey and your username.
3. Check if password is disabled (if you did that): Try connecting with an incorrect password (if prompted). The connection should be rejected with an error like Permission denied (publickey,password).

Troubleshooting

❌ Error: Permission denied (publickey)

• Cause 1: Incorrect permissions on files in ~/.ssh on the server.
  • Solution: On the server, run:

    chmod 700 ~/.ssh
    chmod 600 ~/.ssh/authorized_keys
    

    Ensure the owner of these files and the directory is the target user (not root).
• Cause 2: The public key was copied with errors (extra spaces, line breaks).
  • Solution: Check the ~/.ssh/authorized_keys file on the server. The key must be a single line starting with ssh-ed25519 AAAA....
• Cause 3: SELinux (on RHEL/CentOS/Fedora) is blocking access.
  • Solution: Restore the correct SELinux context:

    restorecon -Rv ~/.ssh
    

❌ Error: Connection refused or No route to host

• Cause: The server is not listening on port 22 (the default SSH port) or is unreachable on the network.
  • Solution:
    1. Check if sshd is running: sudo systemctl status sshd.
    2. Check if the server is listening on the port: sudo ss -tlnp | grep :22.
    3. Check the server's firewall (sudo ufw status or sudo firewall-cmd --list-all). Allow port 22 (or your custom port).

❌ Error: ssh: Could not resolve hostname ...

• Cause: The hostname or IP address is specified incorrectly.
  • Solution: Verify the DNS name or use a direct IP address.

❌ Connection works, but sudo on the server requires a password

• Cause: The sudoers settings do not allow passwordless operation for this user.
  • Solution: This is a separate sudo configuration topic. Configure the /etc/sudoers file via visudo, adding the line username ALL=(ALL) NOPASSWD: ALL (be careful, this grants full privileges without a password).