Generating and Configuring SSH Keys on Linux: A Step-by-Step Guide

Introduction / Why This Is Needed

SSH keys are the standard for secure remote access to Linux servers. Instead of entering a password with each connection, you authenticate using a cryptographic pair: the private key stays with you, and the public key resides on the server. This is not only more convenient but also significantly more secure, as it eliminates password interception and protects against brute-force attacks.

After completing this guide, you will be able to:

• Connect to servers via SSH without entering a password.
• Enhance your server security by disabling password-based authentication.
• Manage multiple servers using a single key or a set of keys.

Prerequisites / Preparation

Before starting, ensure that:

1. You are working in the terminal of a local Linux machine (or WSL2 on Windows).
2. The sshd daemon is installed and running on the remote server (systemctl status sshd).
3. You have credentials (login and password) to access the target server via SSH.
4. The openssh-client package is installed locally (usually present by default).

Check if the client is available:

ssh -V
# Example output: OpenSSH_8.9p1, OpenSSL 3.0.2 15 Mar 2022

If the command is not found, install it:

• Ubuntu/Debian: sudo apt update && sudo apt install openssh-client
• CentOS/RHEL/Fedora: sudo dnf install openssh-clients
• Arch Linux: sudo pacman -S openssh

Step 1: Generate an SSH Key Pair

Modern systems recommend using the Ed25519 algorithm—it is fast and secure. If compatibility with very old systems (e.g., CentOS 6) is required, use RSA.

Option A (recommended, Ed25519):

ssh-keygen -t ed25519 -C "your_email@example.com"

Option B (compatibility, RSA 4096 bits):

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

The program will prompt you for:

1. Key save path—simply press Enter to use the default path (~/.ssh/id_ed25519 or ~/.ssh/id_rsa).
2. Passphrase—a password-like phrase for additional key protection. It is strongly recommended to set one if the key will be used on a laptop or workstation. You can leave it empty (press Enter twice), and the key will be used without a password.

After successful execution, two files will appear in ~/.ssh/:

• id_ed25519 (or id_rsa)—the private key. Keep it secret; never share it with anyone!
• id_ed25519.pub (or id_rsa.pub)—the public key. It is safe to send this to servers.

Step 2: Copy the Public Key to the Server (ssh-copy-id)

The simplest method is the ssh-copy-id utility. It:

1. Connects to the server using your current password.
2. Creates the ~/.ssh directory on the server if necessary.
3. Adds your public key to the ~/.ssh/authorized_keys file.
4. Sets the correct permissions.

Run the command:

ssh-copy-id user@server_ip

Example:

ssh-copy-id deploy@192.168.1.100

You will be prompted to enter the password for user deploy on the server. After successful copying, you will see the message Number of key(s) added: 1.

Alternative: Manual Copying

If ssh-copy-id is not available (rare), copy the key manually:

1. View the contents of your public key:

   cat ~/.ssh/id_ed25519.pub
   

   
   Copy the entire long string that starts with ssh-ed25519 AAAA....
2. Connect to the server:

   ssh user@server_ip
   

3. On the server, create the .ssh directory (if it doesn't exist) and add the key:

   mkdir -p ~/.ssh
   echo "copied_key_string" >> ~/.ssh/authorized_keys
   

4. Set strict permissions on the server (critically important!):

   chmod 700 ~/.ssh
   chmod 600 ~/.ssh/authorized_keys
   

Step 3: Configure ssh-agent for Automatic Operation

ssh-agent is a background process that stores your decrypted private key in memory. This allows you to avoid entering the passphrase with each connection after the first input in a session.

1. Start the agent (it is usually already running in modern desktop environments):

   eval $(ssh-agent -s)
   # Example output: Agent pid 12345
   

2. Add your private key to the agent:

   ssh-add ~/.ssh/id_ed25519
   

   
   If you set a passphrase, enter it now. The key will be loaded into the agent's memory.
3. (Optional) Autoload the key on session start
   Add the following lines to the end of your ~/.bashrc or ~/.profile file:

   # Start ssh-agent if not running
   if ! pgrep -u "$USER" ssh-agent > /dev/null; then
       eval "$(ssh-agent -s)"
   fi
   # Add default key if not already added
   ssh-add -l > /dev/null 2>&1 || ssh-add ~/.ssh/id_ed25519 2>/dev/null
   

   
   After this, the key will be automatically loaded each time you open a terminal.

Step 4: Test Passwordless Connection

Now try connecting to the server:

ssh user@server_ip

Expected result: You should land directly on the server's command line. There should be no password prompts (except possibly a passphrase prompt for the key upon first use in a session).

If the connection was successful, you can disable password authentication on the server for enhanced security:

1. Connect to the server.
2. Edit /etc/ssh/sshd_config:

   sudo nano /etc/ssh/sshd_config
   

3. Find the line #PasswordAuthentication yes and change it to:

   PasswordAuthentication no
   

4. Restart the SSH daemon:

   sudo systemctl restart sshd
   

Important: Before disabling password authentication, ensure your key works and that you have alternative access via another key or a cloud management console in case of an error!

Step 5: (Optional) Create a Configuration for Convenience

If you work with many servers, create a ~/.ssh/config file on your local machine:

nano ~/.ssh/config

Add entries in this format:

# Example for a production web server
Host web-prod
    HostName 192.168.1.100
    User deploy
    IdentityFile ~/.ssh/id_ed25519_web
    Port 22

# Example for a database
Host db-int
    HostName 10.0.5.20
    User dbadmin
    IdentityFile ~/.ssh/id_ed25519
    Port 2222  # if SSH listens on a non-standard port

Now you can connect using a short name:

ssh web-prod

The config also allows setting options like ProxyJump (jumping through a host) or ForwardAgent (agent forwarding).

Verification Checklist

1. Connection works without a password—the primary indicator.
2. Your public string is present in ~/.ssh/authorized_keys on the server (check with cat ~/.ssh/authorized_keys).
3. Permissions on the server: ~/.ssh = 700, ~/.ssh/authorized_keys = 600.
4. On the client: ~/.ssh = 700, id_rsa (private) = 600, id_rsa.pub = 644.
5. The ssh-add -l command shows keys loaded in the agent.

Troubleshooting

Error: Permission denied (publickey,password).

• Cause: The server does not see your key or the key does not match.
• Solution:
  1. Ensure the public key is fully and unaltered copied into ~/.ssh/authorized_keys on the server (one line!).
  2. Check permissions on the server: chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys.
  3. Verify the file owner is your user on the server (chown -R your_user:your_user ~/.ssh).
  4. Ensure PubkeyAuthentication yes is enabled in sshd_config on the server.

Error: Bad permissions: ignore key: ...

• Cause: Overly permissive permissions on the local private key.
• Solution: chmod 600 ~/.ssh/id_ed25519 (or id_rsa).

Error: The authenticity of host '...' can't be established.

• Cause: You are connecting to the server for the first time, and its fingerprint is unknown.
• Solution: This is a warning, not an error. If you trust the IP/domain, type yes. The host key will be saved in ~/.ssh/known_hosts.

Key Not Added to ssh-agent Automatically

• Solution: Check if the key is added: ssh-add -l. If not, add it manually with ssh-add ~/.ssh/id_ed25519. For autoload, verify settings in ~/.bashrc (see Step 3).

ssh-copy-id Fails (Connection refused)

• Cause: The server is not accepting SSH connections (port closed, firewall, service not running).
• Solution: Check if port 22 (or another) is accessible from your IP: nc -zv server_ip 22. Ensure sshd is running on the server: sudo systemctl status sshd. Check the firewall (sudo ufw status or sudo firewall-cmd --list-all).