Setting up a VNC Server on Linux: A Step-by-Step Guide

Introduction / Why This Is Needed

VNC (Virtual Network Computing) is a protocol for remote access to a graphical desktop. A configured VNC server on Linux allows you to control the system from anywhere on the network, as if you were sitting in front of it. This is useful for server administration, working with GUI applications, or accessing your workstation from outside. In this guide, you will learn how to deploy a VNC server on popular Linux distributions.

Prerequisites / Preparation

Before you begin, ensure that:

• You have a Linux system with an installed graphical environment (X11). Wayland may require additional configuration.
• You have administrator privileges (sudo) or access to the root account.
• The system is connected to the network, and you know its IP address (use ip addr or hostname -I).
• A VNC client is installed on the remote computer (e.g., TigerVNC Viewer, TightVNC Viewer, or RealVNC Viewer).

Step 1: Installing the VNC Server

We will use TigerVNC—a modern and well-supported implementation. Install the package using your distribution's package manager.

For Ubuntu/Debian:

sudo apt update
sudo apt install tigervnc-standalone-server tigervnc-common

For CentOS 7/RHEL 7:

sudo yum install tigervnc-server

For CentOS 8+/RHEL 8+/Fedora:

sudo dnf install tigervnc-server

For Debian 11/12:

sudo apt update
sudo apt install tigervnc-standalone-server

After installation, verify the vncserver command is available:

vncserver -version

Step 2: Setting Up the Password and Desktop Environment

Setting the Password

Run vncpasswd to create an access password. Execute the command as the user who will run the VNC sessions (usually your regular user, not root):

vncpasswd

You will be prompted to enter and confirm the password (6-8 characters). Optionally, you can set a view-only password (viewing without control). Remember the password—you will need it when connecting.

Configuring the Desktop Environment

The VNC server needs to know which graphical environment to launch. Create or edit the file ~/.vnc/xstartup (for a user session) or the global config /etc/vnc/xstartup. Example for GNOME (the default in Ubuntu):

#!/bin/bash
xrdb $HOME/.Xresources
xsetroot -solid grey
export XKL_XMODMAP_DISABLE=1
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
gnome-session &

For KDE Plasma, replace the last line with startplasma-x11 &, and for XFCE, use startxfce4 &.

Make the file executable:

chmod +x ~/.vnc/xstartup

You can also create a configuration file ~/.vnc/config for server parameters (optional):

geometry=1920x1080
depth=24
localhost=no
securitytypes=vncauth

Here, geometry is the screen resolution, depth is the color depth, localhost=no allows connections from outside (by default, only localhost), and securitytypes is the authentication method.

Step 3: Configuring the systemd Service

To start the VNC server automatically at boot, create a systemd unit. Typically, the template vncserver@.service is used, where @ is replaced by the display number (e.g., :1).

Create or edit the file /etc/systemd/system/vncserver@.service (requires sudo):

[Unit]
Description=Remote desktop service (VNC)
After=syslog.target network.target

[Service]
Type=forking
ExecStartPre=-/usr/bin/vncserver -kill :%i > /dev/null 2>&1
ExecStart=/usr/bin/vncserver -depth 24 -geometry 1920x1080 :%i
ExecStop=/usr/bin/vncserver -kill :%i
User=your_username
Group=your_username

[Install]
WantedBy=multi-user.target

Replace your_username with your Linux username (not root). The -depth and -geometry parameters can be adjusted to your needs or moved to ~/.vnc/config.

Then run:

sudo systemctl daemon-reload
sudo systemctl enable vncserver@1.service  # for display :1 (port 5901)
sudo systemctl start vncserver@1.service

Check the status:

sudo systemctl status vncserver@1.service

If you need to use a different display (e.g., :2 for port 5902), replace 1 with 2 in the commands.

Step 4: Configuring the Firewall

Open the VNC port in the firewall. By default, display :1 uses port 5901 (formula: 5900 + display number).

For ufw (Ubuntu/Debian):

sudo ufw allow 5901/tcp

For firewalld (CentOS/RHEL/Fedora):

sudo firewall-cmd --permanent --add-port=5901/tcp
sudo firewall-cmd --reload

⚠️ Important: Do not open the VNC port directly to the internet without additional protection. Use an SSH tunnel or VPN, and restrict access by IP through the firewall (e.g., sudo ufw allow from 192.168.1.0/24 to any port 5901).

Step 5: Connecting from a Client

On the remote computer (Windows, macOS, Linux, or mobile device), install a VNC client. In the client's address bar, enter:

server_IP_address:5901

or

server_IP_address:1

Where server_IP_address is the external or internal IP of your Linux machine. When connecting, enter the password set in Step 2. If everything is configured correctly, you will see the Linux desktop.

Verifying the Result

1. Ensure the VNC service is active: sudo systemctl status vncserver@1.service.
2. Check that the port is listening: ss -tlnp | grep 5901 (should show the Xvnc process).
3. Connect from the client and check:
   • Is the desktop displayed?
   • Do the mouse and keyboard work?
   • Can you launch a terminal and run commands (e.g., ls -la)?
4. Try rebooting the server and ensure VNC starts automatically.

Common Issues

Authentication Error or "Password failed"

• Ensure the password was set via vncpasswd for the correct user.
• Check that the ~/.vnc/passwd file exists and has correct permissions (chmod 600 ~/.vnc/passwd).
• If you changed the password, restart the service: sudo systemctl restart vncserver@1.service.

Blank or Black Screen After Connecting

• Check the VNC logs: cat ~/.vnc/*.log and cat ~/.vnc/*.log.old. Look for session startup errors.
• Ensure ~/.vnc/xstartup specifies the correct shell launch command (e.g., gnome-session & for GNOME).
• If the system uses Wayland by default (e.g., in Ubuntu 22.04), switch to Xorg on the login screen (gear icon → "Ubuntu on Xorg").

Port Not Open or Connection Refused

• Check the firewall: sudo ufw status or sudo firewall-cmd --list-all.
• Ensure the service is listening on the port: ss -tlnp | grep 5901.
• Check if a network interface or cloud firewall is blocking the connection (if the server is in the cloud, open the port in the cloud provider's console).

Low Resolution or Incorrect Screen Size

• Change the geometry parameter in ~/.vnc/config or in the service startup command (e.g., -geometry 1920x1080).
• Restart the service after changes.

Performance Issues

• Reduce color depth (depth=16 instead of 24) in the config for faster transfer.
• Use compression: add -compresslevel 9 to the service's ExecStart (if supported).
• Consider using an SSH tunnel for encryption and potential speed improvements in some networks.