Web server security – Part 1: Basic hardening

Web server security – Part 1: Basic hardening

A web server is just another computer and requires basic security configuration. In the first part of our series, we show you how to harden your SSH access and configure your firewall. You can use this configuration for any type of server.

Contents

  1. Requirements
  2. Step by step guide for basic server security
  3. Summary
  4. External files
  5. Changelog

Always stay in the loop!
Subscribe to our RSS/Atom feeds.

Requirements

It is important to understand that your own server means responsibility. A server is just another computer exposed to the internet. An insecure server can easily become a target for automated attacks and quickly join another botnet.

So, before proceeding, read part 0 of this series. Write down your basic idea and a security concept. Document important steps. Do not blindly implement or configure anything. Blindly changing configuration files and hoping that it is somehow secure isn’t a good approach.

For this part, we need:

  • a virtual or physical server
  • an operating system of your choice (We will use Debian 9 in this series. Most steps stay the same if you use Debian 10 or newer.)
  • basic knowledge of shell commands
  • an SSH client on your client (as we assume that we use OpenSSH to connect to the server)
  • a security token (YubiKey, Nitrokey) or an app (FreeOTP) with OATH-TOTP support

Finally, we will use nmap to check our configuration. You can use other or more tools, of course.

Warning
If you are totally new to setting up a server, do not proceed! You are responsible for its security. Securing a server includes more than only some arbitrary configuration. Read this section again.

Step by step guide for basic server security

First of all, you have to install a server operating system with OpenSSH. OpenSSH is oftentimes included in minimal images of operating systems. In the following, we always mean OpenSSH when talking about SSH.

Info
As an experienced and advanced user, you can write your custom Ansible Playbook or use other tools to automate your server setup. This is really useful in case of any serious errors or lockout.

After installing the operating system on your server, you can use SSH to connect to it. Open a terminal on your client and type ssh root@[IP-address-of-your-server]. When connecting for the first time, SSH will ask for confirmation that you actually try to connect to the right server. Check this and type yes, if anything looks good. After confirming, you have to enter your initial password. The initial SSH password should be provided by your server hosting company or you already chose one before.

Step 0: Check for updates

When you are connected to your server for the first time, use Debian’s apt (Advanced Package Tool or the package manager of your OS) to check for updates: sudo apt update. Install any updates and reboot your server: sudo apt upgrade and sudo reboot.

Step 1: Install and configure ufw (your firewall)

Download and install ufw (Uncomplicated Firewall) since ufw is one of the easiest ways to configure your firewall: sudo apt install ufw.

Then, you can check the status of ufw: sudo ufw status which will most likely show Status: inactive.

Change the default rules to:

  • sudo ufw default deny incoming
  • sudo ufw default allow outgoing

After this, you have to allow access to the port used by SSH. This is normally port 22. Allow this by entering: sudo ufw allow ssh. If you plan to change its port, set the actual port number here.

Keep in mind that everyone on the internet will be able to access the SSH service afterwards. Every possible IP address can connect to port 22 of your server. There are hundreds of automated bots on the internet which try to connect to port 22 in order to break into your server. We observed these bots for two hours and saw about 8,000 login attempts. A good practice is to enable SSH just for your personal IP address:

  • sudo ufw allow from [IP-address-of-your-client] to any port 22

The problem is that your ISP may change your public IP address from time to time. This means that you are unable to connect to your server until you get your initially-configured IP address back. You can observe your IP address for several days and most likely you will see that it doesn’t change much. So, you likely want to allow a subnet to connect with your server:

  • sudo ufw allow from [IP-address-of-your-client]/[subnet] to any port 22

This is the common CIDR notation. Your suffix [subnet] will be “24” to “26” in most cases. This drastically restricts the number of hosts which can connect to your port 22. Regularly recheck the list of whitelisted subnets and remove subnets if they aren’t in use anymore.

Finally, you have to start and enable ufw: sudo ufw enable. Confirm that this command may disrupt existing SSH connections by entering y. You should see Firewall is active and enabled on system startup.

Please note that we only allowed connections to port 22 so far. Later, we will also allow access to the ports 80 (HTTP) and 443 (HTTPS), so people can connect to your website. If you don’t plan to run a web server but other server software, only allow the port numbers needed for this software.

Step 2: Create a non-root account

A basic security measure of all operating systems is to use non-root accounts for the most time. This ensures that an attacker doesn’t get all privileges if he exploits a program which runs in a non-root context.

On Debian, enter sudo adduser [username]. You will be prompted for a new password twice. Skip the user information by pressing “Enter” several times if you are the only user on this server and confirm this by pressing Y.

The new user doesn’t have the possibility to get administrative rights by default. To change this, you have to add the new user to the sudo group (other operating systems call this group wheel). Enter sudo usermod -aG sudo [username] to add the new user to the sudo group. You can see all groups of a user by entering sudo groups [username]. In some cases, you also have to install sudo itself: sudo apt install sudo.

Finally, switch to your new account: sudo su [username]. From now on, you have to explicitly enter “root mode” by preceding each command with sudo, if necessary. Don’t use “sudo” every time, but only if needed. Your operating system will clearly indicate that you have to use “sudo” for some commands.

Step 3: Harden your SSH configuration

By default, SSH allows you to log in as “root” and you only need a password. It is considered more secure to allow only non-root users and use cryptographic keys instead of passwords. Furthermore, there are dozens of legacy algorithms allowed by default which we disable.

Step 3a: Switch from passwords to keys

You have to decide whether you want to use RSA keys or newer Ed25519 keys. We recommend Ed25519 keys. Their key size is smaller and computers can use them more efficiently.

Generate a key pair on your local client:

  • ssh-keygen -b 4096 -t rsa -f ~/.ssh/id_[servername] (4096 bit RSA key pair), or
  • ssh-keygen -t ed25519 -f ~/.ssh/id_[servername] (256 bit Ed25519 key pair)

This will generate either a RSA or Ed25519 key pair and store it in your local “~/.ssh/” folder. You can optionally enter a password. This password isn’t related to the SSH connection, but used to encrypt your local private key using 128 bit AES.

After this step, you will find two files in your “~/.ssh/” folder:

  • id_[servername] (private key; keep it secret!)
  • id_[servername].pub (public key; copy it to your server!)

Now, you have to copy the public key to the server by using the following command:

  • ssh-copy-id -i ~/.ssh/id_[servername].pub [username]@[IP-address-of-your-server]

You will have to enter the password of [username] on your server. If anything worked as expected, you see: “Number of key(s) added: 1”.

Open a second terminal window and try to connect to your server without logging out before: ssh -i ~/.ssh/id_[servername] [username]@[IP-address-of-your-server]. If configured correctly, there will be no password prompt since your new key is used for authentication. In case of any errors, do not proceed but fix the issues.

Warning
In case of any errors, do not proceed but fix the issues. Otherwise, you probably lose SSH access to your server.

Step 3b: Configure sshd

When you successfully checked that your new key is in use, connect to your server using SSH as before.

Open the configuration file of sshd: /etc/ssh/sshd_config. Be sure that you actually open sshd_config. ssh_config (no d) is the configuration file for clients.

Change the file accordingly:

  • #PasswordAuthentication yesPasswordAuthentication no (disable password-based login)
  • PermitRootLogin yesPermitRootLogin no (disable root login)
  • #LogLevel INFOLogLevel VERBOSE (log login attempts)

Below # Ciphers and keying add:

1
2
3
4
5
6
# Ciphers and keying
#RekeyLimit default none
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com
HostKeyAlgorithms ssh-ed25519,rsa-sha2-256,rsa-sha2-512,ssh-rsa-cert-v01@openssh.com

Below # Authentication: add:

1
2
3
# Authentication:
AllowUsers [username]
AuthenticationMethods publickey

Now, restart sshd: sudo systemctl restart sshd (or use the command of your operating system).

Again, open a second terminal window and try to connect to your server without logging out before: ssh -i ~/.ssh/id_[servername] [username]@[IP-address-of-your-server]. If configured correctly, you are logged in without any prompts or errors.

Warning
Again: In case of any errors, do not proceed but fix the issues. Otherwise, you probably lose SSH access to your server.

Step 4: Enable 2FA for SSH

For even more security, we will enable two-factor authentication using OATH-TOTP. This step isn’t absolutely necessary but addresses the risk of losing your private SSH key. An attacker who only gets your private SSH key (and the password to decrypt it) won’t be able to access your server using SSH without the correct one-time password.

If you want to set up 2FA, you either need an app on your smartphone or a security token as described above. Hold this device ready and connect to your server.

First of all, we have to install libpam-google-authenticator. This is a module for PAM (pluggable authentication module). PAM is a core component of Linux-based operating systems. On Debian, type: sudo apt install libpam-google-authenticator. This also installs the dependency libqrencode3 (or libqrencode4 for Debian 10).

After installing, start its configuration. Enter google-authenticator. When asked “Do you want authentication tokens to be time-based (y/n)”, confirm by pressing y.

Configuring google-authenticator for the first time.
Configuring google-authenticator for the first time. (🔍 Zoom in)

The recommended configuration is:

  • Do you want me to update your “/home/[username]/.google_authenticator” file (y/n) → y
  • Do you want to disallow multiple uses of the same authentication token? → y
  • […] we allow an extra token before and after the current time […] Do you want to do so? (y/n) → n
  • Do you want to enable rate-limiting (y/n) → y

Now, scan the QR code on your screen using the TOTP app on your phone or on your client. Write down the emergency codes and store them in a secure place.

Info
You probably note that there is a link to a Google website above the QR code. The link is created on your server. You could use it to go to a Google website that dynamically creates QR codes. Since the QR code is already on your screen, you don't have to click the link. So, simply ignore it.

Finally, we have to tell SSH that it must use the 2FA module for PAM. Open /etc/pam.d/sshd on the server:

  • change @include common-auth#@include common-auth
  • add auth required pam_google_authenticator.so to the bottom of the file

Save the file and quit.

Now, open /etc/ssh/sshd_config as before. Change:

  • ChallengeResponseAuthentication noChallengeResponseAuthentication yes
  • AuthenticationMethods publickeyAuthenticationMethods publickey,keyboard-interactive

Save all changes, quit and restart sshd: sudo systemctl restart sshd. When you try to connect to your server in future, there will be a “Verification code:” prompt. Enter the TOTP generated by your app or your emergency codes if necessary.

On codeberg.org, we provide a minimal configuration file for OpenSSH: sshd_config on codeberg.org. Keep in mind that future versions of OpenSSH may include new parameters or parameters in the file can become obsolete. So, check each parameter and do not blindly copy the file!

Step 5: Back up your SSH keys

As always, it is important to back up your cryptographic keys. On your client, go to ~/.ssh/ and back up all files in there. You should also back up the /etc/ssh/sshd_config of your server.

Follow us on Mastodon:
@infosechandbook

Step 6: Test the configuration

Finally, we want to test our configuration. Most things like “Does SSH use my public key?” or “Does SSH use 2FA?” can directly be tested by restarting sshd and connecting with your server again. But you can’t see if SSH uses modern algorithms only.

One possibility is to use built-in commands:

  • on your client, use ssh -G [hostname] to view all parameters in use
  • on your server, use sudo sshd -T to view all parameters in use

If there are any configured parameters missing, check if they are still used by OpenSSH. They may be obsolete due to an update.

Furthermore, there are several online tools available to test your SSH connection. However, we blocked most IP addresses before. So we must use a tool on our local computer which is allowed to connect to port 22 (or the current SSH port). nmap is perfect for this test.

On your client, enter nmap -Pn --script ssh2-enum-algos [IP-address-of-your-server]. nmap will show:

  • kex_algorithms: (2)
  • server_host_key_algorithms: (1)
  • encryption_algorithms: (3)
  • mac_algorithms: (3)
  • compression_algorithms: (2)

(You may see other numbers according to your configuration!)

Use nmap to see which algorithms are used by SSH.
Use nmap to see which algorithms are used by SSH. (🔍 Zoom in)

Step 7: Configure your local SSH client

After configuring and testing server-side configuration, we can still simplify our SSH setup on our local SSH client. Open ~/.ssh/config on your client. This file contains user-specific SSH configuration on your local machine.

The idea is to create a list of global parameters, and server-specific configuration. Let’s have a look at the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# ~/.ssh/config
# Global parameters, valid for all hosts
Host *
    HashKnownHosts yes
    VisualHostKey yes
    IdentitiesOnly yes
    KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
    Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
    MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com
    HostKeyAlgorithms ssh-ed25519,rsa-sha2-256,rsa-sha2-512,ssh-rsa-cert-v01@openssh.com

# Host-specific parameters, only valid for this specific host
Host ishblog
    VerifyHostKeyDNS yes
    LogLevel VERBOSE
    HostName infosec-handbook.eu
    User remoteuser
    IdentityFile /home/localuser/.ssh/id_ishserver

# Host-specific parameters, only valid for this specific host
Host turrisomnia
    LogLevel QUIET
    HostName 192.168.1.1
    User remoteuser
    IdentityFile /home/localuser/.ssh/id_turris

As you can see, the configuration contains global parameters as well as two host-specific entries. These parameters are:

  • HashKnownHosts: Hash all host names and addresses when they are added to ~/.ssh/known_hosts (doesn’t affect existing names and addresses). The idea is to protect identifying information in the file.
  • VisualHostKey: Shows an ASCII art representation of the remote host key fingerprint at login.
  • IdentitiesOnly: Only use the authentication identity files configured in the ssh_config files (~/.ssh/config and /etc/ssh/ssh_config) even if there are more identify files available.
  • KexAlgorithms, Ciphers, MACs, HostKeyAlgorithms: Specific algorithms for different purposes as explained above.
  • VerifyHostKeyDNS: Uses DNS and SSHFP to validate the remote host key. This is only useful if you set a server-side SSHFP resource record.
  • LogLevel: There are different log levels. Valid levels are QUIET, FATAL, ERROR, INFO, VERBOSE, DEBUG (DEBUG1), DEBUG2, and DEBUG3. The default is INFO.
  • HostName: The real host name to log into. IP addresses are also valid, however, you must change them if DNS configuration (A, AAAA resource records) changes.
  • User: The remote user name that is used to log in as.
  • IdentityFile: The local identity file used for authentication. We generated these in step 3a.

For more parameters, have a look at the man page of ssh_config.

Additionally, you can configure your local /etc/ssh/ssh_config file, if needed. This configuration is valid for all local users on your local client.

This article is part of the "Web server security" series.
Read other articles of this series.

Summary

Done! You configured your firewall, so only whitelisted IP addresses can connect to your server using port 22. Moreover, you hardened your SSH configuration and checked it. Finally, you created backups. To access your server, an attacker has to spoof a whitelisted IP address and needs your private SSH key as well as your TOTP device.

Keep in mind that SSH keys are like passwords: Store them securely, only use them on trusted devices, generate new keys from time to time and securely delete old SSH keys. See also our article about modern credential management.

Advanced users can set up an SSHFP record (Secure Shell fingerprint record) on their DNS server.

In part 4 of this series, we show you how to use Fail2ban to block brute-force attacks.

External files

Changelog

  • Jul 14, 2019: Rewrote several sections due to the release of Debian 10 and part 0 of this series.
  • Feb 9, 2019: Added built-in SSH commands to view parameters in use.
  • Jan 16, 2019: Added more complete client-side configuration.
  • Jan 7, 2019: Removed -o flag since the OpenSSH format for private keys is used by default (OpenSSH 7.8+). Added information how the private key is encrypted.

See also