Pros and cons of online assessment tools for web server security

Pros and cons of online assessment tools for web server security

Maybe you already use the Observatory by Mozilla, Webbkoll, SSL Labs or other online assessment tools to get a first expression of the security of a website. All of these tools are easy to use, however, their possibilities to assess the security of a website and especially of a web server are limited. We show you pros and cons of several tools in this article.

Contents

  1. General note
  2. Online assessment tools
  3. Summary
  4. Links
  5. Changelog

Always stay in the loop!
Subscribe to our RSS/Atom feeds.

General note

First of all, there is no tool which provides a holistic view of the security level of a server. Many important aspects can only be tested when you can directly access the web server and check its configuration files. For example, most tools don’t check if the installed web server (Apache, nginx, etc.), PHP packages, database software or operating system (Debian, Ubuntu, etc.) are up-to-date. This is really important, though.

If you are a server administrator, you should also look for internal testing (e.g. nmap, openssl, sslyze and many more cli tools available). Do not solely rely on the following tools!

Online assessment tools

All of the following 9 online assessment tools can be used in the web browser without installing software on your client. You only have to enter the URL of the website you want to be assessed.

Observatory by Mozilla – overview

The Observatory by Mozilla has the following features:

  • It checks the Content Security Policy (CSP)
  • It checks security-relevant HTTP response headers (e.g. HSTS)
  • It checks cookie configuration, HTTPS redirection, referrer policy and so on
  • It checks TLS configuration, CAA, AEAD and PFS
  • It checks for OCSP stapling and cipher preference
  • It shows other online assessment tools for further assessment

You have to keep in mind:

  • HTTP Observatory:
    • Websites can get up to 135 points. A rating of more than 100 points means that you see a green A+. The problem: You get 10 bonus points for secure cookies (HttpOnly, Secure, SameSite) and Subresource Integrity (SRI). You don’t get these points if there are no cookies at all and there is no reason for implementing SRI.
    • X-Frame-Options are assessed by checking the CSP. If undefined in the CSP, the Observatory looks for a header. So, do you need an extra header at all if you set this in your CSP? Yes, because some assessment tools and web browsers only look for the header. Make sure that frame-ancestors (CSP) and your X-Frame-Options header have the same configuration.
    • Websites get 5 bonus points for being “Preloaded via the HTTP Strict Transport Security (HSTS) preloading process”. Nowadays, mainstream web browsers are shipped with a built-in list containing websites which have to be loaded via HTTPS. Administrators can add their website to the list (HSTS preload list, link below). However, after a website is added to the list it can take a long time until the Observatory recognizes this. It is more reliable to directly check the HSTS preload list for your website.
  • TLS Observatory:
    • It doesn’t recognize TLS 1.3 at the moment.
    • The TLS Observatory doesn’t recognize modern cipher suites with ChaCha20-Poly1305. They don’t appear on the list. It only recognizes “CHACHA20-POLY1305-OLD”.
    • There is additional (more technical) information when you click on Scan Summary → Certificate Explainer.
  • SSH Observatory:
    • You can use the SSH Observatory by manually starting the assessment. Please note that the Observatory must be allowed to connect to port 22 of the web server to check the SSH configuration.

Hardenize – another overview

Hardenize is another tool to get a first impression and very similar to the Observatory by Mozilla. However, it also checks the mail server, if available. Its main features are:

  • It checks DNS records, DNSSEC configuration and CAA
  • It checks for TLS (including TLS 1.3) configuration, HTTPS redirection, cookies etc.
  • It checks security features like MTA-STS, TLS-RPT, DANE, SPF and DMARC of the mail server
  • It checks for Certificate Transparency and cipher preference
  • It checks for a Expect-CT header and submits a test report to the provided report-uri
  • It checks provided IPv4 and IPv6 addresses of the domain

You have to keep in mind:

  • You should click on every single check, because there are sometimes additional hints for better configuration or warnings due to bad configuration.
  • The CSP check recommends to set either block-all-mixed-content or upgrade-insecure-requests. We think that this is unnecessary if you don’t embed external content and your website is preloaded (HSTS).
  • It shows that Subresource Integrity (SRI) is required even if files are provided by the same web server (e.g. you check xyz.cz which embeds resources of xyz.sk, but both domains are hosted on the same web server).
  • It recommends HPKP while many security professionals hold the belief that you shouldn’t implement this anymore.
  • Only cipher suites supporting PFS and AEAD are showed in green color. If you are an administrator, you should ensure that you only provide cipher suites which are green.

Qualys SSL Labs – certificate details

SSL Labs mainly checks certificates and TLS configuration of a web server. Its main features are:

  • Certificate:
    • It evaluates Extended Validation, Certificate Transparency, OCSP Must-Staple and CAA.
    • It checks revocation status of certificates.
  • TLS protocols:
    • It supports SSL (2, 3) and all TLS versions (1.0–1.3).
  • Cipher suites:
    • It checks for PFS and AEAD. However, websites can still offer non-PFS and/or non-AEAD cipher suites alongside cipher suites which support it. There is no penalty for this.
  • General information:
    • It shows if the server exposes its signature (however, this isn’t evaluated)
    • It checks whether a website is on the HSTS preload list.
  • SSL Labs checks all IPv4 and IPv6 addresses which belong to the domain name.

Interestingly, SSL Labs recognizes websites which were recently added to the HSTS preload list much more faster than other tools.

High-Tech Bridge – detailed security configuration

High-Tech Bridge offers two different tests for free:

ImmuniWeb SSLScan

  • This test evaluates certificates and ciphers suites including TLS 1.3.
  • It clearly visualizes the certificate chain(s).
  • It evaluates Extended Validation, Certificate Transparency, OCSP Must-Staple, cipher preference and CAA.
  • It shows supported elliptic curves.

You have to keep in mind:

  • This test checks for compliance with PCI DSS, HIPAA and NIST. HIPAA and NIST guidelines don’t accept ECDSA certificates and require you to enable TLS 1.1. If you don’t have to comply with them, ignore this.
  • In comparison with most of the other tools in this article, the rating of cipher suites is somewhat wishy-washy. For example, TLS 1.0 and TLS 1.1 are still considered as “good protocol compatibility, allowing users with older browsers to access your website.” Our recommendation is to disable all TLS versions before 1.2.

ImmuniWeb WebScan

  • This test evaluates enabled HTTP methods, ALPN (part of HTTP/2) and more.
  • It also checks if the web server exposes its server signature and tries to recognize software used on the web server (jQuery, Bootstrap, etc.).
  • It evaluates the CSP and shows tips for better configuration.
  • It evaluates HSTS, Expect-CT and other response headers and shows misconfiguration.
  • It analyzes cookies (HttpOnly, Secure, SameSite).
  • It shows third-party content and the corresponding TLS security level of these external connections (even if the content is on the same web server).

You have to keep in mind:

  • This test shows “Some potentially insecure HTTP methods supported by the web server require your attention.” when you enable the HTTP method HEAD. HEAD allows clients to get the same information as with GET, but the web server omits the payload in its response. It only sends the header data. Disabling HEAD can result in more traffic (clients always get the full payload, even if it is unnecessary) and some tests like CryptCheck’s test for HSTS rely on HEAD (see below).
  • If you run a web application firewall, the test for different HTTP types (like TRACE, POST, DELETE, …) gets confused and shows every HTTP type as enabled. This result is wrong.
  • Both tests still show information regarding the HPKP header (“The server does not enforce HTTP Public Key Pinning that helps preventing man-in-the-middle attacks”). Many security professionals hold the belief that you shouldn’t implement this anymore. Clients also started to drop support for it.
  • NPN (Next Protocol Negotiation) was a draft for the SPDY protocol which isn’t in use nowadays. When you use HTTP/2, you should enable ALPN and disable NPN. Most web browsers already dropped support for NPN.
  • Testing websites with this scanner is really noisy and probably triggers web application firewalls. This leads to wrong testing results (as mentioned above).

CryptCheck – detailed cipher suites

CryptCheck is like a magnifier for TLS and cipher suites. Its features are:

  • It checks algorithms used for key exchange, authentication, encryption and MAC in detail.
  • It checks for PFS and HSTS and shows supported TLS protocols.
  • It rates the strength of keys.
  • It checks all IPv4 and IPv6 addresses belonging to the domain name.

You have to keep in mind:

  • It doesn’t recognize TLS 1.3 or AEAD at the moment.
  • We didn’t find any explanation for colors in use by CryptCheck. Our guess is:
    • blue: modern and good value, secure
    • green: good value, secure
    • yellow: bad value, possibly insecure
    • red: bad value, insecure
    • black: bad value, extremely insecure
    • gray: ?
  • CryptCheck uses the HTTP method HEAD to check for HSTS headers. If you have disabled this in your server configuration, HSTS and HSTS_LONG won’t be recognized by CryptCheck. This results in lower rating of a website.
  • If you configured IPv6 (AAAA record) and your web server is only available via IPv4, CryptCheck won’t recognize HSTS and HSTS_LONG.
  • CryptCheck seems to recognize only one certificate. If you deployed two (e.g. RSA and ECDSA), you will only see one of them.
  • As an administrator, you should immediately get rid of all entries which aren’t blue, green or yellow. Furthermore, you should check if yellow entries are really necessary. Cipher suites using CBC should also be disabled (vulnerable to Lucky 13 attack and no AEAD). SHA-1 and MD5 aren’t colored, but you shouldn’t use both, too.
  • You have to disable AES-128 if you want to get a 100% rating. We don’t see any reasons to do so.
  • CryptCheck can also check the SSH configuration of a server. Of course, you have to allow CryptCheck to connect with port 22 of your web server.

securityheaders.com – security headers only

securityheaders.com rates the same response headers of a website as shown before: CSP, referrer policy, HSTS, X-Frame-Options, X-XSS-Protection, X-Content-Type-Options, Expect-CT, Expect-Staple, Feature-Policy, Network Error Logging and Report-To. It also shows all headers send by the web server (like the Observatory by Mozilla).

Additional checks are:

  • It checks whether the web server exposes its software manufacturer and version information
  • It checks for the Expect-CT header. This header can be used to log errors if Certificate Transparency is deployed.
  • It checks for the Expect-Staple header. This header can be used to log errors if OCSP Must-Staple is deployed.
  • It checks for a Feature Policy. Feature Policies are very new, not widespread and allow admins to enable/disable certain features of web browsers.
  • It checks for a NEL header. The Network Error Logging (NEL) header defines a mechanism enabling web applications to declare a reporting policy that can be used by the user agent to report network errors for a given origin. The logged network errors even include problems occurred before a connection was successfully established.
  • It checks for a Report-To header. The Report-To header is defined in the Reporting API. It aims to provide a best-effort report delivery system that executes out-of-band with website activity and introduces new report types.

CSP Evaluator – assess CSPs

The CSP Evaluator by Google can only evaluate the Content Security Policy of a website. It shows bad configuration and improvement tips. You can enter a URL or a CSP.

Webbkoll – connections and cookies

Some people consider Webbkoll as the ultimate privacy tool. They enter a URL, click on check, wait several seconds and are happy if there are lots of zeros and green check marks. However, Webbkoll has several drawbacks which we already discussed in another article: “Limits of Webbkoll”. Its main features are:

  • It presents you information about HTTPS configuration, HSTS, CSP, Referrer Policy, SRI, localStorage and other security-relevant HTTP response headers.
  • It lists third-party requests and the approximate server location.
  • It shows its users GDPR-relevant information.

Please note that the results are only valid for the link you entered. Other pages of the same website can differ (see three examples). Webbkoll partially uses code of the Observatory by Mozilla for its checks.

PrivacyScore – assessment mix

PrivacyScore is developed by German universities and lists four categories:

  • NoTrack: Most of these tests are also conducted by Webbkoll. It is also checked if web server and/or mail server are located in the EU.
  • EncWeb: These tests include evaluation of response headers, PFS, TLS protocol versions, cipher preference and several attacks on TLS.
  • Attacks: This category mainly checks for response headers like securityheaders.com.
  • EncMail: These checks are mostly the same as in the EncWeb category, but the mail server is checked.

You have to keep in mind:

  • It doesn’t recognize TLS 1.3 or AEAD at the moment.
  • Like the Observatory by Mozilla, websites recently added to the HSTS preload list aren’t recognized for a long time.
  • Even web servers with strict configuration can’t get 100% ratings. For example, we are vulnerable to the BREACH attack (according to this site) due to activated gzip compression. Most other websites are vulnerable to Lucky 13 due to cipher suites which support CBC instead of AEAD.
  • Testing websites with this scanner is really noisy and probably triggers web application firewalls. This leads to wrong testing results.

Follow us on Mastodon:
@infosechandbook

Summary

Hopefully you got a first impression what most tools can test and what they can’t. Many tools evaluate response headers, TLS protocols and cipher suites. Additionally, some tools give extensive information about their rating and tips how admins can improve their security configuration. TLS 1.3 and some newer cipher suites or important features like AEAD or OCSP Must-Staple aren’t supported in most cases.

You should evaluate websites regularly since tests are added, modified or deprecated from time to time.

As mentioned above, you should also always keep in mind that none of these tools (even if you use all of them) can provide a holistic view of the security level of a server.

Changelog

  • Dec 11, 2018: Updated this article including Webbkoll’s update from 2018-11-30.
  • Nov 17, 2018: Updated nearly all feature lists.
  • Nov 1, 2018: Updated securityheaders.com’s feature list.
  • May 26, 2018: Updated High-Tech Bridge products according to their website.
  • May 23, 2018: Removed HPKP in Webbkoll section since Webbkoll removed its recommendation due to this article.

See also