XMPP: Admin-in-the-middle

XMPP: Admin-in-the-middle

In every discussion on instant messaging, some people show up and recommend XMPP-based messengers like Conversations, Gajim, Dino, or Monal. Oftentimes, it is said that these messengers are extremely privacy-friendly due to being federated/decentralized.

We debunked several federation myths and showed issues with XMPP. Furthermore, we showed that privacy policies of several XMPP servers don’t meet basic requirements of the European GDPR.

In this article, we show the perspective of an XMPP server administrator. A server administrator can do basically everything, including injecting arbitrary messages, modifying address books, or logging passwords in cleartext. We strongly recommend always running your own XMPP server (or switch to more secure services).

Contents

  1. Introduction
  2. Setup and testing
  3. Lessons learned and recommendations
  4. Summary
  5. Sources
  6. Changelog

Always stay in the loop!
Subscribe to our RSS/Atom feeds.

Introduction

In August 2018, we looked at XMPP server software to evaluate whether private data of server users is protected against server-side parties. It turned out that there is no real protection in place. Server-side parties can access, manipulate, and delete nearly all data of users. The worst part is that users of XMPP servers can’t learn about this without accessing the server themselves to check its configuration.

Three months later, a German blogger “addressed” our initial findings by claiming that this is typical for communication systems that involve servers. Then, he wrote a lengthy article on how everybody needs to improve XMPP, and repeated all of our findings himself. Finally, he set up his own XMPP server for a short time (without publishing any kind of privacy policy), talked about a security vulnerability in ejabberd that he allegedly discovered, and repeated his appeal to improve XMPP.

In August 2019, we checked ejabberd again—one year after we originally published this article, or 9 months after the German blogger “addressed” our findings. Nothing had changed. The blogger didn’t publish any further information about the alleged security vulnerability in ejabberd, and he didn’t improve anything about XMPP. The obvious goal was to present our findings as a lot of wailing while he actually does something about it. 🤷🏻

Setup and testing

After setting up an ejabberd server, we connected to it using different XMPP clients, and acted like typical users do: Creating accounts and groups, sending messages and other data. Then, we conducted 5 different tests to see to what extent private data is exposed to server-side parties.

Server-side setup

Originally, we tested ejabberd 18.06 on Debian 9. We mostly didn’t change the defaults. This didn’t matter since the end user can’t see most of the configuration when connected to an XMPP server.

In ejabberd.yml, we added the domain name and IP address of our server and enabled “in-band registration”. Furthermore, we opened the ports 5222 (client-to-server connections, c2s) and 5269 (server-to-server connections, s2s).

In August 2019, we tested the same setup again. This time, we used ejabberd 19.08 on Ubuntu 18.04 LTS. The new version of ejabberd has the same issues as before.

Client-side setup

We used three clients: Two Android phones had Conversations 2.2.8 (via F-Droid) and one Linux device had Gajim 1.0.3 installed. In August 2019, we only used the current Gajim nightly version.

Test 1: Monitor all connections

A common argument for XMPP is that state actors, law enforcement, and others can’t easily identify and block XMPP traffic. This is wrong.

First of all, many logically-different XMPP servers are hosted by a very small amount of server hosting providers (see our list on GitHub). Our sample contained more than 1,000 XMPP servers. 50% of these servers were hosted by only 7 different companies that were located in 3 different countries. Moreover, 50% of XMPP servers in our sample were located in Germany.

So it is easy for state actors, law enforcement, and server hosting companies to monitor network traffic of many different XMPP servers since they are physically centralized. If we look at network traffic as seen by third parties, we can clearly identify XMPP traffic.

For this test, we configure tcpdump and Wireshark in the following way:

1
2
3
4
#! /bin/bash
mkfifo /tmp/pcap_file
ssh [username]@[xmppserver] "tcpdump -n -w - -i ens3 '(port 5222) or (port 5269)'" > /tmp/pcap_file
wireshark -k -i /tmp/pcap_file

This allows us to monitor all XMPP-relevant connections in real time as a “third party” since we look at the network traffic routed by a local router.

The unsurprising result:

  • We see all public IP addresses of users of our XMPP server.
  • We see all IP addresses of other servers which host users who chat with users of our server.

Of course, this includes metadata like:

  • IP addresses of the sender and receiver of each packet
  • timestamps
  • approximate location based on IP addresses
  • bytes transmitted (one can differentiate between actual messages and XMPP status messages)
  • number of packets sent/received
  • etc.

In this case, TLS encryption hides the actual content of these packets, of course. However, the above-mentioned metadata is still visible for server hosting companies or other third parties. Thanks to IP addresses, communication patterns, and traffic, which can’t be encrypted, third parties can identify XMPP communication.

Having said this, we noted that the default TLS configuration of ejabberd 18.06 was not that secure:

  • TLS 1.0, TLS 1.1 and TLS 1.2 were enabled while TLS 1.2 and 1.3 are recommended nowadays.
  • There was a self-signed certificate using RSA (2048 bit). For internet-facing services, use CA-signed ECDSA certificates.
  • Small key sizes were in use.
  • Several cipher suites didn’t support perfect forward secrecy and/or AEAD.
Part of the default TLS configuration of ejabberd 18.06. Especially, weak DH params (1024 bit) and outdated TLS versions (1.0 and 1.1) should be disabled.
Part of the default TLS configuration of ejabberd 18.06. Especially, weak DH params (1024 bit) and outdated TLS versions (1.0 and 1.1) should be disabled. (🔍 Zoom in)

Note that several XMPP server admins started to harden their servers for better transport security. This makes it impossible to establish server-to-server connections with their hardened servers if you use the defaults here.

We didn’t retest the server-side TLS configuration of ejabberd 18.09.

Test 2: Use ejabberdctl for information gathering

We switch from the perspective of third parties to the server-side perspective. This not only includes “real” administrators but also server-side attackers with access to ejabberdctl.

ejabberdctl is a very powerful administration script for ejabberd. Server-side parties can use it to gather information about all users of the XMPP server. For instance:

  • See all users connected to the server (connected_users_info), including JID, type of connection, IP address, port number, resource priority, session duration.
  • See all devices of a connected user (user_resources).
  • See all contacts of a user (get_roster).
  • See all MUCs (muc_online_rooms).
  • See all MUC options (get_room_options).
  • See all MUC affiliations (get_room_affiliations) like owner, admin, member, outcast, none.
  • See all MUCs where a user is occupant (get_user_rooms).
  • Read content of vCards (get_vcard).
  • etc.

This shows that a server-side party like an admin can access most private data of users including their contact lists, their group memberships and other personal data like IP addresses, names, devices and so on. The admin doesn’t have to directly access the database or monitor traffic. He can just use ejabberdctl.

Several state-of-the-art messengers like Signal implemented client-side contact and group management. This hides most of the above-mentioned data from server-side parties.

Test 3: Analyze the Mnesia database

We want to learn more about our users. Let’s analyze the database! By default, ejabberd uses Mnesia to store all data. An admin (or any other server-side party) can simply enter sudo ejabberdctl dump dump.txt to dump the whole Mnesia database to dump.txt stored in the Mnesia folder.

The Mnesia dump includes:

  • user-specific data for SCRAM-SHA-1 used for authentication (hashed password, salt, number of iterations)
  • user-specific key material (e.g., for OMEMO PreKeys)
  • PEP (Personal Eventing Protocol) information
  • last activity of users
  • private storage of users (bookmarks like saved MUCs)
  • all MUCs (members, options)
  • everything listed in the section of test 2
  • and more

Of course, all of this data must be stored somewhere. However, as mentioned above, state-of-the-art messengers implement client-side management for most of these things. Client-side management hides data from server-side parties and can only be accessed/changed client-side.

Moreover, note that all of this is stored in cleartext. An attacker with physical access to the server (or root access) can simply copy/read this data.

Test 4: See content of messages and passwords in cleartext

So far, we didn’t read actual messages or passwords. This is also very easy. Server-side parties only have to switch the logging level in their ejabberd configuration file: In ejabberd.yml change the loglevel: 5.

Logging level 5 means “debug mode” or basically “log everything”. If we look at ejabberd.log, we discover:

  • message metadata (message ID, from JID+client to JID, timestamp)
  • message content (if no encryption in use), may include images, files, location data, audio files
  • message status (received by a client, displayed (= read by) to a user)
  • client status (offline, online, typing)
  • non-public user actions (user changed avatar, status message, vCard information or created a private MUC)
  • and more!
This side-by-side comparison shows the perspective of a user (Gajim, on the right-hand side) and the same message from the server-side perspective (log file, on the left-hand side).
This side-by-side comparison shows the perspective of a user (Gajim, on the right-hand side) and the same message from the server-side perspective (log file, on the left-hand side). (🔍 Zoom in)

Additionally, loglevel: 5 not only logs every single message and all activities, but also passwords in cleartext. This happens when a new user registers for the first time, or when a user changes passwords. It doesn’t matter whether SCRAM-SHA-1 is enabled.

Cleartext passwords in ejabberd.log. Every time a user registers an account or changes his password, his password is logged (in-band registration is used in this example).
Cleartext passwords in ejabberd.log. Every time a user registers an account or changes his password, his password is logged (in-band registration is used in this example). (🔍 Zoom in)

Combined with traffic data, an admin can monitor every single action of all users on the server. However, this not only includes users of this server but also users of other servers who are members in MUCs of this server or chat with people on this server.

Info
We would like to comment on typical statements here:
  1. Turn on OTR/OpenPGP/OMEMO! It encrypts your messages!
  2. Use Tor to hide your identity!
Using any kind of encryption that is currently supported by XMPP clients only hides the actual content of messages. Everything else is still in cleartext, including tons of metadata. Using Tor only hides the actual IP address, while information about the XMPP client etc. is still available. So using both only hides a small subset of information. The vast majority of data remains in cleartext.

Test 5: Manipulate user data

Up to this point, the server-side party was somewhat passive in its actions. Let’s manipulate data now! We return to ejabberdctl to:

  • inject fake messages from arbitrary JIDs (send_message chat) even if these users aren’t on this server, see image below (Conversations shows these messages, Gajim ignores them until the sender JID of the message is in the user’s roster)
  • add arbitrary JIDs to a user’s contact list (add_rosteritem) or delete JIDs (delete_rosteritem)
  • arbitrarily change vCards of users (set_vcard)
  • invite users to MUCs (send_direct_invitation) and let them be owner (set_room_affiliation) while kicking the original owner (muc_unregister_nick) or make him member. Of course, admins can change all options of MUCs (change_room_option)
  • delete messages older than x days if not delivered (delete_old_messages)
  • and more
Conversations shows this faked message from snowden@nsa.gov (which can't be true since Mr. Snowden uses Signal!). Gajim only shows such messages after an admin added the fake sender to the user's contact list.
Conversations shows this faked message from snowden@nsa.gov (which can't be true since Mr. Snowden uses Signal!). Gajim only shows such messages after an admin added the fake sender to the user's contact list. (🔍 Zoom in)

This shows that the admin not only can read nearly everything but also manipulate data arbitrarily.

Lessons learned and recommendations

This article clearly shows the problems of communication systems that rely on server-side management. XMPP heavily relies on the server side: groups, contacts, vCards are all managed by the server.

The main problems with XMPP are:

  1. Nearly everything is stored in cleartext on the XMPP server and can be easily accessed.
  2. OMEMO—as state-of-the-art end-to-end encryption—isn’t widespread while some clients dropped OTR which is somewhat awkward since this could force you to disable end-to-end encryption completely. Besides, some people hope or expect MLS (Messaging Layer Security) becoming the next star of encryption for XMPP. Then, we will have OTRv3, OTRv4, OpenPGP, OMEMO, and MLS as competing technologies to encrypt a small amount of data.
  3. Even end-to-end encryption and the Tor network provide little protection since most data remains unencrypted.
  4. Your personal data is shared with other servers as soon as you chat with remote users or join remote MUCs. So your data is exposed to other admins.
  5. Most server-side actions are fully transparent. This means that you can’t detect such actions as a user. For instance, you can’t check whether an admin enabled the above-mentioned logging of every event including cleartext passwords.

Other messaging systems like peer-to-peer messengers (Briar, Jami), or Signal rely on client-side management: groups, contacts, vCards are managed by the locally installed client. This comes with new problems in some cases, for instance all members of a group have the same rights since there is no server to check different roles. However, client-side management hides most data and metadata from servers, and gives server-side parties considerably less information about you and your chat activities.

To mitigate the above-mentioned problems with XMPP, our clear recommendation is that you run your own XMPP server, or use one provided by close friends/family members. If you want to run your own XMPP server, have a look at the (incomplete) checklist below:

Warning
Be aware that any server-side party can still monitor data and metadata as shown before. You must continuously maintain and secure your own XMPP server.

Follow us on Mastodon:
@infosechandbook

Summary

Our findings show that a server-side party can read, manipulate, and delete nearly all data of users. Most of these actions can’t be detected by XMPP clients, so these actions remain undetected. Taking into account that many (European) XMPP servers still don’t come with a privacy policy that meet basic requirements of the European GDPR, or that many XMPP servers are physically centralized (= hosted by a small amount of server hosting providers), the overall situation doesn’t look as privacy-friendly as claimed by some people. It is important to inform people about these issues instead of writing appeals that don’t change anything.

We strongly recommend hosting your own XMPP server in a controlled environment. Of course, this not only requires setting it up. You must continuously monitor and update your own server. If you can’t ensure this, use a well-known messenger like Signal, which enforces end-to-end encryption and comes with client-side management.

Besides, in October 2017, Tor and privacy activist Moritz Bartl (Zwiebelfreunde e.V.) stated that it is easy for state actors to monitor XMPP traffic and identify all server users. Furthermore, he is convinced that most XMPP server and client software is more a leisure project than secure software.

Tested and written in collaboration with Thorsten.

Sources

Changelog

  • Aug 10, 2019: Tested the same setup again using the current version of ejabberd on Ubuntu. Several sections were rewritten.
  • Nov 2, 2018: Added and commented some feedback of our readers.
  • Aug 17, 2018: Added image showing cleartext passwords in ejabberd.log. Added information about Tor in the summary.

See also