Banner image of XMPP: Admin-in-the-middle

XMPP: Admin-in-the-middle

In every discussion on instant messaging, somebody promotes XMPP-based messengers as the “privacy-friendly alternative” to other messengers.

In this article, we show the perspective of an XMPP server administrator. Unsurprisingly, an XMPP administrator (or any other server-side party) can inject arbitrary messages, modify address books, and log passwords in cleartext.

Always stay in the loop!
Subscribe to our RSS/Atom feed.

What is the problem?

XMPP proponents argue that federation and decentralization ensure the privacy and security of your data. However, “federation” isn’t a privacy feature. When we look at the privacy policies of XMPP servers, many policies don’t meet the basic requirements of the European GDPR. Furthermore, we checked more than 1,000 XMPP servers. Only seven companies hosted 50% of the XMPP servers. While XMPP looks decentralized when you see long lists of XMPP servers, it is physically centralized.

When we ignore the “federation and decentralization” part and directly look at the server software, nearly everything happens in cleartext. Yes, well-configured XMPP servers enforce TLS for all connections. Yes, the server should hash your password before storing it. However, federation/decentralization, transport encryption, and hashing before storing don’t protect your XMPP password, address book, group memberships, and so on. Server administrators and other server-side parties can read and modify your data because it is in cleartext.

If you want to use XMPP, we recommend hosting your own server instead of trusting an unknown party somewhere on the internet. Of course, self-hosting requires much more knowledge and time than just using an arbitrary public XMPP server.

Demonstrating the power of an XMPP admin

For this article, we set up the ejabberd Community Server and some XMPP clients. We mostly stayed with the default configuration to make everything as verifiable as possible. However, even a hardened configuration focused on security doesn’t deprive the server admin of their power.

Server-side setup

We set up the ejabberd Community Server:

  • ejabberd 18.06 on Debian 9 (original test in 2018)
  • ejabberd 19.08 on Ubuntu 18.04 LTS (retest in 2019)
  • ejabberd 20.12 on Ubuntu 20.04 LTS (retest in 2020)

In “ejabberd.yml,” we added the domain name and IP address of our server and enabled “in-band registration.” Furthermore, we opened the ports 5222 (client-to-server connections, c2s) and 5269 (server-to-server connections, s2s).

Client-side setup

We checked three XMPP clients:

  • Conversations (via F-Droid)
  • Gajim (Nightly and Stable)

Test 1: Monitor all XMPP connections

In July 2019, we identified the hosters of more than 1,000 XMPP servers (see our list on codeberg.org). Only seven companies hosted 50% of the XMPP servers. These companies were located in three different countries, especially in Germany (again, 50% of the XMPP servers). Besides, XMPP servers use dedicated ports to communicate with other servers or clients.

State actors, law enforcement, and server hosting companies can easily detect and monitor XMPP traffic. We can simulate their perspective with tcpdump and Wireshark:

#! /bin/bash
mkfifo /tmp/pcap_file
ssh [router] "tcpdump -n -w - -i [interface] '(port 5222) or (port 5269)'" > /tmp/pcap_file
wireshark -k -i /tmp/pcap_file

We monitor all XMPP client-to-server and server-to-server connections as a “third party” since we look at the network traffic.

This first test seems trivial but already shows the following:

  • We see the public IP addresses of clients connecting to the XMPP server.
  • We see the IP addresses of other XMPP servers that communicate with our test server.

These observations disclose metadata:

  • IP addresses of the sender and receiver of each packet
  • Approximate location based on IP addresses
  • Timestamps
  • Bytes transmitted (This allows us to distinguish between XMPP status messages and actual user-written messages)
  • Number of packets sent and received

Enforcing TLS between clients and servers still discloses this metadata. State actors, law enforcement, server hosting companies, and internet service providers can derive communication and usage patterns or block XMPP network traffic based on this.

Test 2: Use ejabberdctl for information gathering

We switch from the perspective of third parties to the server-side view. This view includes not only the actual administrator but also server-side attackers with access to ejabberdctl.

“ejabberdctl” is a powerful administration script for ejabberd. Server-side parties can use it to gather information about all users of the XMPP server. For instance:

  • See all users connected to the server (connected_users_info), including JID, type of connection, IP address, port number, resource priority, session duration.
  • See all devices of a connected user (user_resources).
  • See all contacts of a user (get_roster).
  • See all MUCs (muc_online_rooms).
  • See all MUC options (get_room_options).
  • See all MUC affiliations (get_room_affiliations) like owner, admin, member, outcast, none.
  • See all MUCs where a user is an occupant (get_user_rooms).
  • Read the content of vCards (get_vcard).

Server-side parties can access most private data of users, including their contact lists, group memberships, and other personal data like IP addresses, names, and devices. A server-side party doesn’t have to access the database or monitor the network traffic but just use ejabberdctl.

State-of-the-art messengers either implement client-side account management or enforce end-to-end encryption so that the server always sees strongly encrypted data only. In both cases, server-side parties can’t gather this information.

Test 3: Analyze the Mnesia database

By default, ejabberd uses Mnesia to store all data. Server-side parties can enter sudo ejabberdctl dump dump.txt to dump the whole Mnesia database to “dump.txt” in the Mnesia folder.

This Mnesia dump includes:

  • User-specific authentication data for SCRAM-SHA-1 (hashed password, salt, number of iterations)
  • User-specific key material (e.g., for OMEMO PreKeys)
  • PEP (Personal Eventing Protocol) information
  • Last activity of the user (timestamp)
  • Private storage of users (bookmarks, like saved MUCs)
  • MUC-related data (members, options)
  • Everything listed in the section of test 2
  • and more

Of course, for successful communication, either the clients or the server need to store this information. In the case of XMPP, the server stores it in cleartext. As mentioned before, state-of-the-art messengers either implement client-side account management or enforce end-to-end encryption so that the server always sees strongly encrypted data only. In both cases, server-side parties can’t gather this information.

Test 4: See the content of messages and passwords in cleartext

In “ejabberd.yml,” you can change the log level to DEBUG by configuring loglevel: 5. As a result, ejabberd logs everything, for example:

  • Message metadata (message ID, from JID+client to JID, timestamp)
  • Message content, which may include images, files, location data, and audio files
  • Message status (received by a client, displayed (= read by) to a user)
  • Client status (offline, online, typing)
  • Non-public user actions (user changed avatar, status message, vCard information or created a private MUC)
  • and more

If the clients didn’t configure end-to-end encryption for message content, server-side parties can read everything, as shown in the following picture:

An image showing metadata of XMPP.
This side-by-side comparison shows the perspective of a user (Gajim, on the right-hand side) and the same message from the server-side perspective (log file, on the left-hand side). (🔍 Zoom in)

However, loglevel: 5 not only logs every single message and all of the activities mentioned above but also passwords in cleartext. Some XMPP admins insisted that this isn’t true as XMPP servers use SCRAM-SHA-1 for password hashing. However, it doesn’t matter whether you enable or disable server-side password hashing.

ejabberd logs every password in cleartext when somebody registers a new account or changes their password, as shown in the following picture:

An image showing logged passwords in cleartext.
Cleartext passwords in ejabberd.log. Whenever a user registers an account or changes his password, his password is logged (in-band registration is used in this example). (🔍 Zoom in)

Combined with the network traffic (test 1), a server-side party can monitor all users' actions on the server. However, this includes users of this server and users of other servers who are members in MUCs of this server or chat with people on this server.

Info
Client-side encryption (OMEMO, OpenPGP, OTR) only hides the actual content of messages. Everything else is still in cleartext, including metadata and passwords. Using Tor may conceal your real IP address; everything else is still visible. Using Tor + client-side encryption only hides a small subset of the information visible to server-side parties. The vast majority of data remains in cleartext.

Test 5: Manipulate user data

We remained passive so far (tests 1 to 4), meaning we did not modify any user data. However, ejabberdctl allows server-side parties to modify user data, too:

  • We can inject fake messages from arbitrary JIDs (send_message chat), even if these users aren’t on our server.
  • We can add arbitrary JIDs to a user’s contact list (add_rosteritem) or delete JIDs (delete_rosteritem).
  • We can arbitrarily change the vCards of users (set_vcard).
  • We can invite users to MUCs (send_direct_invitation)
  • We can transfer the ownership of MUCs to another user (set_room_affiliation) and kick the original owner (muc_unregister_nick).
  • We can change the MUC configuration (change_room_option).
  • We can delete messages older than x days if not delivered (delete_old_messages).
  • and more

The following screenshot shows an injected message coming from “snowden[at]nsa.gov.” During our original test, the Conversations client showed these messages. Gajim ignored them until the sender JID of the message is in the user’s roster.

An image showing a message allegedly sent by Edward Snowden.
Conversations shows this faked message from snowden@nsa.gov. Gajim shows such messages after an admin added the fake sender to the user's contact list. (🔍 Zoom in)

Lessons learned and recommendations

XMPP relies on server-side account management, which results in the issues described above. Using client-side encryption (OMEMO, OpenPGP, OTR) and Tor only solves a small number of these issues.

Of course, XMPP isn’t the only communication protocol that works this way. However, we rarely see an XMPP proponent who honestly talks about the amount of trust users put into the XMPP administrator.

If you want to self-host an XMPP server, we recommend the following steps:

  • Set up a firewall with a strict ruleset.
  • Regularly update all packages on your server.
  • Install only software that you need and remove everything else.
  • Harden your SSH configuration: Use modern cryptography, 2FA, key-based authentication, non-root login, and an IP allowlist.
  • Use certificates signed by a public certificate authority (e.g., signed by Let’s Encrypt).
  • Harden your TLS configuration: TLS 1.2 and 1.3 + forward secrecy + AEAD, strong parameters for DHE and ECC.
  • Disable federation if you don’t need it.
  • Disable unused XMPP mods (XEPs), esp. “in-band registration.”
  • Disable logging of your XMPP server software if you don’t need it.
  • Enforce OMEMO on your clients (hopefully, they do support it!).
Warning
Securely hosting your own server isn't done by implementing a "Host your own XMPP server in 5 minutes" guide. Keep in mind that you need to keep your server secure continuously.

Summary

Our findings show that a server-side party can read, manipulate, and delete nearly all users' data. XMPP clients can’t detect most of the server-side actions. Considering that many (European) XMPP servers still don’t come with a privacy policy that meets the basic requirements of the European GDPR or that many XMPP servers are physically centralized, the overall situation doesn’t look as privacy-friendly as claimed by some people.

We strongly recommend hosting your own XMPP server in a controlled environment or switching to messengers that either implement client-side account management or enforce end-to-end encryption for account management.

Tested and written in collaboration with Thorsten.

Sources

Note on a statement that we removed in December 2020

Following our initial report in 2018, a German blogger “addressed” our findings by claiming that every server-based communication system comes with the same drawbacks. He wrote a lengthy article on how all of us need to improve XMPP, set up his own XMPP server, and claimed that he found a “security vulnerability” in ejabberd.

After more than two years, nothing changed. He didn’t publish anything regarding the “security vulnerability,” his XMPP server disappeared, and all of the issues with XMPP are still present. The only thing that did change: He now recommends Signal instead of XMPP. We removed our statement from our article due to his U-turn and unproven claims.

Changelog

  • Dec 24, 2020: Retested the same setup with ejabberd Community Server 20.12 on Ubuntu 20.04 LTS. Several sections were rewritten.
  • Aug 10, 2019: Tested the same setup again using the current version of ejabberd on Ubuntu. Several sections were rewritten.
  • Nov 2, 2018: Added and commented on some feedback of our readers.
  • Aug 17, 2018: Added image showing cleartext passwords in ejabberd.log. Added information about Tor in the summary.

Read also