In recent weeks, some of us were heavily involved in preparing and hosting a public CTF contest in the context of this year's European Cyber Security Month (ECSM). More than 1,000 teams registered, nearly 600 solved at least one challenge. Unlike most other CTFs, this one was focused on industrial security.
In this article, we give an insight into preparations and hosting of the CTF event, and discuss some lessons learned.
Always stay in the loop!
Subscribe to our RSS/Atom feeds.
The Syskron Security CTF
The Syskron Security CTF contest was a free online cyber security competition for everyone, but especially for school and university students. CTF means “Capture The Flag”. Teams had to solve security challenges to retrieve flags (text strings). Then, they got points for submitting the flags.
Preparing the contest
In the following, there are some main activities when preparing the contest:
- Preparing the technical platform: In this case, we set up and configured 4 servers (see also Notes on infrastructure). Then, the CTFd framework didn't meet GDPR requirements, so we needed time to customize it. Moreover, you need a domain name, DNS records, an e-mail address, a security concept, a testing environment, and so on.
- Creating challenges: Creating and testing challenges took most time. It is not only about hiding a flag somehow, but also about writing a story, creating hints that don't immediately disclose the flag, defining categories and points, and writing a full internal walkthrough. One challenge included a technical setup that needed monitoring, validation, and some scripting.
- Announcing the event: We announced the event on different platforms. However, the main lesson was that announcing your CTF on ctftime.org is absolutely essential.
- Writing rules and policies: You not only need some challenges and a technical platform, but also rules and policies.
In total, we started preparations about two months prior to the actual event.
26 challenges were provided, split up into 6 categories:
OSINT was all about gathering intelligence from public sources. This included identifying the location of 3 different industrial sites, and finding the default credentials of an industrial VPN solution.
Crypto contained 3 different challenges. They addressed the risks of rolling your own cryptographic scheme, and using easy-to-guess credentials. Additionally, one challenge was about NaCl (the “Networking and Cryptography library”) to show this library for network communication, encryption, decryption, and signatures.
In total, 8 different forensics challenges were provided. One required connecting to an MQTT broker, and then to subscribing to a topic. Another one included data hidden in the Siemens S7 protocol. Then, there were challenges that required participants to analyze log files, finding leaked data, and conducting a dictionary attack. Steganography (the practice of concealing something within a file) was also part of two challenges.
Trivia contained 4 knowledge-based questions. One addressed the computer worm Stuxnet, another one the WannaCry ransomware attack, and two questions addressed security features of OPC UA, a machine-to-machine communication protocol.
The secret category was unlocked by solving the easiest OSINT challenge. It contained the industrial-themed 4-challenge-long story of Serra Raaphorst, an employee of a Dutch manufacturing company in Den Bosch. Raaphorst asked the CTF participant to discover the “dirty secrets” of her boss. Her story required people to decode DTMF (dual-tone multi-frequency signaling), solving a word search puzzle, and visiting the (digital) Library of Babel.
The final category featured 3 different challenges, which weren't related to industrial security.
The (real) background
Of course, one of the main goals of this CTF is raising awareness of industrial security (or OT security). OT (operational technology) security is different from IT (information technology) security. Good practices of IT security don't apply:
- The overall security objective of OT environments is constant availability, and integrity. In most IT environments, confidentiality has top priority. Then, there are also safety requirements for OT environments since they can cause great harm to people.
- Many OT environments consist of special-purpose hardware running special-purpose software. Oftentimes, such hardware must be available around the clock and it runs for decades. Installing security updates means costly downtime. There is also the risk of changing configuration, breaking things (resulting in even more downtime), or there are just no security updates available for many different reasons. In some cases, updating this special-purpose equipment requires hardware upgrades that cost millions of euros.
- On the network level, some machine-to-machine (M2M) communication has to be in realtime. So packet inspection or other active analysis of network traffic can't be applied.
- Of course, most general purpose IT tools (on the attacker's and on the defender's side) don't work in OT environments. For instance, vulnerability scanners can't identify most vulnerabilities in OT environments, anti-malware software breaks availability, and host-based mitigations are oftentimes not available since most OT components run special-purpose operating systems.
There are many more differences (just look on the internet if you are interested in this topic). Keep all of this in mind when you are reading about attacks on industrial environments next time.
Notes on infrastructure
We won't go into detail here. Basically, the whole setup included four servers:
- Submission server: This server ran a modified CTFd 2.1.5 framework, and Nginx as a proxy server. We moved resources hosted by third parties by default (like fonts and CSS files) to our own installation, and added some security to the whole setup. We modified several CTFd files to meet requirements of the European GDPR. One physical server with 8GB memory was sufficient to handle 1,5k users for five days.
- Challenge server: This server ran the MQTT broker that was part of a challenge. Initially, it was also in use to host a landing page.
- Monitoring server: The monitoring server was only internally accessible. It ran Icinga 2 and monitored system resources as well as user/team registrations and other security-related properties of the public-facing servers. It was also used for alerting us in case of anomalies.
- Mail server: The mail server, yeah, was used to send and receive e-mails.
There was no tracking of users, so there aren't any nice graphs showing the origin of the participants of the CTF. At least, we can show the top 10 teams:
There are 5 lessons learned that we want to share:
- In total, 1,500 users registered and created more than 1,000 teams. However, well-known top CTF teams only used one single user account during the contest. Obviously, they shared their credentials, and accessed their account from many different origins. Keep this in mind if you monitor your setup. Moreover, this means that the actual number of users was slightly higher.
- Most flags were directly shown to participants after solving the challenge. However, there were also some challenges requiring users to write the flag. Only the flag format was given. For instance, people had to identify the name of the company of an industrial structure in a picture. The lesson here was that many people didn't read (or understand) the flag format and general directions regarding these flags. Instructions were shown directly on the challenges page. Maybe, such directions must be part of the challenge's description.
- Two different types of hints were provided: Free general hints, and hints that helped to solve the challenge. Teams had to “pay” 10% of the challenge's points for unlocking hints that helped them to solve a challenge. (For example, solving a challenge resulted in +500 pts. So they needed to “pay” 50 pts.) We think this approach is fair since teams that solve challenges without using hints get more points.
- We decided to exclude some teams for violating one of the 6 (common) rules of the CTF. Some teams tried to brute force the flag by entering hundreds of random strings, others tried to brute force the flag by systematically iterating over possible answers. One challenge was easily solvable by guessing since there were only 6 possible answers. Keep this in mind when designing your challenges, and writing your rules.
- While many participants really liked the CTF (currently, the contest is rated 24.xx out of 25 on ctftime.org, ~45 teams voted), some of them reported solving challenges required guessing. At first glance, this looks like something needs to be changed next time. However, it is likely that most people didn't encounter such challenges before, so they didn't know how to easily solve this. The lesson here is that you shouldn't back off from creating challenges that are different from common CTF challenges.
Follow us on Mastodon:
The vast majority of teams rated this CTF as a challenging and enjoyable experience, and appreciated the industrial style of the event. Our infrastructure worked as expected, and there were no disturbances. So maybe the Syskron Security CTF will be back in 2020. Anyway, hosting CTFs is a good way to raise awareness of information security.
- Syskron Security CTF on ctftime.org (including write-ups, scoreboard, and tasks)external link
- Most challenges on GitHub (including descriptions and files)external link
- Most challenges on GitHub (CTFd, an open-source CTF framework)external link