How playing D&D at work can improve your incident response? - Hans Metsoja

How playing D&D at work can improve your incident response?

Speakers: Hans Metsoja, Security Engineer, Opera

Conference: Disobey

YouTube: https://www.youtube.com/watch?v=e_vQ3ZTubTg

Overview

In this insightful talk from Disobey, Hans Metsoja of Opera presented a compelling case for leveraging tabletop simulations, inspired by games like Dungeons & Dragons, to dramatically enhance an organization's incident response capabilities. Moving beyond conventional training methodologies, Metsoja detailed how Opera designed and implemented a gamified, immersive experience to train employees, validate procedures, and foster critical communication and decision-making skills under pressure. The core premise is that by simulating real-world security incidents in a safe, controlled environment, teams can practice their roles, identify procedural weaknesses, and develop the crucial soft skills often overlooked in purely technical training.

The presentation highlighted the unique challenges of incident response, where time is scarce, data is incomplete, and the pressure to make impactful decisions is immense, often leading to human freezing or missteps. Metsoja argued that traditional training, while valuable, often falls short in preparing individuals for this high-stakes environment. Tabletop simulations, by contrast, offer a reusable, feedback-rich, and gamified approach that addresses these gaps, making them a powerful tool for any organization serious about bolstering its cybersecurity resilience.

This approach is particularly relevant given the ever-increasing complexity and frequency of cyberattacks. Organizations are not just looking for technical fixes but for robust, adaptable teams capable of coordinating effectively across different departments—from technical and security teams to legal, PR, and leadership—during a crisis. Metsoja's work at Opera demonstrates a practical, innovative method to cultivate these capabilities, proving that an investment in "play" can yield significant security dividends.

Background

▶ Watch: Introduction: Why incident response training is crucial (0:00)

The genesis of Opera's tabletop simulation initiative stemmed from a standard, yet critical, organizational undertaking: a comprehensive redesign of their incident response (IR) procedures. With new processes, escalation paths, and responsibilities defined, there was an immediate need for effective training for personnel and a robust method to validate whether these newly minted procedures would actually work under real-world conditions. Furthermore, compliance requirements necessitated regular and documented training for all relevant staff.

Traditional security training methods, while having their place, often present a trade-off between usefulness and cost/reusability. Generic video trainings are cheap and scalable but lack engagement and tailored feedback. Classroom trainings offer more interaction but are still broad. Technical labs or CTFs provide hands-on experience for specific skills but might not cover the broader organizational response. Custom, tailor-made trainings are highly effective for specific teams but are expensive and have limited reusability. The challenge for Opera was to find a solution that offered the benefits of tailored, engaging training without the prohibitive cost and single-use limitation.

Incident response, in particular, demands a different kind of preparation. Unlike strategic business decisions where ample time and data are available, IR scenarios require rapid decision-making with incomplete information, often under extreme pressure. This environment can cause individuals to "freeze up" or make suboptimal choices. Metsoja observed that humans are generally poor at making quick, high-stakes decisions without sufficient data, a common characteristic of an unfolding security incident. Opera sought a training methodology that could bridge this gap, allowing individuals to practice these critical skills in a low-stakes environment, receive feedback, and build muscle memory for effective incident management. The answer, they posited, lay in gamified tabletop simulations that focus on communication and decision-making, rather than purely technical problem-solving.

Key Findings

▶ Watch: Introducing tabletop simulations for incident response training (4:40)

Opera's extensive experience running 13 tabletop simulations over 1.5 years, involving over 160 participants across five geographical locations, yielded several profound insights into effective incident response training. The most significant finding was the immense value of tabletop simulations in enhancing communication and decision-making skills, which are often the weakest links during real-world incidents. These simulations provided a safe space for participants to make mistakes and learn from them without real-world consequences, simultaneously exposing flaws in existing IR procedures that might otherwise only surface during a live breach.

A crucial design philosophy behind Opera's simulations was the Kobayashi Maru principle – all scenarios were designed to fail. Participants were unaware of this, believing they could "win," but the scenarios were intentionally difficult, booby-trapped, full of distractions, or progressively worsened regardless of player actions. This approach was instrumental in highlighting several common pitfalls:

• Incident Recognition: Teams frequently struggled with understanding when to declare a formal incident, often delaying the "big red button" moment while trying to gather more information, leading to worsened outcomes. This underscored the importance of reporting sooner rather than later.
• Taking the Game Seriously: Some participants initially viewed the simulation as "just a game," believing they would make all the "right" decisions in a real incident. The failing scenarios demonstrated that mistakes made in a low-pressure environment are likely to be repeated, or amplified, under real pressure.
• Problem-Focused vs. Holistic Response: Highly skilled engineers often focused solely on technical problem-solving (e.g., patching a system) and neglected broader incident management requirements, such as notifying legal teams, the DPO, or press relations. The simulations effectively illustrated the multi-faceted nature of incident response.
• Blaming Culture: Early in some games, teams engaged in unproductive blame-seeking. However, participants quickly realized this hindered mitigation efforts, reinforcing the need to prioritize stopping the incident over determining fault.
• Distractions and False Positives: Scenarios often included multiple, concurrent incidents or red herrings, simulating the chaotic nature of real-world breaches where focus can be easily lost. This trained teams to prioritize and manage competing threats.
• Miscommunication is Inevitable: Despite explicit rules, miscommunication was a pervasive issue in every single game. Teams failed to formally exchange information, leading to duplicated efforts, missed actions, or critical delays. This finding alone highlighted the paramount importance of clear, structured communication.
• Decision Paralysis: Participants often hesitated to make critical decisions, such as taking down a compromised but business-critical system, due to fear of repercussions or lack of clear authority. The simulations provided a context to practice these tough calls and identify necessary escalation paths.

Perhaps the most encouraging finding was the demonstrable learning curve. Control groups who played the game a second time (with a different scenario after about a year) showed significantly better communication skills and a higher self-perceived readiness to tackle incidents compared to first-time players. This empirical evidence validates the effectiveness of these simulations as a continuous learning tool, despite the interesting observation that regardless of actual performance, all teams (both first-time and repeat players) generally perceived their performance as "good" or "okay." This suggests that the learning is often subconscious or reflected in improved processes rather than immediate self-assessment of performance.

Technical Deep Dive

▶ Watch: Key benefits: safe practice, finding procedure errors (6:00)

The core of Opera's successful incident response training lies in its meticulously designed tabletop simulation, which borrows heavily from game design principles. Hans Metsoja, drawing on his extensive background in board games and RPGs, crafted a system that is both engaging and highly effective for learning.

Game Design Philosophy:

Metsoja's initial foray into game design for this purpose involved several key considerations. The scenarios needed to be semi-realistic and relatable to the company's operations but not an exact replica of their specific technical setup. This deliberate abstraction prevents highly specialized engineers from "breaking the game" by applying precise, real-world knowledge that might bypass the intended learning objectives related to process and communication.

Key challenges in designing the game included:

1. Time Progression: Reconciling real-time discussion (e.g., half an hour) with accelerated game time (e.g., two days passing). Clear mechanics were needed to communicate time shifts to players.
2. Team Control and Communication: Establishing rules and a framework to manage interactions among multiple teams, ensuring formal information exchange.
3. Avoiding Dead Ends: Designing a scenario flow that, while challenging, always provides a path forward, preventing players from getting stuck.

Metsoja's design process initially involved a labor-intensive decision tree or graph mapping approach. This meant predicting all possible questions, actions, and outcomes that teams might take based on an initial event. This iterative process, which took 3-4 weeks for the first iteration, involved physically mapping out scenarios with printed papers and strings to visualize all potential branches and ensure game coherence.

Game Structure and Roles:

A typical simulation involves approximately 15 participants, divided into several cross-functional teams, mirroring real-world organizational structures during an incident:

• Technical Team: Focuses on technical investigation and mitigation.
• Product Team: Represents business interests and impact.
• Legal or Privacy Team: Addresses compliance, data breach notification, and legal implications.
• Centralized IT Team / IT Support: Handles infrastructure and user support aspects.
• PR or Communication Team: Manages external and internal messaging.
• Security Team: Coordinates overall security response and investigation.

In addition to these active player roles, the simulation incorporates external elements:

• External Media: Can act as an antagonist or a source of pressure, influencing the scenario.
• Social Media Influence and Posts: Simulate public perception and potential reputational damage.
• Dungeon Master (DM): The critical role, controlled by Metsoja, who manages the game's progression, introduces new information (injects), adjudicates rules, and ensures the scenario remains dynamic.

Game Rules and Mechanics:

The rules are fundamental to enforcing the learning objectives and simulating real-world constraints:

• Formal Information Exchange: Players can only use information formally communicated to their team. Overhearing conversations is not considered formal knowledge. This forces explicit communication.
• Partial Information: Acting on incomplete data is a core challenge, mirroring real incidents.
• Normal Business Operations: The default state is that business continues, preventing premature panic.
• Proportional Actions: Responses must be proportionate to the known information. Discovering a USB stick, for example, does not immediately warrant shutting down all live services. This prevents players from overreacting based on the assumption that "it's a training, so it must get worse."
• Player-Declared Incident: Teams must explicitly decide when to press the "big red button" and declare an incident, a critical decision point often fraught with hesitation in real life.

The DM uses injects—digital or physical slips of paper—to deliver critical pieces of data, new threats, or contextual information, driving the narrative forward. The DM also plays a crucial role in managing the game's pace, using mechanics to speed up or slow down time as needed, and in controlling player behavior, ensuring everyone is included and that no single team "speedruns" the scenario or attempts to "break the game" by focusing on irrelevant technical minutiae over the broader incident. The Kobayashi Maru design, where scenarios are engineered to be unwinnable, is a central mechanic that ensures learning occurs through navigating failure, rather than achieving a perfect solution. This forces players to confront difficult choices, manage escalating crises, and prioritize damage limitation.

Demo / Proof of Concept

▶ Watch: Challenges in designing a tabletop incident response game (6:50)

While the talk did not feature a live technical demonstration or a specific software proof-of-concept, the speaker extensively detailed the practical implementation and outcomes of running 13 tabletop simulations within Opera. The "proof of concept" is the methodology itself, demonstrated through the consistent structure, rules, and learning objectives applied across all these games. The core of the demonstration was the evidence of improved communication and decision-making skills among participants, particularly those in the control groups who played multiple scenarios over time. The speaker illustrated how the designed scenarios, injects, and moderation techniques functioned as the "demo" of their approach, highlighting the common failures and successes observed in the 160+ participants who engaged in this unique training over 1.5 years. The detailed description of the game design, including the decision tree mapping and the Kobayashi Maru philosophy, served to explain how the concept was proven effective.

Defensive Implications

▶ Watch: Mapping out scenarios: the decision tree design process (8:00)

The insights from Opera's tabletop simulations offer actionable strategies for cybersecurity defenders looking to enhance their incident response capabilities beyond traditional technical training.

1. Implement Cross-Functional Tabletop Exercises: Organizations should regularly conduct similar tabletop simulations involving not just security and technical teams, but also legal, privacy (DPO), PR/communications, product, and leadership. This fosters a holistic understanding of incident impact and response, breaking down silos.
2. Validate and Refine IR Procedures: Use these simulations as a living testbed for existing IR plans, checklists, and escalation matrices. The "safe space" environment allows for the identification of ambiguities, gaps, or bottlenecks in procedures without the pressure of a real incident. Any procedural errors identified during a game should lead to immediate updates to official documentation.
3. Prioritize Communication Training: Given that miscommunication was a consistent failure point, organizations must explicitly train for clear, formal, and timely information exchange during incidents. This includes establishing clear protocols for cross-team updates, documentation, and formal handovers. Emphasis should be placed on who needs to know what and when.
4. Cultivate Decision-Making Under Pressure: Train teams to make timely decisions, even with incomplete data. Encourage a bias towards action over paralysis, provided that communication channels are robust enough to allow for course correction. This involves empowering individuals to make critical calls and understanding the escalation paths for high-stakes decisions (e.g., system takedowns).
5. Expand Engineer's Perspective: Ensure technical responders understand their role within the broader incident lifecycle. Training should highlight the importance of engaging legal, DPO, and PR teams, especially in data breach scenarios, beyond merely patching systems or containing technical threats.
6. Prepare for Distractions and Multiple Incidents: Incorporate scenarios with concurrent incidents, false positives, or red herrings. This prepares teams for the chaotic reality where focus can be easily diverted, and resource allocation becomes critical.
7. Engage Leadership in Simulations: Leadership involvement in these exercises can be invaluable. It helps them understand the complexities of incident response, the tough decisions that need to be made, and their role in providing approvals or strategic guidance during a crisis.
8. Iterative Improvement and Measurement: Treat tabletop simulations as an ongoing program. Run different scenarios, measure performance (e.g., adherence to process, communication effectiveness), and collect feedback to continuously refine both the training itself and the underlying IR procedures. The observed improvements in repeat players underscore the value of regular practice.
9. Proactive External Communication Planning: The challenges faced by simulated PR teams highlight the need for pre-approved communication templates, holding statements, and clear protocols for engaging with media and the public during a crisis, even when full information is not yet available. "No comment" or delayed responses can be far more damaging.

By adopting these defensive implications, organizations can move towards a more resilient, coordinated, and effective incident response posture, capable of navigating the complex challenges of modern cyber threats.

Key Takeaways

• Tabletop simulations are highly effective: Gamified tabletop exercises significantly enhance incident response capabilities by focusing on critical communication and decision-making skills under pressure.
• "Kobayashi Maru" design is crucial: Scenarios intentionally designed to fail effectively expose procedural weaknesses, common human errors, and critical gaps in incident response, fostering deeper learning than "winnable" exercises.
• Miscommunication is a pervasive problem: These simulations consistently highlight that communication failures are a primary cause of incident mismanagement, underscoring the need for explicit training in formal information exchange across teams.
• Decisive action beats paralysis: Making a quick decision, even if imperfect, coupled with strong communication for course correction, is generally more effective than delaying action while seeking perfect information.
• Cross-functional understanding is key: Participants gain a vital appreciation for the roles and contributions of different departments (legal, PR, product, technical) during an incident, fostering better collaboration.
• Continuous practice yields results: Repeat participation in varied scenarios leads to measurable improvements in communication, process adherence, and overall team readiness for real-world incidents.

About the Speaker(s)

Hans Metsoja is a Security Engineer at Opera, the company known for its web browsers. He brings a unique perspective to cybersecurity training, drawing heavily on his personal passion and extensive experience with board games, Dungeons & Dragons, and video games. This background informed his innovative approach to designing and implementing gamified tabletop simulations for incident response training. Metsoja played a pivotal role in redesigning Opera's incident response procedures and subsequently developed these simulations as a practical method to train staff and validate the efficacy of the new processes. His work demonstrates a creative application of game design principles to solve complex organizational security challenges.