Hacker v. Triage - Inside Bug Bounty Battleground

Hacker v. Triage - Inside Bug Bounty Battleground

Speakers: Richard Hyunho Im (Security Researcher); Denis Smajlović (Principal Security Consultant, Nova Information Security)

Conference: DEF CON

YouTube: https://www.youtube.com/watch?v=D6p8-XAHOJU

Overview

In the DEF CON talk "Hacker v. Triage - Inside Bug Bounty Battleground," security researcher Richard Hyunho Im and Principal Security Consultant Denis Smajlović delve into the often-strained relationship between external security researchers and the internal teams responsible for triaging and resolving vulnerabilities. The presentation offers a candid look at the "theory versus reality" of bug bounty programs, highlighting the common frustrations experienced by researchers due to slow responses, downplayed severities, and opaque communication, while also shedding light on the internal challenges faced by organizations trying to manage these programs effectively.

The speakers, drawing from their distinct experiences—Im as a prolific bug bounty hunter with multiple Apple credits and CVEs, and Smajlović as a consultant optimizing programs for major tech companies—aim to bridge the communication gap. They emphasize the importance of clear, concise reporting from the researcher's perspective and robust, well-resourced internal processes from the program's side. This talk is crucial for anyone involved in the bug bounty ecosystem, offering actionable advice for both researchers seeking to maximize their impact and organizations striving to build more efficient, equitable, and respected vulnerability disclosure programs.

Ultimately, the talk serves as a call for greater understanding and collaboration. By dissecting real-world examples of both successful and frustrating bug bounty interactions, Im and Smajlović illustrate how mutual respect, improved communication, and adherence to established guidelines can transform a potentially adversarial dynamic into a productive partnership that benefits overall security.

Background

▶ Watch: Talk introduction and speakers' backgrounds (0:00)

The concept of bug bounty programs is elegantly simple: security researchers identify vulnerabilities, report them to companies, the companies validate and fix them, and then reward the researchers. However, as Im and Smajlović reveal, the reality is often far more complex and fraught with friction. Researchers frequently encounter a "disconnect" where their diligently reported issues are met with generic responses, prolonged silence, or outright rejection, even for demonstrably valid findings. This leads to a pervasive sense of frustration, particularly when bounties are low, or public recognition is denied without clear explanation.

From the internal perspective, the problem is often rooted in a fundamental misunderstanding of what running a bug bounty program entails. Many organizations, even large tech companies, jump into bug bounty without allocating adequate resources or establishing clear processes. Smajlović notes that companies often underestimate the workload, attempting to assign bug bounty triage to already overburdened on-call engineers who juggle numerous other responsibilities. This results in slow responses, backlogs, and a lack of consistency. Furthermore, internal politics, particularly involving legal and public relations departments, can complicate the process, leading to situations where valid bugs are downplayed or payouts are minimized to avoid perceived liability or negative press.

Prior work in this space often focused on the technical aspects of finding bugs or the high-level benefits of bug bounty. This talk, however, uniquely zooms in on the operational and interpersonal dynamics. It acknowledges that while platforms like HackerOne or Bugcrowd can assist with initial triage, they are only effective if the internal organization provides them with the necessary tools, documentation, and support. Without these foundational elements, issues are escalated prematurely to internal engineers, leading to fatigue and a breakdown of the entire system. The talk thus frames the problem not just as a technical challenge, but as a systemic and cultural one requiring a holistic approach to improvement.

Key Findings

▶ Watch: The 'complicated relationship' with bug bounty programs (2:00)

The talk's central revelation is the deep-seated "disconnect" that often exists between security researchers and the internal teams managing bug bounty programs. Researchers, driven by a desire for recognition, fair compensation, and seeing vulnerabilities fixed, frequently encounter programs that are under-resourced, inconsistent, and opaque. Conversely, internal teams often struggle with overwhelming workloads, unclear internal policies, and a lack of buy-in from various organizational stakeholders, leading to the very frustrations researchers experience.

A critical finding for researchers is the paramount importance of clear, concise, and reproducible reports. While it's tempting to over-explain, the speakers advocate for quality over quantity, focusing on precise steps to reproduce the vulnerability. Richard Im emphasizes "dotting all your T's, all your eyes" and providing a straightforward proof of concept (PoC). Denis Smajlović refines this by stating that security teams don't need a Wikipedia-level explanation of common vulnerabilities like XSS; rather, they need an actionable description like "endpoint X has Y parameter that's not sanitizing Z." This efficiency significantly reduces the internal team's workload and increases the likelihood of a report being actioned.

For organizations, the key findings revolve around program maturity and resource allocation. Many programs suffer from:

1. Lack of dedicated resources: Triage often falls to on-call engineers, leading to low prioritization and backlogs.
2. Unestablished processes: Inconsistent payout guidelines, fragmented ticketing systems, and unclear internal communication workflows hinder efficient resolution.
3. Inadequate support for external triage: Platforms like HackerOne are rendered ineffective if not provided with proper test accounts, documentation, and credits, forcing unnecessary escalations to internal engineers.
4. Internal resistance: Legal and PR departments can inadvertently undermine program goals by pushing for downplayed severities or suppressed payouts, creating a perception of unfairness.

Perhaps the most empowering finding for researchers is the efficacy of polite but firm pushback. Richard's anecdote with Apple, where he successfully appealed an initial $1,000 bounty for a lock screen bypass (CVE-2022-4198) to $5,000 by referencing Apple's own bounty categories, demonstrates that reasonable reconsideration is possible when a researcher articulates their reasoning clearly and respectfully. This highlights that programs, when well-managed, are open to feedback and correction.

Finally, the talk underscores the immense value of community and reputation. A program's reputation among researchers, spread through word-of-mouth and community channels, directly impacts its ability to attract and retain talent. Organizations that foster positive relationships, communicate transparently, and reward fairly are more likely to receive high-quality reports, leading to a stronger security posture.

Technical Deep Dive

▶ Watch: Caricature of common bug bounty submission frustrations (4:00)

While the talk primarily focuses on the operational and interpersonal aspects of bug bounty programs, it touches upon several technical concepts and methodologies crucial for both researchers and triage teams. The core technical interaction revolves around the proof of concept (PoC) and the clarity of its presentation.

Richard Im, drawing from his experience, emphasizes the creation of "very clear, concise, straightforward" PoCs. He advises researchers to write reports as if the reader "has never touched a computer," ensuring every step is easily reproducible. This meticulous approach is vital for vulnerabilities that might involve complex sequences or specific environmental conditions. For instance, his Microsoft Authenticator Face ID bypass involved a 20-second PoC video, demonstrating a direct and unambiguous method to circumvent a security control. While the specifics of the bypass weren't detailed, the implication is a critical flaw in the authentication flow that allowed unauthorized access despite biometric protection.

Denis Smajlović, from the triage perspective, clarifies that while clarity is essential, over-explaining common vulnerabilities like Cross-Site Scripting (XSS) by copying from OWASP cheat sheets or Wikipedia is counterproductive. Triage teams, being security professionals, understand the basics. Instead, the focus should be on pinpointing the exact technical flaw: "endpoint X has Y parameter that's not sanitizing Z." This directness allows the triager to quickly understand the vulnerability, its location, and the immediate impact without sifting through verbose, generic explanations.

The talk mentions specific types of vulnerabilities that highlight the disconnect:

• Authentication Bypass: Richard's Microsoft Authenticator and Google Drive examples both fall into this category. The Microsoft issue allowed bypassing Face ID for MFA codes, a critical security control. The Google Drive issue involved bypassing app lock on an iPhone by using Safari's share functionality to access content, even when the app was purportedly protected. These illustrate how a security mechanism, once bypassed, can render the entire control meaningless, a point often missed or downplayed by internal teams who might argue the phone had to be unlocked anyway.
• Information Disclosure / Lock Screen Bypass: Richard's Apple finding (CVE-2022-4198) involved using Siri to reveal sensitive information from the last foregrounded app (e.g., a ChatGPT conversation) directly from the lock screen. This is a classic example of a privilege escalation or information leak vulnerability, where a restricted state (locked phone) is circumvented to access protected data. The technical aspect here lies in how Siri's functionality, intended for convenience, inadvertently allowed access to app content without requiring full device authentication.

The technical implications also extend to the tools and skills researchers are expected to possess. Richard mentions a "good baseline knowledge of like how to use Burp" and "how to reverse engineer a program." These are fundamental skills for web application and mobile application security testing, respectively, enabling researchers to identify potential attack vectors that might be missed by others. The ability to creatively apply these skills to find novel bypasses or obscure flaws is what distinguishes successful researchers.

In essence, the technical deep dive, while not presenting new exploit techniques, underscores the importance of a clear, technically sound, and reproducible PoC, combined with an understanding of both the vulnerability type and its business impact. This technical rigor, when communicated effectively, is the bedrock upon which successful bug bounty interactions are built.

Demo / Proof of Concept

▶ Watch: Microsoft Face ID bypass reported, received no bounty (4:45)

The talk, while not featuring live demonstrations, effectively communicated several proofs of concept (PoCs) through detailed anecdotes from Richard Im's bug bounty experiences. These examples served to illustrate both the types of vulnerabilities researchers find and the often-frustrating responses they receive.

1. Microsoft Authenticator Face ID Bypass:

• Vulnerability: Richard discovered a bypass for Face ID in Microsoft Authenticator. This meant that despite the app being configured to require biometric authentication to access Multi-Factor Authentication (MFA) codes, an attacker could circumvent this protection.
• Demonstration: Richard reported this issue with a "very clear 20-second PoC" video, leaving no ambiguity about the bypass method.
• Outcome: Microsoft "patched it extremely quickly," which Richard noted was unusual for them. However, they classified the issue as "moderate" severity, stating they don't pay for moderates or issue public advisories for them. They also imposed a 90-day disclosure restriction, despite the low severity claim. Richard later "clowned on them" after the 90 days, highlighting the inconsistency of a "secret" moderate bug.

1. Google Drive App Lock Bypass (iPhone):

• Vulnerability: This issue involved bypassing the app lock feature in Google Drive on an iPhone. If a user had enabled app lock to protect their Google Drive content with Face ID, an attacker could still access the content.
• Demonstration: The PoC involved opening Safari, using the share icon, and then selecting Google Drive. This action allowed access to the Drive's contents without requiring Face ID authentication, effectively bypassing the configured security control.
• Outcome: Google rejected the report, arguing that the phone had to be unlocked in the first place, thus diminishing the security impact. Richard challenged this, pointing out the absurdity of a security feature that fails if the device is already compromised to that extent. This highlights the "security control fails, therefore it wasn't important" logic that can frustrate researchers.

1. Apple Siri Lock Screen Bypass (CVE-2022-4198):

• Vulnerability: Richard found a lock screen bypass vulnerability (assigned CVE-2022-4198) that allowed an attacker to access sensitive information from the last foregrounded application on a locked iPhone using Siri.
• Demonstration: The PoC involved using Siri to "add this to reminders" while the phone was locked. If an application like ChatGPT was the last app used before locking the phone, Siri would display its content (e.g., a sensitive conversation) without requiring the user to unlock the device.
• Outcome: Apple initially offered a $1,000 bounty, claiming the issue didn't neatly fit into their bounty categories. Richard politely appealed, referencing Apple's own guidelines that described "accessing or access to part of one app's contents by bypassing the lock screen without significant or very technical effort on the physical device." His clear, contextualized appeal led Apple to reconsider and increase the bounty to $5,000, demonstrating the power of polite persistence and adherence to program guidelines.

These anecdotes vividly illustrate the practical application of bug bounty research and the diverse responses researchers can encounter, from quick fixes with frustrating payouts to successful appeals based on clear communication.

Defensive Implications

▶ Watch: Internal organizational hurdles affecting bug bounty programs (6:00)

The insights shared in this talk offer crucial defensive implications for organizations operating or considering bug bounty programs. The primary goal is to transform the "battleground" into a collaborative ecosystem, ultimately strengthening an organization's security posture.

1. Prioritize and Resource Bug Bounty Programs Adequately:

• Dedicated Teams: Do not relegate bug bounty triage to already overburdened on-call engineers. Establish a dedicated team or allocate specific, sufficient resources for managing reports. This ensures consistent attention, reduces backlogs, and prevents triager fatigue.
• Clear Responsibilities: Define clear roles and responsibilities for bug bounty program management, from initial triage to internal communication and remediation tracking.
• Budget Allocation: Allocate appropriate budget not just for bounties, but also for tools, training, and personnel required to run the program effectively.

1. Establish Robust Internal Triage and Remediation Workflows:

• Standardized Ticketing: Implement a well-functioning ticketing system (e.g., Jira) that allows for consistent issue filing, automatic assignment to the correct teams, and clear progress tracking. Avoid fragmented systems where different teams use different trackers.
• Clear Payout Guidelines: Develop and publish consistent guidelines for bounty payouts based on severity, impact, and scope. Inconsistency (e.g., paying $1,000 for a low-severity issue but $800 for a worse one) erodes trust and frustrates researchers.
• Internal Communication: Foster strong internal rapport between the bug bounty team and development/product teams. This allows for quick, informal consultations and ensures that reported issues are integrated into the existing development lifecycle. Teams should be aware of the bug bounty program and its importance.

1. Empower External Triage Partners:

• Provide Necessary Tools: If utilizing external triage services (e.g., HackerOne, Bugcrowd), equip them with the tools they need to effectively verify vulnerabilities. This includes proper test accounts (with credits, if applicable, for testing purchase flows), up-to-date documentation about the product/service, and clear verification procedures.
• Reduce Escalations: By empowering external triagers, organizations can significantly reduce the number of irrelevant or poorly described issues escalated to internal engineers, allowing the internal team to focus on confirmed, high-impact vulnerabilities.

1. Foster Transparent and Sincere Communication:

• Avoid Generic Responses: Generic "thank you" messages or prolonged silence are major sources of researcher frustration. Even if full details cannot be disclosed, sincere communication about the status, limitations, and reasoning behind decisions (e.g., severity classification) is crucial.
• Be Open to Pushback: Encourage polite and firm challenges from researchers. This provides valuable external data points and can lead to reconsiderations, as demonstrated by Richard's Apple bounty appeal.
• Manage Expectations: Clearly communicate program scope, rules, and expected timelines. If delays occur, provide updates.

1. Secure Organizational Buy-in:

• Cross-Functional Alignment: Ensure that all relevant internal departments—especially legal, public relations, and executive leadership—understand and agree upon the fundamental principles and operational realities of the bug bounty program. This prevents situations where legal concerns override fair compensation or PR fears lead to downplaying legitimate issues.
• Commitment to Disclosure: If a program promises public advisories or recognition, ensure the organization is prepared to follow through, even for critical findings.

1. Cultivate a Positive Reputation and Community Engagement:

• Researcher Retention: A well-run program with fair payouts and good communication will attract and retain top researchers. Researchers talk to each other, and a positive reputation is invaluable for scaling a program.
• Community Involvement: Utilize social media, partner platforms, and events like live hacking sessions to engage with the security community and promote the program. This attracts new talent and fosters collaboration.

By implementing these defensive strategies, organizations can move from a reactive, frustrating "battleground" to a proactive, collaborative environment where security researchers are valued partners in identifying and mitigating risks.

Key Takeaways

• Clarity and Conciseness are King for Researchers: Submit short, to-the-point reports with clear, easily reproducible proof-of-concept steps. Avoid verbose, generic explanations of common vulnerabilities. This significantly increases the likelihood of your report being actioned efficiently.
• Polite Persistence Pays Off: Researchers should not hesitate to politely and firmly challenge program decisions, especially regarding severity or bounty amounts, by referencing program guidelines and providing additional context. The Apple Siri lock screen bypass example demonstrates that reasonable reconsideration is possible.
• Organizations Must Prioritize and Resource Bug Bounty Programs: Effective programs require dedicated resources, clear internal processes, consistent payout guidelines, and support for external triage teams. Relying on overburdened on-call staff leads to backlogs, frustration, and program failure.
• Transparent and Sincere Communication is Crucial: Both sides benefit from open communication. Programs should avoid generic responses and provide honest feedback, even when full details cannot be revealed. Researchers should communicate respectfully, even when frustrated.
• Community and Reputation Drive Success: A program's reputation among researchers directly impacts its ability to attract high-quality submissions and retain talent. Organizations should actively engage with the security community and foster positive relationships to scale their programs.
• It's Never Too Late to Start Bug Bounty Research: Aspiring researchers should not be deterred by the perceived saturation of the field. Persistence, continuous learning of modern techniques (e.g., Burp Suite, reverse engineering), and creative thinking can still lead to significant findings and successful bug bounty careers.

About the Speaker(s)

Richard Hyunho Im is a dedicated security researcher with a relatively short but highly impactful career in bug bounty. In just a year and a half, he has achieved significant recognition, including being credited by Apple over 15 times, earning three CVEs (with one more pending), and ranking among the top 25 in OpenAI's bug bounty program. He has also received credits from Google and Microsoft for his vulnerability findings. Richard's experiences highlight the perspective of a successful, persistent researcher navigating the complexities of various bug bounty programs.

Denis Smajlović is a Principal Security Consultant at Nova Information Security. With a background spanning penetration testing and software development, Denis brings an invaluable internal perspective to the bug bounty landscape. His professional focus involves assisting organizations, including multiple big tech companies in the Bay Area, in establishing and optimizing their bug bounty programs. Denis's expertise lies in understanding the operational challenges, internal processes, and strategic improvements necessary for companies to run efficient and effective vulnerability disclosure initiatives.