Resolution Revolution: Turbocharging Security Ticketing Timelines

Resolution Revolution: Turbocharging Security Ticketing Timelines

Speakers: Shrui Data Gupta, Product Security Engineer, Adobe; Joe, Product Security AI and Data Engineer, Adobe

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=5TmIj7gwcP0

Overview

In an era defined by an exponential surge in software vulnerabilities, security teams are perpetually overwhelmed, struggling to manage an ever-increasing volume of security tickets with limited resources. The talk "Resolution Revolution: Turbocharging Security Ticketing Timelines" by Shrui Data Gupta and Joe from Adobe addresses this critical challenge head-on, presenting an innovative approach that leverages Artificial Intelligence (AI) to streamline vulnerability remediation workflows. Their presentation at VulnCon outlines Adobe's journey in developing an in-house AI-powered platform designed to accelerate resolution timelines, improve communication between security engineers and developers, and foster a more proactive security posture.

The core problem tackled is the widening gap between the skyrocketing number of vulnerabilities—a nearly 1000% increase in CVEs since 2010 and 360% since 2017—and the stagnant growth of cybersecurity staffing. This imbalance leads to unmanageable deadlines, unmet SLAs, and a pervasive sense of "security toil" for both security and development teams. Adobe's solution centers on integrating a unified knowledge base with advanced AI capabilities, aiming to redistribute bandwidth, provide tailored guidance, and empower developers to fix vulnerabilities more efficiently, thereby shortening crucial security ticketing timelines.

This article will delve into Adobe's strategic shift from attempting full AI code fixes to a more pragmatic, human-in-the-loop AI assistance model. It will explore the specific AI-driven capabilities developed, including intelligent code/configuration fix recommendations, an expert finder, best practices guidance, and similar ticket identification. The insights shared by Gupta and Joe offer a blueprint for large enterprises navigating complex, heterogeneous environments to enhance their vulnerability management and foster a culture of shared security responsibility.

Background

▶ Watch: Introduction and the escalating vulnerability crisis (0:00)

The landscape of software security is increasingly complex and challenging, characterized by a relentless escalation in the number of reported vulnerabilities. As highlighted in the talk, the volume of CVEs (Common Vulnerabilities and Exposures) has seen an astronomical rise, with projections indicating 40,000-50,000 new CVEs in 2025, representing a 1000% increase since 2010 and a 360% increase since 2017. This surge, partly attributed to changes in how CVEs are tracked, creates an untenable situation for security teams. While the number of vulnerabilities explodes, security resources and staffing levels struggle to keep pace, with cybersecurity positions increasing by only 60-70% since 2017—a stark contrast to the vulnerability growth.

This disparity leads to several critical pain points within organizations, particularly at large enterprises like Adobe:

• Overwhelmed Security Teams: Security engineers are inundated with tickets originating from various sources, including public and invite-only bug bounty programs, internal and external penetration tests, and supply chain vulnerability patching. Triage and support efforts alone are insufficient to manage the sheer volume.
• Unmanageable Deadlines and SLAs: The inability to keep up with vulnerability remediation directly impacts Service Level Agreements (SLAs), leading to missed deadlines and increased organizational risk.
• High Developer-to-Security Ratio: The rapid pace of code generation, often with insufficient secure coding practices, exacerbates the problem. A single security engineer might be responsible for securing the output of hundreds of developers, creating a bottleneck.
• Knowledge Challenges: Critical information needed for remediation is often scattered and fragmented. Communication about vulnerability resolution occurs across disparate channels—Slack, Jira comments, Git comments, in-person discussions—making it difficult to track and consolidate. Documentation is frequently fragmented across wikis, shared drives, and Git repositories, or is simply outdated or non-existent. This leads to significant gaps in knowledge among developers and even security staff regarding specific vulnerabilities or their resolution.

The cumulative impact of unaddressed vulnerabilities is severe, ranging from reputational damage and significant financial losses to personal and systemic risks for the company and its users. Recognizing this, Adobe sought to leverage AI as a strategic tool to redistribute the immense bandwidth demands on both developers and security staff, ultimately aiming to foster a more efficient and secure development lifecycle. The fundamental challenge was bridging the expertise gap between security engineers (focused on exploits, risks, CVEs, CPEs) and developers (focused on code flow, structure, and implementation) to achieve a unified goal of vulnerability remediation.

Key Findings

▶ Watch: Bridging security and developer expertise with AI (4:00)

Adobe's initial foray into using AI for vulnerability resolution began with an ambitious goal: AI code fixing capable of generating automated pull requests. This early experimentation, starting around October 2023, quickly revealed the significant challenges and limitations of current AI capabilities in real-world, complex scenarios. While simple, dummy code examples could be fixed by AI, the reality of production-level code proved far more intricate.

Key findings from this initial phase included:

• Difficulty in Training Data Acquisition: A major hurdle was the scarcity of high-quality, labeled training data. Real-world code files, often hundreds of lines long, rarely come with clear, labeled instances of specific vulnerabilities and their corresponding fixes. Attempting to pull entire repositories to find vulnerabilities or training base LLMs on historical vulnerability logs proved problematic due as the same vulnerable files might appear across multiple tickets, leading to a poor, inconsistent training set.
• Inconsistent Output and Contextual Gaps: AI models struggled with inconsistent output formats and frequently lacked the necessary context for effective fixes. Simple, one-liner issues like basic SQL injection or cross-site scripting (XSS) might be addressable, but complex logic spanning multiple files or dynamic code flows were beyond the AI's current capabilities.
• Limitations of Code Alone: The most crucial insight was that providing code snippets alone was insufficient. Developers and product teams required more robust solutions, even beyond a queued pull request. They needed assurance, context, and a deeper understanding of why a fix was necessary and how it integrated with their specific product's architecture and libraries.

These initial "failures" were not setbacks but critical learning opportunities. Adobe pivoted its strategy, realizing that a full AI code fix was not yet feasible for complex enterprise environments. Instead, the focus shifted to an AI-assisted, human-in-the-loop model. The team identified several effective techniques:

• Advanced Prompting Techniques: Employing methods like chain-of-thought prompting (providing vulnerable code, fixed code, and an explanation of why the fix works) significantly improved AI output quality.
• Quality Measurements and Logging: Implementing robust systems for tracking and evaluating the performance of AI-generated suggestions became essential for continuous improvement.
• Context is King: The more context provided to the AI, the better the results. This included not just code snippets but also Jira ticket information, links to Git files, and detailed product metadata.

By engaging with internal customers—principal scientists, security leaders, and product teams—Adobe identified a clear value proposition: integrating AI with existing knowledge to provide tailored, proactive security suggestions. This led to the development of a customized microservice approach, delivering accurate, product- and vulnerability-specific recommendations, ultimately forming the foundation of their unified knowledge base.

Technical Deep Dive

▶ Watch: Cautionary tale: initial struggles with AI code fixing (5:00)

Adobe's "Resolution Revolution" is built upon a unified knowledge base that merges bug details, code data, and expert insights, accessible through both Jira and Slack. This system is designed around a set of distinct, AI-powered capabilities, each addressing a specific pain point in the vulnerability remediation workflow. The underlying architecture leverages Large Language Models (LLMs), vector databases, and Retrieval Augmented Generation (RAG) principles to provide contextual and actionable intelligence.

The core technology stack employed includes:

• LLM Provider: Azure OpenAI models, chosen for enterprise-level trust and capabilities.
• Orchestration Frameworks: LangChain and LangGraph for building, evaluating, logging, and monitoring LLM-based solutions, and for constructing agentic and graph-related workflows.
• Vector Database: Pinecone, used for storing and efficiently searching vectorized embeddings of various knowledge assets.

Here are the four key capabilities developed:

1. Coder/Configuration Fix:

• Objective: To provide developers with clear, representative examples of how to fix a vulnerability, especially when the initial ticket quality is inconsistent or lacks specific code snippets.
• Mechanism: When a security ticket comes in, a query is sent to an AI orchestrator. The orchestrator utilizes chain-of-thought prompting to pass the vulnerability details, along with explanations and examples, to the LLM. The LLM then generates and returns fix recommendations, which are integrated directly into the security ticket (Jira) or Slack channel.
• Output: The output includes:
• Reason for the code: An explanation of why the code is vulnerable.
• Vulnerable code example: Either an actual snippet from the ticket (if available) or a "representative snippet" demonstrating what the vulnerable pattern might look like.
• Fixed code example: A clear example of how to correct the vulnerability.
• Reason for the fix: An explanation of why the proposed fix works, serving as a sanity check for the LLM's output and aiding developer understanding.
• Benefit: This capability manages expectations by not promising a full PR-ready fix but provides concrete, actionable guidance, reducing the cognitive load on developers.

1. Expert Finder:

• Objective: To quickly identify internal Subject Matter Experts (SMEs) for specific vulnerability types or product areas, reducing wait times and fostering collaboration.
• Mechanism: Historical Jira tickets, particularly their comments, are processed by an LLM to identify individuals who frequently contribute to or resolve specific issues, effectively "assigning" them as experts. This expert information, along with a vectorized version of the ticket content, is stored in a vector database. When a new ticket arrives, its content and metadata are passed to the AI orchestrator, which performs a semantic search in the vector database for similar tickets. The results are then re-ranked, and an expert is extracted based on criteria such as the most frequent resolver or the expert associated with the most similar tickets.
• Output: The Jira ticket is updated with the name of the subject matter expert and a brief explanation of why they are considered an expert for that issue.
• Considerations: The speakers noted potential OGC (Office of General Counsel) or GDPR implications when identifying individuals by name, particularly for European operations, necessitating careful implementation and privacy considerations.

1. Best Practices Guidance:

• Objective: To educate developers on how to fix not just a specific vulnerability instance, but an entire class of vulnerabilities, thereby scaling security guidance and promoting proactive secure coding.
• Mechanism: This capability employs a RAG (Retrieval Augmented Generation) approach. A curated knowledge base containing internal security guidance, platform-specific recommendations, approved libraries, and company tech stack information is vectorized and stored in the Pinecone vector database. When a ticket comes in, the AI orchestrator queries this vector database to retrieve semantically similar guidance related to the vulnerability (e.g., XSS). This retrieved context is then combined with a system prompt and fed to the LLM, which summarizes the information and curates the output for the ticket.
• Output: The guidance includes:
• A summary of the vulnerability (e.g., CWE, type).
• Adobe-specific recommendations: Company-approved libraries or platforms for mitigation.
• Short-term fixes: Immediate actions to resolve the specific bug.
• Long-term fixes: Systemic approaches to address the root cause and prevent future occurrences of the vulnerability class.
• Benefit: This scales manual security guidance, adds crucial company-specific context (e.g., identity and access management platforms), and empowers developers to understand and prevent entire categories of bugs.

1. Similar Tickets:

• Objective: To provide developers with historical context, identify recurring patterns, and help understand the security posture of a product by referencing previously resolved issues.
• Mechanism: All historical tickets (e.g., 15,000-20,000 tickets) are vectorized and stored in the vector database. When a new ticket is created, the AI orchestrator performs a semantic search to retrieve the top similar tickets. A re-ranking logic is then applied to fine-tune the search results, ensuring high accuracy of retrieval. The LLM processes these similar tickets to identify trends and summarize relevant information, which is then added to the new ticket.
• Output: The output includes:
• A count of similar tickets found within a specific timeframe (e.g., "97 other tickets in the last 3 months").
• References to the top similar tickets.
• Statistics on the statuses of these similar tickets (e.g., in progress, blocked, open).
• Severity trends (e.g., how many are high, critical, medium risk).
• Benefit: This is particularly useful for new developers to quickly gain context, helps identify systemic issues (e.g., recurring XSS vulnerabilities in a product), and provides insights into the overall security health of a product line.

Throughout the development of these capabilities, Adobe emphasized continuous feedback and evaluation, involving vulnerability management teams and developers. This human-in-the-loop approach ensured the recommendations were useful, understandable, and actionable. Model output evaluations (LLM evals) were also critical, focusing on metrics like hallucination, accuracy, and crucially, understandability and actionability of the generated guidance. This iterative process of starting small, gathering feedback, and continuously improving quality has been central to their success.

Demo / Proof of Concept

▶ Watch: Why direct AI code fixing wasn't feasible (7:50)

The talk illustrated how these AI-powered capabilities are seamlessly integrated into Adobe's existing developer workflows, primarily through Jira tickets and Slack channels. The goal was to make security guidance accessible and embedded where developers already operate, rather than requiring them to seek it out in separate systems.

For the Coder/Configuration Fix capability, the speakers presented a visual example of how the AI's output is pinned to a Jira ticket. This output is structured clearly, beginning with a section explaining the "Reason for the code" vulnerability. This is followed by either an actual vulnerable code snippet (if available in the original ticket) or a "representative snippet" if the initial ticket was less detailed. Crucially, an "Example of fixed code" is provided, demonstrating the recommended solution, accompanied by a "Reason for the fix" which serves as both an explanation for the developer and a sanity check for the LLM's logic. This ensures developers not only get a fix but also understand the underlying security principle.

The Expert Finder capability was shown to integrate directly into Jira tickets as well. The output simply states the "Subject Matter Expert" and provides a concise explanation of "Why they're the expert," derived from their historical contributions to similar issues. This immediate identification of a go-to person significantly reduces the time developers spend searching for help.

The Best Practices Guidance demonstration showed how Adobe-specific recommendations are tailored and delivered. For an XSS vulnerability, the output included a summary of the bug (e.g., CWE, type) and then offered "Adobe specific recommendation" such as using a particular approved library for mitigating XSS. This was further broken down into "Short-term fixes" for immediate remediation of the specific bug and "Long-term fixes" that encourage developers to think systemically about the class of problem, identifying root causes and preventing future occurrences. This structured guidance aims to educate and empower.

Finally, the Similar Tickets capability was demonstrated by showing how the AI identifies patterns and provides references within Jira. For a newly discovered XSS vulnerability in "Product X," the system might report, "We found 97 other tickets that are very similar to this ticket in the last 3 months." It then provides references to the "Top three tickets" and details their current "States statuses" (e.g., in progress, blocked, open) and "Severity trends" (e.g., how many are high, critical, medium risk). This gives developers and security teams a quick, data-driven overview of the vulnerability's prevalence and impact within their product, aiding in prioritization and strategic planning.

These demonstrations collectively illustrate how Adobe's AI-driven solutions are designed to be practical, context-aware, and seamlessly integrated into existing workflows, reducing friction and enhancing the overall efficiency of vulnerability management.

Defensive Implications

▶ Watch: The crucial insight: humans remain essential in the loop (8:20)

The insights and capabilities developed by Adobe offer significant defensive implications for organizations grappling with escalating vulnerability counts and resource constraints. By strategically applying AI to vulnerability management, security teams can shift from a reactive, overwhelmed posture to a more proactive, intelligent, and scalable defense.

1. Reduce Human Toil and Scale Security Expertise: AI, particularly LLMs, can automate tedious, data-heavy aspects of security workflows that were previously manual. By providing automated fix recommendations, expert identification, and best practices, security engineers can offload repetitive tasks, allowing them to focus on more complex, strategic security challenges. This effectively scales the impact of limited security expertise across a larger developer base.

1. Empower Developers as First-Line Defenders: A crucial defensive strategy is to embed security directly into the development lifecycle. By providing developers with immediate, context-rich, and actionable guidance directly within their existing tools (Jira, Slack), the barrier to secure coding is significantly lowered. Educating developers on why vulnerabilities occur and how to implement long-term fixes transforms them into active participants in security, rather than just recipients of bug reports. This "developer empathy" approach fosters a culture where secure products are built by design.

1. Enhance Context and Accelerate Remediation: The emphasis on "context is king" is a powerful defensive principle. Generic security advice often falls flat. By tailoring AI outputs with company-specific libraries, platforms, and historical data, the recommendations become immediately relevant and actionable. This precision accelerates remediation timelines, reducing the window of exposure to known vulnerabilities. The ability to quickly find experts and reference similar past tickets further reduces friction and learning curves, especially for new team members.

1. Proactive Problem Identification and Systemic Risk Reduction: The "Similar Tickets" capability transforms historical data into predictive intelligence. By identifying recurring patterns of vulnerabilities across products or timeframes, security teams can pinpoint systemic weaknesses, prioritize architectural changes, or implement widespread preventative measures. This moves beyond individual bug fixes to address root causes, significantly reducing the overall attack surface and systemic risk.

1. Build a Unified, Actionable Knowledge Base: Fragmented knowledge is a defensive weakness. Creating a unified knowledge base that is actively leveraged by AI ensures that organizational learning is captured, accessible, and applied consistently. This living repository of best practices, past resolutions, and expert insights becomes a critical asset for continuous improvement in security posture.

1. Iterative and Adaptive Security Improvement: The methodology of starting small, iterating, and gathering continuous human feedback is essential for deploying effective AI in security. This adaptive approach allows organizations to refine their AI models, ensuring they remain relevant and accurate as threats evolve and internal systems change. It also builds trust in the AI's recommendations among end-users.

While the talk focused on in-house development due to Adobe's heterogeneity, smaller companies can leverage vendor solutions that offer similar capabilities. The core defensive takeaway is that AI is not here to replace security professionals but to augment their capabilities, making them more efficient, effective, and capable of handling the scale of modern cybersecurity challenges.

Key Takeaways

• AI for Toil Reduction: AI, especially Large Language Models (LLMs), is a powerful tool for automating manual and tedious aspects of security workflows, significantly reducing "human toil" in vulnerability management.
• Context is King: General-purpose AI models can be highly effective when enriched with specific, tailored context, such as company-approved libraries, product metadata, and historical vulnerability data. This customization is crucial for actionable recommendations.
• Developer Empathy: Empowering developers by making security guidance easy, accessible, and integrated into their existing workflows is paramount. Understanding their pain points and workflows helps foster a culture where secure product development is a shared responsibility.
• Unified Knowledge Base: Consolidating fragmented security knowledge (documentation, expert insights, past resolutions) into a single, AI-searchable knowledge base is critical for consistent, scalable, and efficient vulnerability remediation.
• Iterative Development: Building AI solutions in security requires starting small, continuously iterating based on human feedback, and rigorously evaluating model outputs to ensure accuracy, understandability, and actionability.
• AI is Accessible: Security engineers, even without prior AI expertise, can acquire the necessary skills to build and deploy effective AI solutions, as numerous high-quality resources are available. AI is a tool to empower, not replace, human expertise.

About the Speaker(s)

Shrui Data Gupta is a Product Security Engineer at Adobe, currently part of the AI and data engineering team. With four years of experience at Adobe, Shrui played a key role in developing the AI-powered solutions discussed in the talk, focusing on enhancing security ticketing timelines and fostering a proactive security posture.

Joe is also a Product Security AI and Data Engineer at Adobe, bringing two years of experience at the company to the team. Prior to his work at Adobe, Joe spent 13 years in the US intelligence community, contributing a wealth of experience in complex data environments and security challenges to the development of these innovative AI solutions.