Securing the Future: Navigating AI Vulnerabilities and Evolving Security Practices

Securing the Future: Navigating AI Vulnerabilities and Evolving Security Practices

Speakers: Lisa Bradley, Senior Director, Product and Application Security, Dell Technologies; Sarah Evans, Security Innovation Research, Office of the CTO, Dell Technologies

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=7FwUNeAT4y8

Overview

This talk, "Securing the Future: Navigating AI Vulnerabilities and Evolving Security Practices," delivered by Lisa Bradley and Sarah Evans of Dell Technologies at VulnCon 2025, addresses the burgeoning security challenges introduced by the rapid integration of Artificial Intelligence (AI) and Machine Learning (ML) into modern software products. As AI transitions from niche applications like personalized recommendations to core functionalities such as code generation, bug identification, and real-time decision-making, it fundamentally alters the cybersecurity landscape, presenting new attack surfaces and demanding a proactive evolution of existing security paradigms.

The speakers highlight that while AI offers immense benefits, its hurried adoption often overlooks critical security considerations, creating fertile ground for novel vulnerabilities and sophisticated attacks. The presentation meticulously dissects the new risk areas emerging within AI systems, emphasizing the need to protect not only the data fed into these systems but also the intricate AI-specific components—such as training models, inference models, and their responses—alongside the underlying application infrastructure.

This talk is critically important for anyone involved in software development, product security, IT operations, and compliance. It underscores that traditional vulnerability management and supply chain security practices are insufficient for the complexities of AI. Bradley and Evans advocate for a comprehensive, converged approach—termed Secure X-Ops—and propose a Universal Attestation Framework to ensure the integrity, traceability, and resilience of AI-powered products against an evolving threat landscape. Their insights offer a roadmap for organizations to proactively adapt their security strategies, fostering collaboration and leveraging machine-readable attestations to navigate the future of AI securely.

Background

▶ Watch: Talk Introduction and Speaker Backgrounds (0:00)

The integration of AI into software products has seen exponential growth, evolving from simple personalized content and recommendations to sophisticated capabilities. Initially, AI augmented user experiences through chatbots and virtual assistants. More recently, its role has expanded dramatically, with AI tools now capable of generating code snippets, identifying bugs, supporting coding efforts, and enhancing automation testing for higher software quality. Beyond development, AI is increasingly employed for analyzing historical data to predict future trends, behaviors, and processing real-time data for rapid decision-making, fundamentally enhancing product capabilities across the board.

This rapid expansion, however, has introduced entirely new dimensions of risk. Sarah Evans, bringing her background in IT and security operations, emphasized the need to understand these novel risks inherent in machine learning models and large language models (LLMs). Internally at Dell, a dedicated threat modeling team has been developing a library of AI threats, while also leveraging crucial industry resources like the OWASP Top 10 for Machine Learning and the OWASP Top 10 for Large Language Models. These resources provide a structured way to identify where AI components introduce new or expanded risks within an application's architecture.

The speakers articulate three critical aspects that demand attention when securing AI. First, protecting the data that is fed into the AI system is paramount. This includes proprietary information and intellectual property (IP) that, if leaked or compromised, could have severe consequences. Second, security hygiene must extend to the AI-specific components themselves, encompassing training models, inference models, their responses, and the underlying training data. Each of these elements represents a potential point of vulnerability. Lastly, it is crucial not to neglect the foundational application security hygiene of the system on which the AI operates. The rapid pace of AI adoption often leads to shortcuts, making systems vulnerable if basic security principles are not rigorously applied. Consequently, AI-related vulnerabilities can manifest across all stages of the AI life cycle and supply chain, from data sourcing and model training to deployment and ongoing maintenance. This can lead to insidious issues, such as a vulnerability in the training data resulting in malicious or incorrect output for end-users, requiring a new level of communication and action from product maintainers to consumers.

Key Findings

▶ Watch: Identifying AI Threat Sources and Mitigating Risk (4:00)

The talk identifies several critical findings that underscore the urgency and complexity of securing AI in modern applications:

1. Three Main Threat Areas: The speakers predict an increase in threat actor activity within three core areas:

• Poor System Architecture and Operational Hygiene: The rapid adoption of AI often leads to rushed implementations, resulting in inadequate security controls and design flaws that create rich attack surfaces.
• Poorly Integrated Supply Chain Security: The reliance on upstream AI technology, including third-party data sets, pre-trained models, and plugins, introduces significant supply chain risks. A vulnerability in any of these external components can compromise the entire system, especially if robust security practices are not enforced across all suppliers.
• Inadequate Data Management and Classification: Insufficient management and classification of data fed into and processed by AI systems can lead to Intellectual Property (IP) leakage or infiltration of data sets, directly impacting the integrity and confidentiality of AI operations.

1. Multifaceted Classification of AI Vulnerabilities: Unlike traditional software vulnerabilities, AI vulnerabilities require a more nuanced classification approach. This involves considering:

• The stage in the AI life cycle where the vulnerability is identified (e.g., sourcing, training, hosting, production, maintenance).
• The goals of the bad actor (e.g., sabotage, fraud, IP theft).
• The attack types employed (e.g., poisoning, interference, backdooring, reprogramming, evasion). This collective consideration is essential for effectively defining and assigning vulnerabilities, potentially even for CVE (Common Vulnerabilities and Exposures) assignment in this nascent space.

1. AI Supply Chain as a Primary Attack Vector: The concept of a supply chain extends beyond physical components to encompass third-party data sets, pre-trained models, and plugins used in AI development. Just like any traditional supply chain, AI is only as secure as its weakest link. Attacks can involve modifying or replacing machine learning libraries or models, or poisoning training data samples, underscoring the critical need for rigorous security checks throughout the AI development pipeline.

1. Extension of Existing Security Practices is Imperative: The talk strongly advocates for extending established software vulnerability remediation practices to encompass AI components. This includes developing comprehensive inventories of AI assets, leveraging AI Software Bills of Materials (AI SBOMs), and coordinating communication plans across engineering teams to respond effectively when AI-specific vulnerabilities are reported. This proactive approach is vital for enterprise-level security and compliance.

1. The Need for a Universal Attestation Framework: To address the complex, interconnected nature of AI security, the speakers propose a Universal Attestation Framework. This framework aims to be automated, integrated, and easy to adopt, designed to achieve three core objectives: establish integrity, report on that integrity, and verify that integrity across all AI components and stages. It represents a convergence of attestations from various domains (hardware, software, data, models, runtime environments) to meet evolving compliance requirements and enable rapid, verifiable incident response.

These findings collectively highlight that AI security is not merely an extension of traditional cybersecurity but a distinct, complex domain requiring a fundamental shift in how organizations approach risk, vulnerability management, and compliance.

Technical Deep Dive

▶ Watch: Applying OWASP Top 10 to Machine Learning Models (5:30)

The technical core of the presentation delves into specific AI threat sources, vulnerability types, and the architectural implications for robust security. Sarah Evans highlighted the utility of frameworks like the OWASP Top 10 for Machine Learning and OWASP Top 10 for Large Language Models as foundational resources for identifying and categorizing AI-specific threats, mapping them to different architectural components within an AI application.

A significant focus was placed on data poisoning, a prevalent type of cyber attack where malicious actors intentionally manipulate the data used to develop or train AI/ML models. Several manifestations of data poisoning were detailed:

• Label Flipping: In this attack, correct labels within the training data are swapped with incorrect ones, misleading the model during its learning phase. For instance, an image correctly labeled "cat" might be maliciously relabeled "dog," causing the model to misclassify similar images in the future.
• Data Injection: This involves adding entirely new, malicious data points to the training set. These injected data points are designed to corrupt the model's learning process or introduce specific biases.
• Backdoor Attack: A more sophisticated form of poisoning, a backdoor attack trains the model to behave normally under most circumstances but to exhibit a specific, often malicious, behavior when a particular "trigger" or input pattern is present. This allows an attacker to control the model's output under specific conditions without immediately revealing the compromise.
• Modifying Training Data Without Changing Labels: This subtle attack alters the features or content of training data points while keeping their original (correct) labels. The goal is to subtly shift the model's decision boundaries or introduce vulnerabilities that might not be immediately apparent through label checks. The overarching consequence of data poisoning is that end-users, when interacting with the poisoned system, are likely to receive bad or malicious output.

The concept of AI supply chain attacks was extensively explored, shifting the traditional understanding of supply chain security from physical components (like those in a laptop) to the digital assets integral to AI. This includes third-party data sets, pre-trained models, and plugins. The analogy of "you're only as secure as your weakest point" applies directly to this digital supply chain. An attack can occur if an attacker modifies or replaces a machine learning library or model within the system, especially if it involves poisoning a sample within the training data. This necessitates rigorous vetting and security practices for all suppliers and components throughout the AI development and deployment pipeline.

To combat these evolving threats, the talk emphasizes the need to extend existing vulnerability remediation practices to new AI components. This requires a comprehensive inventory that goes beyond traditional software to include:

• Data sets: The raw data used for training and validation.
• Models: The trained AI/ML models themselves.
• Model cards: Metadata accompanying a model, detailing its training data, intended use, performance, and ethical considerations.
• Vector databases: Specialized databases optimized for storing and querying high-dimensional vectors, often used in large language models for efficient retrieval of relevant information.
• Retrieval Augmented Generation (RAG) architecture: A design pattern for LLMs that combines retrieval of relevant information from a knowledge base with text generation, enhancing accuracy and reducing hallucinations.
• Agents and agent architectures and frameworks: Software components that can perceive their environment, make decisions, and take actions, often leveraging LLMs.
• Knowledge graphs: Structured representations of knowledge that capture entities, relationships, and semantic information, increasingly used to augment AI systems.

The speakers highlighted Google's Controls for AI Supply Chain Security, which offer four core recommendations: capture metadata, increase integrity, organize your metadata, and share with others. These principles are fundamental for building traceable, auditable, and secure AI systems.

A pivotal concept introduced was Secure X-Ops, representing the necessary convergence of various operational domains: DevOps, DevSecOps, DataOps, and ML/LLM Ops. This unified approach is essential for building, deploying, and responding to incidents involving complex AI components. The fragmented nature of these operations in many organizations creates gaps that threat actors can exploit.

Finally, the talk outlined the vision for a Universal Attestation Framework. This framework would need to be automated, integrated, and easy to adopt, focusing on three key actions:

1. Establish integrity: Ensuring that components (hardware, software, data, models) are in a known, trusted state.
2. Report on that integrity: Generating machine-readable evidence of integrity at various stages.
3. Verify that integrity: Allowing for independent verification of these attestations.

This framework aims to converge attestations from traditionally separate domains—secure hardware, data sets, secure software development practices, runtime environments, and models/model cards—into a cohesive, verifiable system. Such a framework would be critical for demonstrating due diligence, meeting compliance requirements like the EU's Cyber Resiliency Act, and enabling rapid, informed responses to AI-related security incidents.

Demo / Proof of Concept

▶ Watch: Three Critical Aspects for Securing AI Vulnerabilities (6:00)

While the talk did not feature a live technical demonstration or a hands-on proof of concept in the traditional sense, it effectively presented a compelling use case to illustrate the practical implications of evolving AI vulnerabilities and the necessity for the proposed security advancements.

The primary use case discussed revolved around the Cyber Resiliency Act (CRA), a significant consumer compliance regulation originating from the European Union. This act applies to devices with digital elements, encompassing a broad range of products from laptops and infrastructure to standalone software. The CRA mandates that manufacturers and vendors must:

1. Ensure products are shipped without known exploitable vulnerabilities.
2. Rapidly report any known exploitable vulnerabilities discovered after a product has been deployed and is in customer use.
3. Maintain patches and security updates for those systems over time.

The speakers then projected how this regulation would extend to AI vulnerabilities in the coming years. They posed a critical scenario: what if a data set used to train a model was later discovered to have been poisoned? Under an AI-inclusive CRA, an organization would be legally obligated to:

• Immediately identify all products containing models trained on that compromised data set.
• Notify affected customers about the potential impact.
• Swap out the vulnerable model or provide remediation.

This scenario highlights a significant gap in current enterprise security operations. To comply, organizations would require machine-readable data points and robust due diligence throughout their development processes. This means being able to trace precisely:

• Which models were trained on specific data sets.
• Where models were fine-tuned with potentially compromised data.
• How these models were eventually rolled up into containers and applications, and subsequently shared with customers.

This use case serves as a powerful illustration of why the industry needs to proactively develop capabilities for inventorying AI components, creating AI Software Bills of Materials (AI SBOMs), and implementing comprehensive attestation mechanisms. It underscores the future imperative for organizations to not only know about vulnerabilities but also to prove their diligence and rapidly respond across the entire AI supply chain, from data ingestion to final product deployment.

Defensive Implications

▶ Watch: AI Vulnerabilities Found Across the Entire Lifecycle (8:00)

The detailed analysis of AI vulnerabilities and evolving threat landscapes presented in this talk yields several crucial defensive implications for organizations integrating AI into their products and services:

1. Proactive Extension of Security Practices: The most significant defensive implication is the need to proactively integrate AI-specific risks into existing security frameworks rather than treating AI as an isolated concern. This means extending established vulnerability management processes, supply chain security protocols, and compliance mechanisms to explicitly cover AI components.

1. Comprehensive AI Inventory and AI SBOMs: Organizations must develop and maintain exhaustive inventories of all AI components. This includes not only the AI/ML models themselves but also the data sets used for training, model cards detailing their provenance and characteristics, vector databases, RAG architectures, AI agents, and knowledge graphs. The adoption of AI Software Bill of Materials (AI SBOMs) will be critical to track dependencies, identify potential vulnerabilities upstream, and facilitate rapid response to security incidents.

1. Enhanced Data Protection and Management: Given the prevalence of data poisoning and IP leakage risks, robust controls for data management and classification are paramount. This involves implementing stringent access controls, encryption, integrity checks, and data lineage tracking for all data used in AI training and inference. Organizations must ensure that proprietary or sensitive information is rigorously protected from infiltration and unauthorized disclosure.

1. Rigorous AI Supply Chain Vetting: The reliance on third-party data sets, pre-trained models, and external plugins necessitates a rigorous vetting process. Defenders must assess the security hygiene of all AI suppliers, scrutinize the integrity of external components, and implement mechanisms to detect tampering or poisoning throughout the AI supply chain. This requires a shift from simply trusting external components to continuously verifying their integrity.

1. Fostering Cross-Functional Collaboration (Secure X-Ops): The concept of Secure X-Ops is not merely a theoretical framework but a practical necessity. Security teams must break down silos and collaborate closely with development (DevOps), data science (DataOps), and machine learning operations (ML/LLM Ops) teams. This converged operational model ensures that security is integrated from the design phase through deployment and incident response, creating a holistic and agile defense posture.

1. Development of Machine-Readable Attestations: To meet future compliance demands (like the Cyber Resiliency Act) and enable efficient incident response, organizations must invest in tools and processes for generating automated, integrated, and verifiable attestations for all AI components. These attestations should cover hardware, software, data sets, models, and runtime environments, providing cryptographically signed evidence of integrity and compliance.

1. AI-Specific Threat Modeling: Incorporating AI-specific threats, as highlighted by frameworks like the OWASP Top 10 for ML/LLM, into threat modeling exercises is crucial. This proactive approach during the design and development phases allows organizations to identify and mitigate AI-related risks before they become exploitable vulnerabilities in deployed systems.

By embracing these defensive implications, organizations can move beyond reactive security measures to build truly resilient and trustworthy AI systems, safeguarding against the unique and evolving threats of the AI era.

Key Takeaways

• AI introduces novel vulnerabilities: The rapid integration of AI and ML into software creates new attack surfaces, demanding a fundamental shift in traditional cybersecurity approaches.
• Extend existing security practices: Current vulnerability management, supply chain security, and compliance frameworks must be proactively expanded to comprehensively cover AI-specific components, including data sets, models, and associated metadata.
• Data poisoning and AI supply chain attacks are critical threats: Malicious manipulation of training data and compromises within the AI component supply chain (third-party models, data, plugins) pose significant risks to the integrity and reliability of AI systems.
• Embrace "Secure X-Ops": A converged operational model, integrating DevOps, DevSecOps, DataOps, and ML/LLM Ops, is essential for holistic security, enabling seamless collaboration and incident response across the AI lifecycle.
• Develop a Universal Attestation Framework: A future-proof security strategy requires an automated, integrated, and easy-to-adopt framework to establish, report, and verify the integrity of all AI components and processes, crucial for compliance and rapid remediation.
• Foster cross-functional collaboration: Addressing the multifaceted challenges of AI security necessitates breaking down silos and encouraging communication and expertise sharing across different enterprise and industry stakeholders.

About the Speaker(s)

Lisa Bradley is a Senior Director for Dell Technologies, leading the Product and Application Security space. Her team, affectionately known as "Prada," focuses on product remediation and dependency assurance, ensuring robust product security and proactive management of vulnerabilities within software dependencies. Additionally, the PERT (Product Emergency Response Team), which drives remediation of control gaps beyond just vulnerabilities, also falls under her purview. Lisa brings over 12 years of experience in the security domain, having earned her PhD from NC State University and her undergrad from Geneseo. Her career spans significant roles at IBM and Nvidia before joining Dell over five years ago. She has also contributed to academia, teaching on the side for 12 years.

Sarah Evans works in the Office of the CTO at Dell Technologies, where she specializes in security innovation research. Her role involves leveraging her extensive background in IT operations and security operations to influence the early design stages of Dell products and services. She focuses on anticipating how emerging technologies, particularly in AI and ML, might impact customer security and supply chains. Sarah holds an MBA from Missouri State University and has taught in their computer information systems program. A veteran of the Air Force, she retired after 21 years of service. Over the past year, she further honed her expertise by earning a certificate in AI and ML from a university, bringing specialized knowledge to Dell's security strategy.