SoK: Efficiency Robustness of Dynamic Deep Learning Systems

SoK: Efficiency Robustness of Dynamic Deep Learning Systems

Speakers: Ravishka Rathnasuriya

Conference: USENIX Security

YouTube: https://www.youtube.com/watch?v=2Dk2hc6K3n0

Overview

This article delves into the critical and emerging field of efficiency robustness in dynamic deep learning (DDL) systems, based on the USENIX Security talk by Ravishka Rathnasuriya from the University of Texas at Dallas. The presentation introduces a novel perspective on adversarial machine learning, shifting focus from traditional accuracy-based attacks to computational cost inflation. As deep learning models become ubiquitous, particularly in real-time, resource-constrained, and edge environments, their efficiency is paramount. This talk highlights a fundamental vulnerability: the very adaptivity that makes DDL systems efficient can be exploited by adversaries to force models into computationally expensive execution paths, leading to significant performance degradation, resource exhaustion, and potential denial-of-service (DoS) attacks.

The core contribution of this research is a comprehensive taxonomy of efficiency attacks, categorized by three dimensions of dynamic behavior in DDL systems. By systematically analyzing how adversaries can manipulate inputs to trigger higher computational loads, the work uncovers widespread vulnerabilities across various DDL architectures, from multi-exit DNNs and auto-regressive language models to spiking neural networks and object detection systems. This research is crucial for the secure and sustainable deployment of AI, urging the security community to prioritize efficiency robustness alongside traditional accuracy and privacy concerns.

Background

▶ Watch: Introduction: Efficiency and dynamic deep learning vulnerabilities (0:00)

The increasing sophistication and widespread deployment of deep learning systems have made efficiency a critical concern. Modern applications, ranging from autonomous vehicles and real-time speech processing to IoT devices and large language models, demand not only high accuracy but also rapid inference and minimal resource consumption. This demand has spurred the development of dynamic deep learning (DDL) systems, which are designed to adapt their computations to the complexity of the input. Unlike static models that process every input with a fixed computational budget, DDL systems can adjust their internal execution paths, number of inference iterations, or output generation based on internal decision-making or input characteristics. This adaptive behavior, while promising significant efficiency gains, introduces a new attack surface.

The problem arises because DDL systems rely on internal logic (e.g., confidence scores, gating mechanisms, non-maximum suppression thresholds) to decide how much computation to expend. An adversary can craft adversarial inputs that, while imperceptible to humans and semantically valid, intentionally trigger these internal mechanisms to demand maximum computational effort. This differs fundamentally from traditional adversarial attacks, which aim to degrade a model's accuracy or elicit incorrect classifications. Efficiency attacks, as formalized in this research, seek to maximize the computational cost (e.g., energy consumption, latency, floating-point operations) of a DDL system, given a perturbed input $\delta$ such that the total input $X + \delta$ remains within the valid input space and $\delta$ is imperceptible.

The talk categorizes the adaptive behaviors of DDL systems into three dimensions:

• D1: Computations per Inference: The system adjusts internal execution paths within a single forward pass. Examples include skipping certain layers, neurons, or solver steps based on confidence scores or gating logic.
• D2: Number of Inference Iterations per Input: Models generate or refine outputs across multiple iterations, where the number of iterations is determined at runtime. This is common in auto-regressive models (e.g., language models generating tokens until a stop token appears).
• D3: Number of Outputs Sent to Downstream Modules: The quantity of outputs a model produces at inference can vary, typical in applications like object detection and multi-object tracking, where non-maximum suppression (NMS) thresholds govern the final output count.

The adversary's goals are clear: maximize computation cost, maintain imperceptible perturbations, and ensure adversarial inputs remain realistic and valid. Attackers can leverage both white-box knowledge (full model access, gradients) for evasion attacks and black-box knowledge (no model access, input-output patterns) or even poisoning attacks (injecting malicious data during training to influence runtime behavior).

Key Findings

▶ Watch: Three dimensions of dynamic behaviors (D1, D2, D3) (2:20)

The central finding of this research is the identification and systematic categorization of a pervasive class of efficiency vulnerabilities across a wide array of dynamic deep learning systems. The talk presents a comprehensive taxonomy that maps dynamic behaviors to specific mechanisms, vulnerable system models, and corresponding attack strategies. This framework demonstrates that the very mechanisms designed for efficiency in DDL systems—such as early exits, adaptive step-size solvers, sparse computations, iterative generation, and output filtering—can be systematically exploited by adversaries.

Key findings include:

1. Ubiquity of Vulnerability: Efficiency attacks are not confined to a niche set of models but affect diverse DDL architectures, including multi-exit DNNs, ODENets, Spiking Neural Networks (SNNs), dynamically sparse models, auto-regressive language models, and object detection pipelines.
2. Significant Real-World Impact: The attacks can lead to severe performance degradation, resource exhaustion, and even denial-of-service conditions. Specific examples cited include a drop in successful inferences from 10,000 to 3,500 on IoT devices, delays up to five times, a 50% decrease in inference efficiency for ODENets, 30% battery drain on mobile devices, and a 251% increase in inference time for video processing, causing frame rates to drop from 40 FPS to 16 FPS.
3. Novel Attack Strategies: The research outlines various white-box and black-box attack strategies tailored to each dynamic behavior, demonstrating how adversaries can manipulate internal states (e.g., confidence scores, step sizes, spiking activity, sparsity patterns, EOS token probabilities, bounding box counts) to inflate computational cost.
4. Limitations of Existing Defenses: Current adversarial defenses, such as SVM-based input validation and mitigation techniques like JPEG compression or spatial smoothing, are largely ineffective against efficiency attacks. While some mitigation offers partial recovery of efficiency, it often comes at a significant cost to model fidelity, and detection mechanisms can fail when combined with these transformations.
5. New Research Directions: The work highlights critical open problems, including the need for joint optimization in attack strategies, extending attacks to new architectures (e.g., Mixture of Experts), developing hardware-agnostic black-box attacks, balancing step size and energy efficiency in ODENets, and creating robust defenses specifically for efficiency robustness.

In essence, the research reveals that the pursuit of efficiency in deep learning has inadvertently introduced a new vector for adversarial exploitation, demanding a fundamental re-evaluation of security paradigms for AI systems.

Technical Deep Dive

▶ Watch: D1 attacks: Suppressing skipping conditions (white-box) (5:00)

The core of this research lies in its detailed taxonomy, which systematically breaks down efficiency attacks based on the three dimensions of dynamic behavior in DDL systems.

D1: Computations per Inference

This dimension focuses on models that adjust their internal execution path during a single forward pass.

1. Skipping Mechanisms:

• System Models: Multi-exit DNNs, multi-exit Language Models, and multi-exit Vision Transformers. These models save computation on "easy" inputs by allowing early termination or skipping blocks.
• Attack Strategy: Adversaries introduce perturbations that suppress the skipping conditions, forcing the model to process all residual blocks or gates, even for simple inputs.
• White-box Attacks:
• Gradient Manipulation Attacks: Use internal gradients to bypass early exits.
• Multi-objective Optimization: Craft perturbations that balance imperceptibility and cost, avoiding conflicting gradients.
• Gradient Approximation Based Attacks: When gradients are hard to obtain, approximate them via entropy maximization to lower intermediate classifier confidence.
• Generative Adversarial Networks (GANs): Generators craft natural-looking inputs that trigger heavy computations.
• Gradient Manipulation via Importance Token: Targets and perturbs or replaces key tokens to suppress early exits in language models.
• Black-box Attacks: Use learned estimators or genetic search to evolve costly inputs.
• Poisoning Attacks: Poison training data to bias the model towards inefficient execution paths at inference time.
• Real-world Impact: On devices like the Galaxy S9 Plus, successful inferences can drop from 10,000 to just over 3,500. IoT deployments can experience delays up to five times.

1. Step-size Based Attacks:

• System Models: ODENets (Ordinary Differential Equation Networks). These treat forward passes as solving differential equations, adaptively picking step sizes. Smaller steps improve accuracy but cost more compute.
• Attack Strategy: Perturb inputs to force the numerical solver to take smaller step sizes more often, propagating through the solver to inflate the number of integration steps.
• Translatability: Attacks can be tried across different solvers and architectures.
• Real-world Impact: Can cause inference efficiency drops of up to 50% in DNN-based compiler vulnerabilities.

1. Spiking Attacks:

• System Models: Spiking Neural Networks (SNNs), designed to mimic biological neurons, processing information via discrete spikes. Their event-driven design is attractive for low-power applications.
• Attack Strategy: Use selected gradients to bypass the non-differentiability of spikes, optimizing inputs to maximize spiking activity while maintaining correct outputs.
• Variations: Full backpropagation through time or reduced time steps via proxy models.
• Current Limitations: No black-box or poisoning attacks evaluated yet.
• Real-world Impact: Can lead to power budget collapse in neuromorphic or low-power systems (e.g., cameras, wearables) and memory overload, forcing edge devices to offload to cloud inference.

1. Sparsity-based Attacks:

• System Models: Accelerators (e.g., ASICs, quantization transformers) that rely on dynamic sparsity optimizations by skipping zero or near-zero activations to save time and energy.
• Attack Strategy: Adversaries exploit this by intentionally breaking sparse patterns, forcing hardware to run denser or more expensive computations.
• White-box Attacks:
• Focus on layer-wise sparsity: Manipulate gradients to shift activations away from zero, making sparse layers dense.
• Gradient approximations: Inject synthetic outliers to disrupt quantization, causing models to switch from low-precision to high-precision computation.
• Black-box Attacks: Analyze input-output patterns and use surrogate gradient estimation to reduce sparsity.
• Poisoning Attacks: Modify sparsity-inducing parameters to increase non-activation with minimal data poisoning.
• Real-world Impact: Affects safety-critical IoT and Machine Learning as a Service (MLaaS) systems. Batch sensitivity is critical; smaller batch sizes can increase memory usage by up to 12%, while larger batches see only a 1.8% increase.

D2: Number of Inference Iterations per Input

This dimension targets models that dynamically determine the number of steps to run before halting.

1. Dynamic Inference Iterations:

• System Models: Primarily auto-regressive models like decoders in Neural Image Caption Generation (NICG), Neural Machine Translation (NMT) systems, speech systems, and Vision-Language Models (VLMs). These models typically stop when an End-Of-Sequence (EOS) token is predicted.
• Attack Strategy: Adversarial perturbations subvert the likelihood of the EOS token, causing the generation process to continue indefinitely or for an excessive number of steps.
• White-box Attack Strategies:
• Gradient Guided Sequence Control: Propagate through the decoder to push down the EOS probability.
• Token Replacement and Removal: Use gradients to replace or remove tokens signaling stopping, eliminating shortcuts for early generation.
• Black-box Attacks:
• Genetic or Evolutionary Algorithm Strategies: Evolve candidate inputs via mutation and crossover with a fitness function (instead of gradients).
• Surrogate Models: Train or reuse a surrogate model to discover and transfer examples.
• Real-world Impact: On mobile devices, NMT attacks can drain ~30% battery in 300 runs (compared to 1% for benign inputs). On cloud devices, such attacks can cause translations to be up to 6,000 times slower.

D3: Number of Outputs Sent to Downstream Modules

This dimension concerns models where the number of outputs varies at inference time, often filtered by post-processing steps.

1. Output Generation Attacks:

• System Models: Primarily object detection or tracking pipelines. Detectors emit a set of bounding boxes, filtered by Non-Maximum Suppression (NMS), before being passed to trackers.
• Attack Strategy: Adversarial inputs cause the detector to generate a large number of redundant, phantom, or overlapping bounding boxes, vastly increasing the NMS and association costs for downstream modules.
• White-box Attack Strategies:
• Point Cloud Manipulation: Perturbations in LiDAR-based systems to create phantom objects.
• Bounding Box Manipulation: Force detectors to emit redundant and overlapping boxes, multiplying NMS and association costs.
• Spatial Attention: Adversarial noise redirects attention to irrelevant regions, generating spurious detections.
• Feature Manipulation: Perturbations densify activations, disrupting sparsity or quantization, forcing costly full-precision computation.
• Translatability: Ensemble-based or universal perturbations can increase cross-model risk.
• Poisoning/Backdoor Attacks: Implant triggers during training so specific inputs later cause "bounding box explosions."
• Real-world Impact: Attacks on video processing can increase inference time by 251%, dropping frame rates from 40 FPS to 16 FPS.

Demo / Proof of Concept

▶ Watch: Real-world impact of skipping attacks (IoT, latency) (6:30)

While the talk did not feature a live, interactive demonstration, the research effectively demonstrates the feasibility and severe impact of these efficiency attacks through extensive experimental evaluation and presentation of real-world impact figures. The "real-world impact" sections for each attack category serve as compelling proof-of-concept, quantifying the degradation in performance and resource consumption.

For instance, the impact on skipping mechanisms was demonstrated by showing a drop in successful inferences on a Galaxy S9 Plus from 10,000 to 3,500, and a five-fold increase in delays for IoT deployments. ODENet attacks were shown to reduce inference efficiency by 50%. For auto-regressive models, a neural machine translation attack was shown to drain 30% of battery in 300 runs on mobile devices, and cause 6,000 times slower translation on cloud devices. Object detection attacks demonstrated a 251% increase in inference time for video processing, reducing frame rates from 40 FPS to 16 FPS. These specific, quantifiable results highlight the practical efficacy of the proposed attack vectors.

Furthermore, the talk included an experimental evaluation of existing defenses against a subset of these efficiency attacks (NICG slowdown, deep slow, and slow attacks). This involved testing detection-based defenses (e.g., an SVM classifier trained on intermediate states or hidden features to flag suspicious inputs) and mitigation-based defenses (e.g., applying transformations like JPEG compression and spatial smoothing). The results from these experiments further solidified the proof-of-concept by demonstrating that even with defense mechanisms in place, the attacks could still significantly degrade performance, or the defenses themselves introduced unacceptable trade-offs. For example, while JPEG compression partially restored efficiency, it significantly compromised the model's fidelity (measured by BLEU score for NICG slowdown attacks), and spatial smoothing was largely ineffective. Detection mechanisms, while working reasonably well on clean inputs, failed once transformations were applied, becoming overconfident but losing separability between benign and adversarial inputs.

This rigorous experimental validation, rather than a live demo, serves as the primary proof of concept for the identified efficiency vulnerabilities and the limitations of current defensive strategies.

Defensive Implications

▶ Watch: Implications and future research opportunities (9:00)

The research rigorously evaluated the efficacy of existing adversarial defenses against the newly categorized efficiency attacks, revealing significant limitations and underscoring the urgent need for novel, dedicated solutions. Two primary types of defenses were investigated: detection-based and mitigation-based.

Detection-based defenses typically rely on input validation, often employing a classifier (e.g., a Support Vector Machine (SVM)) to flag suspicious inputs before they trigger expensive computations. This SVM classifier was trained on intermediate states or hidden features to distinguish between benign and adversarial samples. The experimental results indicated that while these detectors could perform "reasonably well" on clean and adversarial inputs in isolation, their effectiveness dramatically diminished when combined with mitigation techniques like JPEG compression or spatial smoothing. Specifically, "detection essentially fail[ed]," with the model becoming "overconfident" but losing its ability to separate benign from adversarial inputs. This implies that current detection strategies are brittle and easily circumvented when attackers introduce subtle modifications or when standard image processing is applied, which can be a common scenario in real-world pipelines.

Mitigation-based defenses aim to weaken the effect of adversarial perturbations rather than blocking them outright. The study tested common transformations such as JPEG compression and spatial smoothing, which are known to reduce high-frequency noise that many adversarial attacks rely on. The findings for multi-exit networks and NICG slowdown attacks were particularly insightful:

• NICG Slowdown Attacks: Without defenses, the output length exploded from 10 tokens to 62 tokens. JPEG compression reduced this to 11 tokens, partially recovering efficiency, but at a severe cost to fidelity, resulting in a "very low" BLEU score. Spatial smoothing was "largely ineffective."
• Multi-exit Networks: JPEG and smoothing provided "partial recovery" of efficiency. However, the models "remain highly vulnerable under L-infinity attacks." The "efficiency benefit of these models cannot be reliably preserved under adversarial conditions."

These results lead to several critical defensive implications:

1. Existing defenses are insufficient: Current detection and mitigation techniques, primarily designed for accuracy-based attacks, are not robust against efficiency attacks. They either fail outright, offer only partial recovery, or introduce unacceptable trade-offs between efficiency and model fidelity.
2. Need for dedicated efficiency robustness: The security community must develop new defense mechanisms specifically tailored to counter computational cost inflation. These defenses need to be robust to various attack strategies (white-box, black-box, poisoning) and resilient to combinations of attacks and common data transformations.
3. Holistic approach: Future defenses should consider a holistic approach that integrates robust input validation, adaptive resource management, and potentially model-level modifications that are inherently resistant to efficiency exploitation. This might involve re-thinking the design principles of DDL systems themselves to build in robustness from the ground up.
4. Balance between accuracy and efficiency: Any new defense mechanism must carefully balance the preservation of efficiency with maintaining the model's intended accuracy and utility. Sacrificing fidelity to regain efficiency, as seen with JPEG compression, renders the defense impractical.
5. Addressing transferability: As black-box and transferability attacks become more prevalent, defenses must be effective against attacks generated on surrogate models or universal perturbations.

In summary, the research highlights a significant gap in current AI security practices, emphasizing that merely protecting accuracy is no longer enough. The efficient and sustainable deployment of DDL systems necessitates a dedicated focus on their efficiency robustness.

Key Takeaways

• Efficiency is a New Attack Vector: Beyond accuracy and privacy, the computational efficiency of dynamic deep learning (DDL) systems is a critical and exploitable vulnerability, leading to resource exhaustion and Denial-of-Service (DoS) attacks.
• Ubiquitous Vulnerabilities in DDL Systems: The adaptive mechanisms in DDL models (e.g., early exits, adaptive solvers, dynamic sparsity, iterative generation, output filtering) are widely susceptible to adversarial manipulation, impacting diverse architectures from multi-exit DNNs to auto-regressive LMs and object detectors.
• Significant Real-World Impact: Efficiency attacks can drastically degrade performance, causing multi-fold increases in latency (e.g., 6,000x slower translation), severe resource drain (e.g., 30% battery in 300 runs), and operational failures (e.g., frame rate drops from 40 FPS to 16 FPS).
• Existing Defenses are Ineffective: Current adversarial defenses, including SVM-based detection and mitigation techniques like JPEG compression or spatial smoothing, are largely insufficient. They either fail under realistic conditions or introduce unacceptable trade-offs between efficiency recovery and model fidelity.
• Urgent Need for Novel Defenses: There is a critical requirement for new, dedicated defense mechanisms that are specifically designed for efficiency robustness, capable of withstanding both white-box and black-box attacks without compromising model utility.
• Open Research Frontiers: Future work should focus on developing joint optimization strategies for attacks, extending attacks to emerging architectures (e.g., Mixture of Experts), creating hardware-agnostic black-box attacks, and ensuring a robust balance between accuracy and efficiency in DDL systems.

About the Speaker(s)

Ravishka Rathnasuriya is a researcher from the University of Texas at Dallas. The presented work, "SoK: Efficiency Robustness of Dynamic Deep Learning Systems," is a collaborative effort with Tingili Sinu, Sohi Song, Masal Haki, Sim Chin, and Drang, all affiliated with the University of Texas at Dallas. Their research focuses on understanding and addressing the security implications of dynamic behaviors in deep learning systems, particularly concerning their computational efficiency.