Chimera: Creating Digitally Signed Fake Photos by Fooling Image Recapture and Deepfake Detectors

BarraCUDA: Edge GPUs do Leak DNN Weights

Speakers: Peter Horvath, Lukasz Chmielewski, Léo Weissbart, Lejla Batina (Radboud University); Yuval Yarom (Ruhr University Bochum)

Conference: USENIX Security

YouTube: https://www.usenix.org/conference/usenixsecurity25/presentation/horvath

Overview

This article delves into BarraCUDA, a groundbreaking side-channel attack meticulously detailed in a paper presented at USENIX Security. The research, spearheaded by Peter Horvath and a team from Radboud University, Ruhr University Bochum, and Masaryk University, definitively answers a critical question in hardware security: Are proprietary implementations of neural networks on GPUs vulnerable to parameter extraction using side-channel analysis? Their affirmative findings expose a significant vulnerability in popular Nvidia Jetson devices, which are widely deployed in edge computing applications ranging from autonomous vehicles to intelligent cameras.

The core of the BarraCUDA attack lies in exploiting the inherent computational process of deep neural networks (DNNs) on Graphics Processing Units (GPUs). Specifically, it targets the convolution operation, a fundamental building block in many DNNs, by observing that it must be computed as a sequence of partial sums. Each of these partial sums, the researchers discovered, leaks critical information about one or a few of the network's parameters. By employing Correlation Electromagnetic Analysis (CEMA), BarraCUDA can effectively recover the weights and biases of real-world convolutional neural networks, which are often considered the intellectual property (IP) bedrock of modern AI products.

The implications of BarraCUDA are profound. With DNNs becoming central to countless products and services, the confidentiality of their trained parameters is paramount. The ability for an adversary with physical access to an edge device to extract these parameters represents a direct threat to IP, potentially enabling model replication, adversarial attacks, or reverse-engineering of proprietary AI functionalities. This work not only establishes a novel attack vector but also highlights the urgent need for robust countermeasures to secure AI models deployed on GPU-accelerated edge platforms.

Background

The past decade has witnessed an unprecedented surge in the adoption and sophistication of machine learning, particularly deep learning. Deep Neural Networks (DNNs) now underpin state-of-the-art performance across a vast array of applications, including image classification, object detection, natural language processing, and autonomous systems. The development of these DNNs is a resource-intensive endeavor, requiring specialized datasets, extensive computational power for training (often spanning days or weeks on high-performance GPUs), and expert knowledge in architecture design and optimization. Consequently, the trained parameters—the weights and biases—of these models represent significant intellectual property, safeguarding which is a top priority for businesses.

A growing trend involves deploying these sophisticated DNNs onto edge devices, such as intelligent cameras, drones, and embedded systems in autonomous vehicles. These deployments introduce a critical security challenge: a strong threat model where the adversary is assumed to have unfettered physical control over the product. This physical access opens the door to side-channel attacks (SCA), which exploit unintended physical leakages (e.g., power consumption, electromagnetic emanations, timing variations) from electronic devices to extract secret information.

Prior research has demonstrated the feasibility of SCA against neural network implementations on other hardware platforms. For instance, electromagnetic side-channel analysis and microarchitectural attacks have successfully targeted CPUs (Batina et al. [13], Yan et al. [62]). Similarly, commercial deep-learning accelerators on FPGAs have been shown vulnerable to parameter extraction via power analysis (Gongye et al. [25]). However, GPUs have remained a more formidable target for parameter extraction. GPUs are the dominant hardware platform for deep learning due to their massive parallel processing capabilities and the robust software ecosystem, particularly Nvidia's CUDA platform, which facilitates efficient DNN deployment. The complexity and inherent parallelism of GPU architectures, characterized by Single-Instruction-Multiple-Thread (SIMT) execution and non-deterministic thread scheduling, introduce a high amount of noise and make traditional SCA techniques significantly more challenging. Until BarraCUDA, attacks on GPU implementations had only succeeded in recovering the network architecture (e.g., layers, sizes) but not the actual, sensitive parameters (e.g., Guri et al. [16], Luo et al. [29], Wu et al. [41]). This research directly addresses this gap, demonstrating that even with these complexities, GPUs are not immune to parameter extraction via side-channel analysis.

Key Findings

The BarraCUDA research delivers several critical findings that fundamentally alter the understanding of GPU security in the context of deep learning inference:

• Affirmative Vulnerability: The work definitively proves that proprietary implementations of neural networks running on GPUs are indeed vulnerable to parameter extraction using side-channel analysis. This refutes the prior assumption that GPU complexity rendered such attacks infeasible.
• Successful Attack on Nvidia Jetson Devices: The attack, named BarraCUDA, was successfully demonstrated on popular Nvidia Jetson devices, specifically the Jetson Nano (featuring a Maxwell GPU) and the Jetson Orin Nano (featuring an Ampere GPU). These devices are widely used in real-world edge AI applications, highlighting the practical impact of the vulnerability.
• Parameter Extraction from Real-World CNNs: BarraCUDA achieved the recovery of weights and biases from a real-world convolutional neural network architecture, specifically the baseline EfficientNetB0. This is a significant advancement over previous GPU attacks that could only recover network architecture.
• Exploitation of Partial Sums: The core innovation of BarraCUDA is its reliance on the observation that the convolution operation, a fundamental computation in DNNs, is executed as a sequence of partial sums. Each partial sum calculation, depending on a small number of weights and inputs, creates a discernible leakage that can be exploited.
• Correlation Electromagnetic Analysis (CEMA): The attack effectively utilizes CEMA, a statistical side-channel technique, to correlate predicted intermediate values (based on guessed weights) with actual electromagnetic emanations from the GPU. Both Hamming weight (HW) and Hamming distance (HD) leakage models were found to be exploitable.
• Support for Reduced Precision Data Types: BarraCUDA demonstrated successful parameter extraction for both FP16 (half-precision floating-point) and INT8 (8-bit integer) parameters. This is crucial as reduced precision is commonly used in edge devices for optimized inference performance.
• Localized Leakage: The researchers successfully identified specific physical locations on the Jetson Nano and Orin Nano PCBs (e.g., between capacitors, on the SoC surface) that exhibit exploitable electromagnetic leakage (Figure 2).
• High-Performance Analysis Tool: To overcome the challenge of processing millions of traces from noisy GPU environments, the team developed a custom CUDA-based implementation of CEMA. This tool achieved a significant speedup, being 5 to 10 times faster than existing multi-threaded CPU implementations like the JlSCA library.

These findings collectively establish a new frontier in hardware security for AI, underscoring that the sophisticated parallelism of modern GPUs does not inherently protect the intellectual property embedded in deep learning models from determined adversaries with physical access.

Technical Deep Dive

The BarraCUDA attack operates under a specific threat model where the adversary has unfettered physical access to the target edge device. This enables the attacker to open the device, place electromagnetic probes at sensitive locations, and monitor emanations during inference. The attacker is assumed to know, or be able to recover, the DNN model architecture (e.g., number of layers, types, sizes), building on prior work (Guri et al. [16], Luo et al. [29], Wu et al. [41]). Furthermore, for certain data types like INT8, the attacker needs the ability to choose and control the inputs fed to the model, while for FP16, random known inputs suffice.

The attack procedure is divided into two main phases: profiling and parameter extraction (Figure 1). The profiling phase, performed on an identical hardware device (without known weights), is crucial for understanding how information about weights and biases leaks. This knowledge then guides the parameter extraction from the victim device.

The core observation underpinning BarraCUDA is how the fundamental convolution operation is computed. A convolution, represented as csum = x w = sum(wi xi), where w are weights and x are inputs, cannot be computed in a single atomic step. Instead, it is broken down into a sequence of partial sums, sj = sj-1 + wj * xj. Each of these intermediate computations involves a small number of weights and inputs. If the attacker knows the previous partial sum (sj-1) and the current input (xj), they can hypothesize a weight (wj), predict the resulting leakage, and correlate this prediction with measured side-channel traces.

Profiling Phase: Unveiling GPU Internals

The profiling phase is a sophisticated process designed to pinpoint where and when specific weight-dependent computations leak.

1. Software Reverse-Engineering: Since the proprietary CUDA binaries are unavailable, the researchers used cuobjdump from the CUDA Toolkit to disassemble the GPU code. This allowed them to analyze the assembly instructions and understand the high-level structure of how convolutional and dense layers are implemented. They identified three distinct blocks of operations within the CUDA functions:

• Init Block: Initializes accumulator registers (R0-R63) and loads weights and inputs into higher registers (R64+). Registers are 32-bit, holding either two FP16 values or four INT8 values.
• Convolutional Block: This is where the core convolution operations and partial sum computations occur, involving repeated vectorized loads and arithmetic instructions.
• ReLU Block: Adds biases and applies activation functions like ReLU. For FP16, two partial sums are combined. For INT8, results are converted to floating-point before bias addition and ReLU.

By observing electromagnetic emanations (Figure 3), these blocks, and even separate CUDA function calls for different layers or memory copies, could be clearly distinguished.

1. Leakage Localization (Spatial and Temporal):

• EM Probe Positioning: To find optimal leakage points, the researchers used both Test Vector Leakage Assessment (TVLA) and intermediate-value correlation experiments. On the Jetson Nano, promising locations were found between capacitors in the power-supply circuit (Figure 2a) and on the SoC surface. For the Jetson Orin Nano, exploitable signals were found between capacitors (Figure 2b), with higher-frequency probes also picking up signals on the chip surface. The locations between capacitors were generally preferred for ease of placement.
• TVLA (Fixed vs. Random): This statistical method was primarily used to identify the precise time points in the traces where a specific weight's computation leaks. Two sets of traces are collected: a "fixed" set where a target weight is constant, and a "random" set where the target weight varies randomly. All other weights and inputs are kept constant. Welch's t-test is then applied to compare the distributions of values at each time point. An absolute t-value greater than 4.5 (|t| > 4.5) indicates significant leakage (Figure 4). This process is repeated for all weights and biases to create a map of leakage points.
• Trace Acquisition and Preprocessing: A Lecroy 8404M-MS oscilloscope was used at a sampling rate of 10 GS/s, with a Langer MFA-R 0.2-75 near-field probe. The nsys tool from the CUDA Toolkit helped identify operation execution times, allowing the oscilloscope's SmartTrigger to reliably capture relevant inference segments. Collected traces often contained jitter and clock instability. While elastic alignment (Gongye et al. [60]) was used for initial leakage detection, static alignment (Moro et al. [42]) on the identified leaky parts of the raw traces proved more effective for the actual CEMA attack, yielding higher TVLA peaks and correlation.

Parameter Extraction Phase: CEMA Implementation

With the profiling complete, the attack proceeds to extract parameters using Correlation Electromagnetic Analysis (CEMA).

1. Leakage Models: The attack relies on standard side-channel leakage models:

• Hamming Weight (HW): Assumes leakage is proportional to the number of set bits in a value, often seen during data transfer.
• Hamming Distance (HD): Assumes leakage is proportional to the number of bit flips when a register is overwritten, often seen during register updates (HD(sj-1, sj)).

Both models were found to be exploitable.

1. CUDA-based CEMA: Given the massive number of traces (millions) and potential candidates (especially for FP16), a highly optimized CEMA implementation was essential. The researchers developed a custom CEMA tool in CUDA, parallelized across three levels: dataset chunks, candidates, and samples. This CUDA implementation achieved a remarkable 5x to 10x speedup compared to CPU-based multi-threaded libraries like JlSCA on an AMD Ryzen 7950X CPU, making large-scale attacks feasible.

1. Targeting FP16 Convolution:

• The FP16 implementation uses the HFMA2 instruction, which performs two half-precision fused-multiply-adds in parallel within a 32-bit register. This effectively splits the convolution computation across two channels, which are later summed.
• The leakage model targets each 16-bit partial sum written into the accumulator register. The search space for FP16 weights was restricted to [-5, 5], encompassing 35,530 possible candidates, as most CNN parameters fall within this range.

1. Targeting INT8 Convolution and Dense Layers:

• The INT8 implementations utilize the IDP.4A instruction, which performs a 4-way dot product and accumulation, summing results into a 32-bit accumulator.
• A key challenge here is that partial sums often depend on multiple (up to four) 8-bit weights simultaneously, leading to a much larger candidate search space (2^32).
• To overcome this, a chosen-input attack is employed: by setting specific input channels to zero, the attacker can isolate the dependency to a single 8-bit weight, reducing complexity. This is particularly effective for dense layers where input sizes are large.
• For deeper layers where chosen inputs might be harder to craft, two strategies are proposed: collecting significantly more traces with random inputs and filtering for instances where specific input channels are zero, or solving systems of linear equations to generate inputs that result in zero inputs to deeper layer channels. The ReLU activation function and quantization processes naturally enhance sparsity, aiding these approaches.

By meticulously profiling the GPU's low-level operations and leveraging a highly optimized CEMA, BarraCUDA demonstrates a practical and effective method for extracting sensitive DNN parameters from edge GPUs.

Demo / Proof of Concept

While the paper does not contain a dedicated "Demo" section, Section 5, "Parameter Extraction Results," serves as a comprehensive proof of concept, detailing the successful application of the BarraCUDA framework to extract parameters from a real-world neural network on target devices.

The researchers selected the baseline EfficientNetB0 architecture, a widely recognized convolutional neural network, for their experiments. They deployed this model on both the Nvidia Jetson Nano (for FP16 parameter extraction) and the Nvidia Jetson Orin Nano (for INT8 parameter extraction).

FP16 Parameter Extraction on Jetson Nano

• Target: Weights and biases of the first two convolutional layers. The architecture was slightly modified to include biases in the first layer for demonstration purposes.
• Leakage Models: Both the Hamming Weight (HW) and Hamming Distance (HD) leakage models were successfully exploited.
• Weight Extraction: The attack targeted individual 16-bit FP16 weights by focusing on their corresponding partial sums (sj) during convolution. Figures 5a-5h illustrate the key rank and correlation over the number of traces, showing successful recovery of weights in both the first and second layers. For example, the third weight in the first layer (value 0.8223) and the second/third weights in the second layer (values -0.5137 and -0.6406 respectively) were recovered. The HD model, while effective, showed a slightly slower convergence behavior compared to HW.
• Bias Extraction: Biases were also successfully extracted using the HD leakage model by targeting the register update from csum to cout (HD(csum, cout)). Figures 6e-6f demonstrate the bias key rank dropping quickly to 0 within 5 million traces.
• Trace Requirements: While individual weights and biases varied, an upper bound of 20 million traces was generally sufficient for successful extraction of FP16 parameters on the Jetson Nano.
• Scale: From the first two convolutional layers, 9564 FP16 parameters were successfully extracted (Table 2).

INT8 Parameter Extraction on Jetson Orin Nano

• Target: Weights of the first convolutional layer and weights of dense layers.
• Convolutional Layer Weight Extraction:
• For the first layer with a single input channel, partial sums depend on a single 8-bit weight, simplifying the attack. Figures 7a-7d show the successful recovery of the 4th and 5th INT8 weights in a kernel, typically requiring an average of 300,000 traces for the correct key candidate to reach rank 0.
• Both HW and HD models were exploitable.
• Impact of Batch Size: Experiments showed that increasing the batch size from 1 to 16 did not significantly alter the number of traces required (500,000 traces were sufficient). While larger batch sizes increase the number of concurrently executing threads, the physical parallelism on smaller GPUs is limited, suggesting the main impact is a linear increase in execution time rather than a direct increase in attack complexity for vertical attacks.
• Dense Layer Weight Extraction:
• A neural network with a single dense layer (512 nodes, input size 784) was targeted.
• Due to the larger input size, partial sums in dense layers typically depend on multiple INT8 weights (up to four), leading to a 32-bit complexity (2^32 candidates).
• To mitigate this, a chosen-input attack was employed: by setting three out of four input channels to zero, the dependency was reduced to a single 8-bit weight. This allowed for successful extraction, with an average of 300,000 traces needed for the correct key candidate to reach rank 0 (Figures 7e-7h).
• Deeper Layers Challenge: For deeper INT8 layers where partial sums intrinsically depend on four 8-bit weights, the complexity becomes 2^32 candidates. The authors propose two strategies:

1. Random Inputs Filtering: Collecting a significantly larger number of traces and selecting only those where inputs to specific deeper layer channels are zero.
2. Chosen Inputs: Solving systems of linear equations to generate initial inputs that result in zero inputs to specific channels in deeper layers.

• Scale: From the first convolutional layer, 288 INT8 parameters were extracted. Biases in INT8 implementations are FP32, making their extraction computationally prohibitive for the current attack setup (Table 2).

Attack Feasibility and Resources

The overall attack timeline for the Jetson Nano (FP16, batch size 1) involved 10 days for trace collection and 1-2 days for trace alignment. For the Jetson Orin Nano (INT8, batch size 1), this was reduced to 1 day for collection and 1 day for alignment. With a batch size of 16 on Orin Nano, collection and alignment took 5 days. Once traces are aligned, parameters can be extracted at a rate of one weight every 5-6 minutes. The entire process is highly parallelizable, suggesting that attacks on moderate-sized models are well within the capabilities of well-resourced adversaries, even if a full model extraction (millions of parameters) remains computationally expensive.

Defensive Implications

The success of BarraCUDA underscores the critical need for robust countermeasures to protect sensitive DNN parameters deployed on GPU-accelerated edge devices. Traditional side-channel attack (SCA) mitigation strategies, while generally applicable, need to be specifically adapted for the unique challenges of GPU architectures.

1. Physical Security: The most fundamental defense, as highlighted by Nvidia's recommendations, is to prevent physical access to the device. If an adversary cannot place probes, the attack cannot proceed. This includes securing enclosures, tamper-detection mechanisms, and deploying devices in physically controlled environments.
2. Electromagnetic Shielding and Noise Introduction:

• Proper Shielding: Encasing the GPU or the entire device in electromagnetic shielding materials can significantly reduce the strength of emanations, decreasing the Signal-to-Noise Ratio (SNR) for an attacker.
• Noise Introduction: Deliberately introducing random or pseudo-random noise into the electromagnetic emanations or power consumption profiles can obfuscate the leakage signals, making correlation attacks much harder. However, this often comes with engineering challenges and potential power consumption overheads.

1. Algorithmic Countermeasures Specific to DNNs:

• Shuffling: One effective countermeasure, also suggested in the CSI-NN paper (Batina et al. [13]), is to shuffle the order of multiplications within the neural network layers. If the attacker cannot predict the sequence of operations that lead to partial sums, it becomes significantly harder to construct accurate leakage models and correlate them with measured traces. This disrupts the deterministic execution pattern BarraCUDA relies on.
• Masking: Masking schemes aim to decouple the processed data from the side-channel measurements by splitting sensitive intermediate values into multiple random shares. Each share is processed independently, such that no single share's leakage reveals information about the original secret. While highly effective in cryptographic contexts, applying masking to complex DNN operations on highly parallel GPUs is a significant research challenge and typically incurs a substantial performance overhead, which might be undesirable for real-time edge applications.
• Accumulator Initialization: A specific countermeasure proposed by the authors for convolution is to initialize the accumulator registers with the bias of the kernel instead of zero. This would force the adversary to guess a b+w1 pair simultaneously, increasing the complexity. For FP16, this would mean guessing 32 bits (16 for bias, 16 for weight). However, this countermeasure is not directly applicable to INT8 implementations where the bias is often a 32-bit float, making such a combined guess computationally prohibitive for the attacker.

1. Hardware-Level Security Features: Future GPU architectures could integrate hardware-level countermeasures. This might include dedicated secure enclaves for sensitive computations, hardware-enforced randomization of instruction scheduling, or built-in noise injection capabilities.
2. Monitoring and Detection: Implementing robust monitoring solutions on edge devices could potentially detect anomalous electromagnetic emanations or power consumption patterns indicative of an ongoing side-channel attack.

The research highlights that protecting DNN implementations in security or privacy-sensitive applications remains an open and complex problem. While the attack currently requires significant resources, the increasing value of AI models and the accessibility of powerful side-channel equipment mean that these vulnerabilities cannot be ignored. A multi-layered defense strategy combining physical security with algorithmic and hardware-level countermeasures is essential to safeguard intellectual property on edge GPUs.

Key Takeaways

• Edge GPUs are Vulnerable: Nvidia Jetson Nano and Orin Nano devices, common platforms for edge AI, are vulnerable to electromagnetic side-channel attacks for extracting Deep Neural Network (DNN) weights and biases.
• BarraCUDA Leverages Partial Sums: The attack exploits the fundamental computation of DNNs, specifically the sequential calculation of partial sums within convolutional and dense layers, which leak information about individual weights and biases.
• Profiling and CEMA are Crucial: A meticulous profiling phase involving reverse-engineering CUDA binaries and utilizing Test Vector Leakage Assessment (TVLA) is essential to identify and localize specific leakage points, followed by Correlation Electromagnetic Analysis (CEMA) for parameter extraction.
• Real-World Effectiveness: BarraCUDA successfully extracted FP16 and INT8 parameters from a real-world CNN (EfficientNetB0), demonstrating practical feasibility, albeit requiring millions of traces (up to 20 million for FP16, 3 million for INT8) and significant computational resources.
• Performance Optimization: A custom CUDA-based CEMA implementation was developed, achieving a 5-10x speedup over CPU-based alternatives, which is critical for processing the large datasets required for GPU side-channel attacks.
• Countermeasures are Needed: While traditional physical security, shielding, and noise injection can help, specific algorithmic countermeasures like shuffling computation order and masking are vital, though they often come with performance overheads. Nvidia recommends preventing physical access.

About the Speaker(s)

The research behind BarraCUDA was a collaborative effort by a team of distinguished academics in the field of hardware security and cryptography.

Peter Horvath, Lukasz Chmielewski, Léo Weissbart, and Lejla Batina are affiliated with Radboud University, a prominent institution known for its strong research in cybersecurity and digital security. Their work often focuses on side-channel analysis, cryptographic implementations, and the security of embedded systems.

Yuval Yarom is affiliated with Ruhr University Bochum, another leading institution with significant contributions to hardware security and side-channel research. Yuval Yarom is particularly recognized for his extensive work on microarchitectural side-channel attacks and cache-based timing attacks, bringing deep expertise in exploiting subtle hardware leakages.

Collectively, the authors' expertise spans various facets of hardware security, side-channel analysis, and the security implications of advanced computing paradigms like deep learning on specialized hardware. Their combined knowledge was instrumental in tackling the complex challenge of side-channel attacks on modern GPU architectures.