No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML

No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML

Speakers: Ziqi Zhang, Postdoctoral Researcher, University of Illinois Urbana-Champaign; Chen Gong; Yifeng Cai; Yuanyuan Yuan; Bingyan Liu; Shuai Wang

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=x5aXAonEJWY

Overview

In an era where machine learning (ML) models are both immensely valuable and deeply integrated into private applications, their secure deployment on edge devices presents significant challenges. This talk, delivered by Ziqi Zhang, a postdoctoral researcher at the University of Illinois Urbana-Champaign (UIUC) and based on his Ph.D. work at Peking University, delves into the critical security vulnerabilities of existing TEE-Shielded DNN Partition (TSDP) solutions for on-device ML. The core issue arises from the inherent conflict between the computational demands of large models and the limited resources of Trusted Execution Environments (TEEs), leading to a partitioning strategy that inadvertently compromises privacy.

The research critically re-evaluates the efficacy of 10 representative TSDP schemes against a strong adversary model, demonstrating that these solutions, despite prior empirical claims, offer minimal protection against sophisticated model stealing and membership inference attacks. The fundamental flaw is identified in the common "training before partition" paradigm, where privacy-sensitive information becomes distributed across the entire model. To address this, the speakers propose TSlice, a novel "partition before training" strategy designed to isolate and protect only the truly privacy-critical components, achieving black-box level security with significantly reduced overhead.

This work matters because it challenges the foundational assumptions underpinning a widely adopted approach to securing on-device ML. As ML models become more pervasive and handle increasingly sensitive data, understanding and mitigating these architectural vulnerabilities is paramount. The findings provide a stark warning to developers and organizations relying on current TSDP schemes and offer a robust, empirically validated alternative that redefines how privacy and utility can be balanced in the deployment of confidential ML.

Background

▶ Watch: On-device ML security challenges and TEEs (0:00)

The rapid advancement and widespread adoption of large machine learning models (LLMs) have introduced a duality: they are expensive assets to train, often involving millions of dollars in resources, and they are frequently applied to highly sensitive tasks, such as medical diagnostics, financial analysis, and personal data processing. Traditionally, deploying these models to edge devices (e.g., smartphones, IoT devices) exposes them to significant security risks. A malicious device owner can gain white-box access to the model, meaning they can inspect its internal parameters (weights, biases) and architecture. This level of access enables powerful attacks, notably membership inference attacks (determining if a specific data point was part of the training set, revealing sensitive training data) and model stealing attacks (extracting the model's intellectual property and functionality).

To counter these threats, Trusted Execution Environments (TEEs) have emerged as a promising defense. A TEE is a secure area on a main processor designed to provide strong guarantees of data confidentiality and code integrity, even against a compromised operating system (OS). By deploying ML models within a TEE, the device owner can no longer access the internal information of the model, theoretically downgrading potent white-box attacks to less accurate and slower black-box attacks. However, TEEs come with significant limitations: they have restricted memory, lack native GPU support, and generally offer lower computational throughput compared to unshielded CPU/GPU execution. Directly deploying entire large models into a TEE would introduce prohibitive overhead, making them impractical for real-world on-device ML.

This challenge led researchers to propose TEE-Shielded DNN Partition (TSDP) solutions. The core idea behind TSDP is to partition the ML model into two parts: a "large but unimportant and privacy-irrelevant" part, which is offloaded to a GPU for fast computation, and a "small but critical and privacy-related" part, which is shielded by the TEE for high security. The insight was that if a "perfect" partition could be designed, keeping the TEE-shielded part minimal yet critical, then the offloaded GPU part would leak "almost no privacy," allowing for both low latency and high security.

Over the past five years, numerous TSDP solutions have been published in top-tier conferences. The speakers surveyed 15 such papers, categorizing 10 relevant ones into five classes based on their partitioning strategy:

1. Shading deep layers: Shielding the final layers of the neural network.
2. Shading shallow layers: Shielding the initial layers of the neural network.
3. Shading large magnitude weights: Protecting weights deemed more "important" due to their higher absolute values.
4. Shading middle layers: Shielding intermediate layers within the network.
5. Shading nonlinear layers: Protecting layers with non-linear activation functions, often considered critical for model expression.

All these prior solutions relied on empirical results, typically using surrogate model-based attacks to evaluate security. They generally concluded that attackers could not directly use the offloaded (GPU) part of the model to perform effective attacks. However, the talk posits a critical question: "does this conclusion hold in the era of LLMs?" The underlying concern is that these empirical observations might be based on assumptions about model architecture or adversary capabilities that are no longer valid, especially given the evolving nature of deep learning models and the increasing sophistication of attacks. The speaker highlights that "large magnitude weights are important, but we cannot guarantee the small and medium magnitude weights contain no privacy," suggesting a potential blind spot in existing defenses.

Key Findings

▶ Watch: Questioning prior TSDP empirical security in large models (3:17)

The central discovery of this research is that existing TEE-Shielded DNN Partition (TSDP) solutions, despite prior empirical evidence, are fundamentally insecure against strong adversaries in the context of modern large machine learning models. The researchers found that these solutions fail to provide adequate protection against both model stealing and membership inference attacks, essentially offering security levels comparable to deployments without any TEE protection for the offloaded parts.

Specifically, the evaluation revealed:

1. Ineffectiveness of Prior TSDP Schemes: When tested against a strong adversary with access to public models and public data, the representative TSDP solutions across all five categories (shading deep layers, shallow layers, large magnitude weights, middle layers, and nonlinear layers) exhibited significant privacy leakage. For instance, in model stealing, a "no-shade" (fully offloaded) baseline showed an attack performance 4.26 times higher than a full "black-box" (fully TEE-protected) baseline. Critically, schemes like DarkNet (shading deep layers) and the Magnitude-based approach (shading large magnitude weights) still showed attack performances approximately 3.9 times higher than the black-box baseline. This indicates that the majority of the model part offloaded to the GPU still exposes a substantial amount of privacy, blurring the line between TSDP-protected and fully white-box exposed models. The defense effectiveness of these TSDP solutions was found to be "similar to the white box defense."
2. Failure of Configuration Adjustments: The research explored whether simply adjusting deployment configurations (e.g., shading more layers or weights) could improve security. While shading more components within the TEE generally increased security, it invariably led to a proportional decrease in utility (higher computation cost). The "security-utility tradeoff" was inherent in all settings, and no "honest spot" or optimal configuration could be found that achieved black-box security without incurring prohibitive utility costs. The optimal configurations for different models and datasets were highly varied, making a generalized, effective tuning strategy impractical.
3. Fundamental Limitation: "Training Before Partition": The core reason for the observed vulnerabilities was identified as the common "training before partition" strategy employed by all existing TSDP solutions. In this paradigm, the entire model is trained using private data before partitioning. Consequently, all model weights, regardless of their magnitude or layer depth, are updated by and potentially contain privacy-sensitive information. This makes it inherently difficult, if not impossible, to perfectly separate truly privacy-related weights from privacy-irrelevant parts, leading to leakage from the offloaded components.
4. Proposed Solution: "Partition Before Training": Based on this fundamental insight, the researchers propose a novel "partition before training" strategy. This paradigm first separates the model into privacy-related and unimportant parts before training. Then, private training data is used only to update the "privacy-related model slices." This guarantees that any model part offloaded to the GPU is never updated by private data and thus contains no privacy-sensitive information, while the critical, privacy-related slices are securely contained within the TEE. This shift in paradigm forms the basis of their proposed secure TSDP solution, TSlice.
5. TSlice Achieves Black-Box Security with Low Overhead: Experimental results demonstrate that TSlice can provide "black-box level protection" with the "lowest utility cost." It significantly reduces computation cost while maintaining an accuracy drop of only 0.34%. Furthermore, it does not increase attack performance in other aspects. On real devices using the toym framework, TSlice dramatically improves throughput compared to traditional black-box (full TEE) protection.

These findings collectively highlight a critical gap in the current understanding and implementation of TEE-based security for on-device ML and offer a validated, paradigm-shifting approach to address it.

Technical Deep Dive

▶ Watch: Detailed methodology for evaluating TSDP security (4:00)

The research meticulously evaluated existing TSDP solutions against a strong adversary model. This adversary is assumed to have white-box access to the offloaded model part (on the GPU), as well as access to a public model and public data. This reflects a realistic scenario where an attacker might have general knowledge of a model's architecture and access to similar, non-sensitive datasets, which they can leverage to infer information about the target private model.

The evaluation pipeline involved two primary attack types:

1. Model Stealing: The objective is to copy the functionality of the victim model. A higher accuracy of the surrogate model on the target task indicates a more successful model stealing attack.
2. Membership Inference: The goal is to determine if a specific data point was part of the victim model's training dataset, thereby leaking sensitive information about the training data. The accuracy of a confidence-based attack was used as the metric.

The evaluation process for an attack pipeline against a TSDP solution proceeds as follows:

• The attacker first uses the offloaded model part (from the victim's device) and a public model to initialize a surrogate model for the TEE-shielded part of the victim model.
• For the TEE-shielded part, the attacker utilizes the corresponding part of the public model to "neutralize" it, essentially using public information to approximate the protected component.
• This neutralized model is then used to train the surrogate model for model stealing.
• Finally, the trained surrogate model is employed to perform membership inference attacks.

The study included multiple datasets, model architectures, and metrics, selecting representative solutions from each of the five TSDP categories for in-depth analysis:

• DarkNet: Chosen for schemes that shade deep layers.
• Daab: Representative of solutions shading shallow layers.
• Magnitude paper: For approaches focusing on shading large magnitude weights.
• Statter: Selected for methods shading intermediate layers.
• ShadowNet: Representing solutions that shade nonlinear layers.

The results were stark. Taking model stealing as an example, the "no-shade" baseline (where the entire model is on the GPU, completely exposed) showed an attack performance 4.26 times higher than the "black-box" baseline (where the entire model is in the TEE, offering the highest security but lowest utility). Alarmingly, TSDP solutions like DarkNet and the Magnitude approach still exhibited attack performances around 3.92 times and 3.91 times higher than the black-box baseline, respectively. This quantitative evidence underscores that these TSDP defenses offer security levels strikingly similar to a fully exposed white-box scenario. The core reason, as the speakers articulate, is that these TSDP solutions "keep only a little weights" in the TEE, while "the majority model part on the GPU still expose a large amount of privacy."

The researchers then investigated whether adjusting deployment configurations could improve security. This problem was formulated as finding an optimal configuration that minimizes utility cost while satisfying a security constraint (i.e., the security difference between the configuration and black-box is less than a threshold). Empirically, they found that across all TSDP solutions, the security-utility tradeoff was unavoidable: "security can only be improved by Shading more layers and wids." Plotting shaded FLOPs (computation amount) against attack success rate showed that each curve extended from upper-left (low shaded computation, high attack success) to lower-right (high shaded computation, low attack success). The shapes of these curves varied significantly across models and datasets, meaning that no universal optimal configuration exists. This led to the conclusion that "it is difficult to improve the security of TSDP solutions by only changing the configurations."

The fundamental limitation was identified as the "training before partition" strategy. Because all weights are updated by private data during training, privacy is diffused throughout the entire model. This makes it impossible to perfectly delineate privacy-related from privacy-irrelevant parts post-training.

To overcome this, the speakers proposed the "partition before training" strategy, which forms the basis of TSlice. The core idea is to first separate the model into a privacy-related part (called private model slices) and an unimportant public backbone before any training with private data commences. The crucial step is then to only update the private model slices using private training data. This design guarantees that:

1. Except for the private model slices, all other model parts (the public backbone) are never updated by private training data and therefore contain no privacy.
2. These private model slices, being the only privacy-sensitive components, are then exclusively deployed within the TEE.

The TSlice pipeline consists of two main steps:

1. Slice Training: Multiple small, private model slices are integrated into a larger public backbone model. The private training data is used solely to update these private slices, ensuring that the public backbone remains privacy-agnostic. This step aims to compress the functionality of the private model into these small slices while ensuring the overall model capacity.
2. Slice Simplification: This iterative process prunes or reduces the size of unimportant slices, further minimizing the TEE footprint while preserving critical functionality.

Beyond the partitioning strategy, TSlice also addresses communication security between the TEE and the GPU/CPU. The internal features of the DNN, which could potentially leak weight information, are encrypted by the TEE. The speakers mention using a technique called one-hand pad proposed in Solum for this purpose, which is described as "highly and efficient." This allows the CPU/GPU to perform computations over ciphertext, sending the results back to the TEE for decryption, thus protecting intermediate feature activations.

Experimental results confirmed TSlice's effectiveness, demonstrating "black-box level protection" with the "lowest utility cost." When evaluating performance losses on other aspects, TSlice introduced an accuracy drop of only 0.34%, and it did not increase attack performance in other attack vectors.

Demo / Proof of Concept

▶ Watch: Analyzing security-utility trade-offs across configurations (7:00)

While the talk did not feature a live, interactive demonstration in the traditional sense, the researchers implemented and evaluated their proposed TSlice solution through a robust framework. They utilized PyTorch and an SDX SDK (likely referring to an Intel SGX SDK, given the context of TEEs) to build an evaluation framework named toym. This framework is described as "a T-based confidential homogeneous framework for DM models," indicating its capability to deploy and evaluate deep learning models within TEEs.

The toym framework served as the proof of concept for TSlice, allowing the researchers to quantify its performance on real devices. Through this implementation, they demonstrated that TSlice significantly improves throughput compared to the black-box protection baseline (where the entire model is placed within the TEE). This empirical validation on a concrete framework underscores the practical viability and efficiency of the TSlice approach, moving beyond theoretical claims to demonstrate tangible performance benefits in a TEE-constrained environment.

Defensive Implications

▶ Watch: Identifying the fundamental flaw in existing TSDP approaches (8:09)

The findings of this research carry profound defensive implications for organizations and developers deploying machine learning models on edge devices, particularly those relying on TEE-shielded solutions. The primary takeaway is that existing TEE-Shielded DNN Partition (TSDP) strategies are largely insufficient to protect against sophisticated adversaries in the modern ML landscape. Defenders can no longer assume that merely partitioning a pre-trained model and placing a "critical" part in a TEE will secure their intellectual property or user data privacy.

Here are specific actions and considerations for defenders:

1. Re-evaluate Existing TSDP Deployments: Any system currently utilizing a "training before partition" TSDP strategy should be critically re-assessed. The research indicates that such deployments might be providing a false sense of security, exposing models to white-box level model stealing and membership inference attacks. Security audits should focus on the actual privacy leakage from the offloaded components, rather than relying solely on empirical claims from prior work.
2. Adopt a "Partition Before Training" Paradigm: The most significant defensive implication is the shift towards the "partition before training" paradigm, as demonstrated by TSlice. Defenders should design their ML pipelines to isolate privacy-sensitive components before training with private data. This ensures that only the truly critical model slices are updated by sensitive information and subsequently shielded by the TEE, while the larger, performance-critical public backbone remains privacy-agnostic and can be safely offloaded to GPUs.
3. Prioritize Privacy Isolation at Design Time: Instead of attempting to retroactively "strip" privacy from a fully trained model, security must be designed in from the ground up. This involves carefully identifying which parts of the model learn from private data and ensuring these are the only parts that interact with such data, thereby confining privacy leakage potential.
4. Consider Communication Security: Beyond partitioning, the communication channel between the TEE and external computational units (CPU/GPU) must also be secured. Techniques like the "one-hand pad" mentioned, or other secure computation protocols, are essential to prevent leakage of intermediate feature activations, which can themselves reveal sensitive information.
5. Benchmarking Against Stronger Adversaries: Defenders should adopt a more rigorous threat model, assuming adversaries with access to public models and data. Evaluations should not solely rely on simple black-box attacks but incorporate sophisticated surrogate model-based attacks for both model stealing and membership inference, as demonstrated in this research.
6. Balance Utility and Security Consciously: While TSlice offers a more favorable security-utility tradeoff, defenders must still make informed decisions. The research clearly shows that simply increasing the TEE-shielded portion to improve security will inevitably degrade performance. The goal is to achieve black-box level security for private components while minimizing the TEE footprint, which TSlice facilitates.

In summary, the era of large models demands a fundamental re-thinking of TEE-based ML security. Defenders must move beyond superficial partitioning strategies and embrace architectural changes that inherently isolate and protect privacy from the earliest stages of model development and deployment.

Key Takeaways

• Existing TEE-Shielded DNN Partition (TSDP) solutions are largely ineffective against strong adversaries, exposing models to significant model stealing and membership inference attacks.
• The primary vulnerability of current TSDP schemes stems from the "training before partition" paradigm, which diffuses privacy-sensitive information across the entire model.
• Simply adjusting deployment configurations or shading more layers in existing TSDP solutions does not resolve the fundamental security flaw and inevitably leads to an unfavorable security-utility tradeoff.
• TSlice introduces a novel "partition before training" strategy, wherein only designated "private model slices" are updated with private data and subsequently shielded by the TEE.
• TSlice achieves black-box level security with minimal accuracy drop (0.34%) and significant throughput improvements compared to full TEE protection, as validated by the toym evaluation framework.
• Defenders must re-evaluate current TSDP deployments and adopt architectural approaches like TSlice that prioritize privacy isolation from the design phase, ensuring only truly sensitive components interact with private data and are TEE-protected.

About the Speaker(s)

The primary speaker for this presentation is Ziqi Zhang, who is currently a postdoctoral researcher at the University of Illinois Urbana-Champaign (UIUC). The work presented was conducted during his Ph.D. studies at Peking University in China. He collaborated with several researchers on this project, including Chen Gong, Yifeng Cai, Yuanyuan Yuan, Bingyan Liu, and Shuai Wang, from both Peking University and Hong Kong University of Science and Technology (HKUST). His research focuses on the intersection of machine learning and security, specifically addressing the challenges of deploying secure and private ML models in resource-constrained environments like edge devices.