Truman: Constructing Device Behavior Models from OS Drivers to Fuzz Virtual Devices

Truman: Constructing Device Behavior Models from OS Drivers to Fuzz Virtual Devices

Speakers: Zheyu Ma, Tsinghua University

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=i6V2DaEGv9k

Overview

In the rapidly expanding landscape of cloud computing, hypervisors serve as the foundational layer, orchestrating virtual machines and ensuring the efficient, isolated, and secure sharing of hardware resources. Major cloud providers like AWS and Azure fundamentally rely on these hypervisors. However, their critical role also makes them a prime target for sophisticated attackers, with vulnerabilities in virtual devices posing a significant threat for virtual machine (VM) escapes.

This talk introduces Truman, a novel fuzzing framework developed by Zheyu Ma from Tsinghua University, in collaboration with Jun Lab and EPFL. Truman addresses the inherent complexity and security risks associated with virtual devices by automating the construction of detailed device behavior models directly from open-source operating system (OS) drivers. This innovative approach enables highly effective, dependency-aware fuzzing of virtual devices, even those with closed-source implementations.

Truman's significance lies in its ability to overcome the limitations of traditional fuzzing techniques, which often struggle with the intricate inter-message, intra-message, and state dependencies characteristic of virtual device communication. By systematically inferring these dependencies from readily available drivers, Truman enhances test coverage and significantly improves the chances of discovering critical vulnerabilities, thereby bolstering the security posture of hypervisors and the entire cloud infrastructure.

Background

▶ Watch: Introduction, hypervisor security and virtual device vulnerabilities (0:00)

Cloud computing has fundamentally reshaped resource and application management, with hypervisors at its core. These hypervisors act as a crucial intermediary between host hardware and virtual machines, managing communication with virtual devices through mechanisms such as Port-Mapped I/O (PIO), Memory-Mapped I/O (MMIO), or Direct Memory Access (DMA). While essential for emulating hardware and facilitating guest OS interactions, this inherent complexity introduces a significant attack surface. Bugs within virtual device implementations can be exploited by malicious guest operating systems to achieve VM escape, potentially leading to memory corruption or arbitrary code execution on the host system.

Previous research in virtual device fuzzing has explored various avenues, each with its own set of limitations. Tools like Hypercube and Nyx focused on providing tailored fuzzing environments but often lacked deep semantic analysis of device interactions. Others, such as VDF, targeted specific interfaces like PIO and MMIO but struggled with comprehensive coverage. Grammar-based fuzzers, including vshhat and Morph, improved input generation by defining device-specific grammars; however, this process was largely manual and time-consuming, hindering scalability. Semantic-focused tools like Vaso attempted to infer device behavior from the virtual device's source code, yet they often struggled with generalization across diverse virtual devices, particularly when the virtual device implementation was closed-source.

These gaps highlighted a critical need for a more automated, semantic-aware, and generalizable approach. The primary challenges in fuzzing virtual devices stem from two intertwined issues:

1. Managing Inter-message and Intra-message Dependencies: Virtual devices adhere to strict operational sequences (e.g., setup, configuration, operation, cleanup). Violating this order results in invalid inputs that fail to properly test the device. Furthermore, individual messages often contain field-level dependencies, where components within a register or pointers in a nested DMA buffer structure must maintain specific relationships. Traditional fuzzers often generate inputs that disregard these crucial dependencies, leading to inefficient testing.
2. Understanding State Dependency: Device operations are often valid only within specific internal states (e.g., initialization, operational, error states). This challenge is compounded for "bus hidden devices," such as those utilizing the VirtIO protocol, where internal states are not directly observable by the fuzzer. Without visibility into these states, crafting effective fuzzing inputs becomes exceedingly difficult.

Truman's key insight to overcome these challenges lies in a fundamental asymmetry: while virtual devices themselves are frequently closed-source, their corresponding device drivers, which operate within the guest operating system, are typically open-source. These drivers must be accessible to developers and adhere to standard specifications that dictate how the virtual device and its driver interact, including message formats, field relationships, and expected behaviors. Essentially, the virtual device and its driver are two sides of the same coin, both conforming to the same behavioral blueprint. By systematically analyzing these open-source drivers, Truman can reconstruct accurate behavior models of virtual devices, forming the foundation for efficient, dependency-aware fuzzing.

Key Findings

▶ Watch: Key challenges: managing inter/intra-message and state dependencies (3:00)

Truman's evaluation demonstrates its significant advancements in virtual device fuzzing, yielding impressive results across multiple metrics and hypervisor platforms. The key findings presented include:

• Superior Code Coverage: Truman achieved higher code coverage in 19 out of 29 evaluated virtual devices when compared against leading fuzzing tools such as Morph, Vazzo, and AFL++. This enhanced coverage directly translates to a more thorough exploration of the target system's logic and state space, significantly increasing the probability of uncovering latent vulnerabilities.
• Discovery of Numerous Vulnerabilities: Truman successfully identified both previously reported vulnerabilities and a substantial number of new security flaws. Its ability to replicate complex bug conditions validated its accuracy and comprehensive exploration capabilities.
• 54 New Vulnerabilities, Including Six CVEs: Beyond known issues, Truman discovered a remarkable 54 new vulnerabilities across various virtual device implementations. Crucially, six of these were assigned Common Vulnerabilities and Exposures (CVE) identifiers, underscoring the severity and impact of these findings. These CVEs highlight critical security weaknesses that could potentially be exploited for VM escape.
• Effectiveness Across Diverse Hypervisors: Truman proved effective in discovering bugs across a range of major hypervisors, encompassing both open-source platforms like QEMU and VirtualBox, as well as closed-source commercial solutions such as VMware and Parallels. This broad applicability demonstrates the generalizability of Truman's approach, confirming its utility in securing a wide spectrum of virtualization environments.

These findings collectively establish Truman as a robust and highly effective solution for securing hypervisors by systematically identifying vulnerabilities in virtual devices, representing a significant step forward in the field of virtualized system security.

Technical Deep Dive

▶ Watch: Truman's key insight: leveraging open-source OS drivers (4:30)

Truman's sophisticated design is structured around a two-step framework engineered to comprehensively address the challenges of fuzzing virtual devices. This framework leverages the availability of open-source device drivers to infer the complex behavioral models of often closed-source virtual devices.

The first step involves the extraction of inter-message and intra-message dependencies. Truman achieves this by performing in-depth static analysis on open-source device drivers. For inter-message dependencies, which dictate the valid sequence of operations, Truman employs call graph and control flow graph traversal. By analyzing how different driver functions interact and call each other, it reconstructs the logical order in which messages must be exchanged between the driver and the virtual device. This ensures that fuzzing inputs adhere to the required operational sequence, preventing the generation of meaningless or invalid test cases. For intra-message dependencies, which govern the relationships between individual fields within a single message, Truman utilizes backward data flow analysis. This method traces how data values flow and are manipulated within the driver, identifying which fields depend on others, such as a control field enabling or disabling specific features in another field, or pointers requiring specific memory allocations. This ensures that mutated fields maintain logical consistency within the message structure.

The second critical component of Truman's design is its ability to analyze state dependencies. Certain device operations are only valid when the device is in a specific internal state (e.g., initialized, configured, operational, or error state). To infer these states and their transitions, Truman meticulously examines the interactions between the bus driver (e.g., the VirtIO bus driver) and the device driver. This analysis allows Truman to deduce the device's internal state machine and how specific message exchanges influence these states. This state-aware approach is particularly vital for bus hidden devices, such as those utilizing the VirtIO protocol, where internal states are not directly observable. By understanding these state transitions, Truman can generate fuzzing inputs that are precisely tailored to specific contexts, ensuring meaningful exploration of device behaviors that traditional, state-agnostic fuzzers would likely miss.

The static analysis performed by Truman is highly detailed and robust. During the Q&A session, the speaker elaborated that the device driver code is first converted into an LLVM Intermediate Representation (IR). Subsequently, advanced static analysis techniques are applied, leveraging methods from tools like SVF (Static Value-Flow Analysis Framework). Specifically, Truman conducts flow-sensitive, path-sensitive, and field-sensitive static analysis.

• Flow-sensitive analysis considers the order of statements within a program.
• Path-sensitive analysis distinguishes between different execution paths, understanding how dependencies might vary based on conditional branches.
• Field-sensitive analysis tracks dependencies at the granularity of individual fields within data structures, which is crucial for intra-message dependency extraction.

These sophisticated analyses enable Truman to construct comprehensive dependency graphs and accurate device behavior models.

With these refined behavior models, Truman implements a dependency-aware fuzzing engine designed to perform generation and mutation at multiple granularities:

• Message Level: Truman can modify individual fields within a message, exploring how specific changes affect the device's behavior while respecting intra-message dependencies.
• Sequence Level: It tests how altering the order or combination of multiple messages impacts the device. This is crucial for discovering vulnerabilities related to state transitions, such as improper handling when moving from a configuration state to an operational state.
• State Level: By simulating different internal contexts based on the inferred state machine, Truman explores how the device behaves under specific internal states. This allows the fuzzer to reach deep, hidden code paths that are often inaccessible to traditional, less context-aware fuzzers.

By combining these two steps—automated dependency extraction from open-source drivers and multi-level dependency-aware fuzzing—Truman bridges significant gaps in existing fuzzing techniques, offering a more robust and effective solution for identifying vulnerabilities in virtual devices.

Demo / Proof of Concept

▶ Watch: Analyzing state dependencies, crucial for bus-hidden devices (8:00)

The presentation primarily focused on the architectural design, methodology, and empirical evaluation of Truman, rather than a live demonstration or a step-by-step proof-of-concept walkthrough of the tool in action. While the talk detailed the mechanisms of dependency extraction and fuzzing, it presented the effectiveness of Truman through its quantitative results.

The evidence for Truman's capabilities is provided by its successful discovery of both known vulnerabilities and a significant number of new bugs, including six CVEs, across various major hypervisors. This robust validation through real-world vulnerability findings serves as the primary proof of concept for Truman's efficacy in securing virtual devices.

Defensive Implications

▶ Watch: Truman's multi-level dependency-aware fuzzing approach (8:50)

Truman's research offers critical insights and actionable intelligence for defenders, particularly those involved in hypervisor development, cloud infrastructure security, and virtual machine management. The defensive implications are multi-faceted:

1. Prioritize Virtual Device Security: The discovery of 54 new vulnerabilities, including six CVEs, underscores that virtual devices remain a significant and often overlooked attack surface. Defenders must recognize that flaws in virtual device emulation can directly lead to hypervisor compromise and VM escape, making their security paramount.
2. Integrate Driver-Based Behavior Modeling into Development Cycles: Hypervisor developers should consider integrating automated tools like Truman into their continuous integration/continuous deployment (CI/CD) pipelines. Leveraging open-source device drivers to build behavior models for virtual devices can provide an efficient and scalable method for identifying security flaws early in the development lifecycle.
3. Enhance Input Validation and State Management: The findings highlight that many vulnerabilities likely stem from inadequate handling of complex inter-message, intra-message, and state dependencies. Developers of virtual devices should implement more rigorous input validation at all levels and ensure robust state machine management to prevent unexpected behaviors and memory corruption.
4. Focus on Bus-Hidden Devices: The specific challenges with bus hidden devices (like those using VirtIO) indicate a need for focused attention on these areas. Their opaque internal states make them harder to test with traditional methods, suggesting that methodologies like Truman's, which infer states from driver interactions, are crucial for their comprehensive security assessment.
5. Audit Existing Virtual Device Implementations: Given Truman's success in finding bugs in widely deployed hypervisors (QEMU, VirtualBox, VMware, Parallels), security teams should initiate or expand audits of their existing virtual device codebases. Special attention should be given to older or less frequently updated virtual device emulations.
6. Monitor for CVEs and Apply Patches Promptly: Defenders must stay informed about newly discovered CVEs related to virtual devices and apply patches expeditiously. The six CVEs identified by Truman are critical indicators of exploitable flaws that require immediate remediation.
7. General Applicability to Open-Source/Closed-Source Scenarios: The speaker noted that Truman's approach could be applied to other scenarios involving open-source drivers and closed-source hardware or software. This suggests that the methodology of inferring behavior from publicly available interfaces can be a powerful defensive strategy beyond just hypervisor security.

In essence, Truman provides a powerful reminder that the interaction between guest OS drivers and virtual hardware is a rich area for security vulnerabilities. By understanding and proactively addressing the complexities of device communication and state transitions, defenders can significantly fortify the security of virtualized environments.

Key Takeaways

• Virtual Devices are Critical Attack Surfaces: Bugs in virtual devices pose a severe threat, potentially leading to hypervisor compromise and VM escape in cloud environments.
• Leveraging Open-Source Drivers for Closed-Source Devices: Truman's core innovation is its ability to infer accurate behavior models of closed-source virtual devices by statically analyzing their corresponding open-source OS drivers.
• Automated Dependency Extraction: The tool automates the extraction of complex inter-message, intra-message, and state dependencies, overcoming a major limitation of traditional fuzzing techniques.
• Multi-Level Dependency-Aware Fuzzing: Truman employs a sophisticated fuzzing engine that operates at message, sequence, and state levels, ensuring comprehensive exploration of device behaviors and hidden code paths.
• Significant Vulnerability Discovery: Truman achieved higher code coverage in 19 out of 29 devices and discovered 54 new vulnerabilities, including six CVEs, across major hypervisors like QEMU, VirtualBox, VMware, and Parallels.
• Broad Applicability: The methodology of inferring behavior from open-source drivers is generalizable and can be applied to other scenarios involving open-source software interacting with closed-source components.

About the Speaker(s)

The primary speaker for this presentation was Zheyu Ma, representing Tsinghua University. The work presented on Truman was a collaborative effort involving Tsinghua University, Jun Lab, and EPFL (École Polytechnique Fédérale de Lausanne), highlighting a strong academic partnership in cutting-edge security research.