JDD: In-depth Mining of Java Deserialization Gadget Chains

JDD: In-depth Mining of Java Deserialization Gadget Chains

Speakers: Xinyu Xing, Associate Professor, Johns Hopkins University; Bo Li, PhD Student, Fudan University

Conference: Black Hat Asia

YouTube: https://www.youtube.com/watch?v=HWMjP7uFA1s

Overview

Java deserialization vulnerabilities represent a critical and persistent threat within modern application security, often leading to severe consequences such as Remote Code Execution (RCE). This talk introduces JDD, an innovative tool designed for the in-depth mining of Java deserialization gadget chains. Developed by researchers from Johns Hopkins University and Fudan University, JDD employs a novel fragment-based, bottom-up gadget search approach combined with data flow-aided payload construction to overcome the significant challenges faced by traditional detection and exploitation methods.

The core problem JDD addresses is the exponential complexity involved in identifying all possible deserialization gadget chains and then crafting the specific malicious serialized objects (known as injection objects) required to trigger them. Traditional top-down static analysis suffers from "path explosion," while generating valid payloads is often an almost insurmountable task for random fuzzing. JDD's methodology drastically reduces this complexity, enabling the discovery of previously unknown vulnerabilities and providing a robust framework for understanding and mitigating this prevalent attack vector.

The research presented is highly impactful, demonstrating JDD's ability to outperform existing state-of-the-art tools on benchmarks and, notably, to uncover 127 zero-day gadget chains in six widely-used Java applications. The insights gained from JDD's findings not only highlight the continuous evolution of these threats but also provide crucial information for developers and security professionals to build more resilient defenses against Java deserialization attacks.

Background

▶ Watch: Introduction to JDD and Java serialization basics (0:00)

Java serialization and deserialization are fundamental processes used across a multitude of application scenarios, including inter-process communication, state exchange, and caching. Serialization converts a Java object into a stream of bytes, allowing it to be stored or transmitted. Deserialization is the inverse process, reconstructing the object from that stream of bytes, faithfully restoring its fields and state.

The vulnerability arises when an attacker can manipulate the types and values of serialized data during the deserialization process. This control can lead to unintended consequences, most critically RCE. The OWASP Top 10 has consistently recognized this risk, initially as "Insecure Deserialization" (ranked #7 in 2017) and later encompassed within "Software and Data Integrity Failures" (still among the top 10 in 2021). High-profile vulnerabilities like Log4Shell (CVE-2021-44228), which impacted tens of thousands of machines within hours of its disclosure, underscore the severe impact of deserialization-related flaws. Given that Java serialization is a built-in feature deeply integrated into various frameworks like RMI and HTTP sessions, completely avoiding or replacing it is often impractical, making robust detection and defense mechanisms essential.

To illustrate, consider a simple proof-of-concept. A server receives a serialized object from a client and uses ObjectInputStream.readObject() to reconstruct it. If an attacker crafts a malicious HashMap instance, the readObject() method will recursively reconstruct its contents. During this process, specifically when HashMap.put() is called, the key.equals() method is invoked to handle potential hash collisions. This equals() method is polymorphic, meaning its actual implementation depends on the runtime type of the key object. An attacker can substitute a custom EvilExample class for the key, whose equals() method, when invoked, triggers a call to Runtime.getRuntime().exec(), executing arbitrary commands. This sequence of method calls, from the initial deserialization to the final malicious action, is known as a gadget chain. The specially crafted serialized object is the injection object.

Defending against these vulnerabilities is notoriously difficult. Prior approaches, such as blacklisting or whitelisting specific classes, are often insufficient. Attackers can frequently find alternative gadget chains that bypass these restrictions, leading to persistent threats. For example, the talk highlights how a blacklist designed to mitigate CVE-2020-2883 in WebLogic, which blocked AbstractExtractors.compare, could be bypassed by an alternative chain involving ExtractorComparator.compare, still leading to the same severe consequences. This demonstrates that simply blocking known gadgets is a game of whack-a-mole, as new bypasses are constantly discovered due to the vast and complex landscape of Java libraries.

The primary challenges for existing detection tools stem from two major issues:

1. Path Explosion: Traditional top-down static analysis, which attempts to trace all potential paths from a deserialization source to a security-sensitive sink, faces an exponential increase in candidate search paths. This is exacerbated by dynamic method invocations (e.g., virtual calls, reflection), where a single method call can have numerous implementations (e.g., Object.toString() can have 329 implementations in an application), leading to an unmanageable number of program states and execution paths.
2. Payload Generation Complexity: Even if a gadget chain is identified, generating a valid injection object to trigger it is extremely difficult. These objects often have complex, deeply nested structures with intricate field-related constraints and dependencies. Random mutation-based fuzzing is highly unlikely to stumble upon a useful payload that satisfies all these conditions.

JDD was developed to specifically address these two unresolved challenges, aiming to build a more foolproof defense against the ever-evolving landscape of Java deserialization attacks.

Key Findings

▶ Watch: Proof of Concept: Client/server deserialization interaction (4:00)

JDD's core innovation lies in its two-pronged approach: a fragment-based summary and bottom-up search for gadget chains, and data flow-aided directed fuzzing for payload construction. This methodology directly tackles the path explosion and payload generation challenges, leading to significant advancements in detecting Java deserialization vulnerabilities.

The key findings and contributions of JDD include:

• Effective Mitigation of Path Explosion: By breaking down complex gadget chains into smaller, manageable fragments and employing a bottom-up search strategy, JDD drastically reduces the search complexity from exponential to polynomial. This is achieved through a hybrid analysis approach that retains precise state information within fragments and path information between them, avoiding both state and path explosion. The bottom-up strategy, starting from security-sensitive sinks and working backward, focuses the search on relevant paths.
• Automated and Guided Payload Generation: JDD leverages static taint analysis to construct detailed data flow dependencies between potential injection object fields. These dependencies, along with extracted class hierarchical, field-related, and implicit constraints, are modeled using an Injection Object Constraint Description (IOCD) structure. This IOCD guides a directed fuzzer to generate highly effective and valid injection objects, significantly improving the exploitability verification process.
• Superior Detection Performance: In evaluations against well-known benchmarks, JDD significantly outperformed all existing state-of-the-art tools, detecting 19 previously unknown or undetected gadget chains. This demonstrates its enhanced precision and efficiency compared to baseline methods.
• Discovery of 127 Zero-Day Gadget Chains: Beyond benchmarks, JDD identified 127 zero-day gadget chains in six popular, real-world Java applications. The affected developers were notified, highlighting JDD's practical impact in uncovering critical, exploitable vulnerabilities before they could be weaponized.
• Insights into Attack Surface Expansion and Bypass Techniques: JDD's analysis revealed how attackers can broaden or alter their attack surface by replacing specific fragments within existing gadget chains. For instance, it demonstrated how a chain relying on the Groovy library could be adapted to attack protocols like Hessian (widely used in distributed systems) by replacing a fragment. Another example showed how to adapt a chain from FastJSON to Jackson (a library included in Morton applications) to exploit new targets. JDD also uncovered alternative fragments that bypass common blacklist defenses, such as replacing AbstractExtractors.compare with ExtractorComparator.compare or finding new ways to trigger hash collisions and toString() invocations.
• Identification of Critical Reusable Gadget Fragments: The research identified and documented several notable reusable gadget fragments. These fragments, such as those bridging unsafe reflection with execution effects, are crucial components in many zero-day chains and provide valuable intelligence for both attackers and defenders.

Overall, JDD represents a significant leap forward in understanding and combating Java deserialization vulnerabilities, offering a powerful, open-source tool for both research and practical application security.

Technical Deep Dive

▶ Watch: Exploiting HashMap.put via polymorphic key.equals() (5:50)

JDD's architecture is meticulously designed to address the inherent complexities of Java deserialization vulnerability detection, operating in two primary stages: gadget chain detection and injection object generation.

Stage 1: Gadget Chain Detection (Bottom-Up Gadget Search)

The first stage focuses on identifying potential gadget chains using a novel fragment-based summary and bottom-up search approach. This is crucial for overcoming the "path explosion" challenge.

1. Fragment Identification:

• JDD begins by identifying potential deserialization entry methods (e.g., HashMap.readObject() or put()).
• From these entry points, it performs static taint analysis to trace method calls.
• A "fragment" is defined as a sequence of method calls starting from an initial method and ending at the next dynamic invocation. Dynamic invocations are critical branching points (e.g., equals(), toString(), reflection calls) that can have many different implementations depending on the runtime type, and thus are where the control flow can be redirected by an attacker. Breaking the chain at these points helps manage complexity.
• For each fragment, JDD records:
• Taint Mapping (Bottom-up Behavior): This describes how taint (attacker-controlled data) flows from the fragment's last method to its first method. This "reverse" mapping is key to the bottom-up approach.
• Linking Conditions: These are the conditions that must be met for the control flow to successfully transition from one fragment to the next. They ensure reachability.
• Exploit Conditions (for Sync Fragments): For fragments that end in a security-sensitive sink (e.g., Runtime.exec(), file I/O operations), JDD records specific conditions. These include which parameters must be tainted (i.e., controllable by the attacker) when the sink method is invoked.

1. Hybrid Analysis for State and Path Management:

• To avoid both state explosion (tracking too many precise program states) and path explosion (tracking too many execution paths), JDD employs a hybrid approach.
• Within fragments, JDD retains detailed state information. This allows for precise analysis of data flow and conditions local to a fragment.
• Between fragments, JDD only retains path information, specifically the summarized taint mappings and linking conditions. This abstraction prevents the exponential growth of states when chaining fragments.

1. Fragment Training (Bottom-Up Composition):

• Unlike traditional top-down approaches that start from a source and try to reach a sink, JDD adopts a bottom-up search.
• It starts from known security-sensitive sinks (e.g., command injection, file read/write, JNI execution) and works backward, composing fragments.
• The core idea is that the exploit conditions recorded for a sink fragment, combined with the reverse taint mapping of pre-linkable fragments, allow JDD to accurately compute the specific taint requirements needed for the preceding fragment.
• By applying the linking conditions, JDD ensures that control flow reachability is maintained as fragments are composed. This focused, constraint-driven composition process significantly reduces redundant and ineffective searches, making the process much more efficient.

Stage 2: Injection Object Generation (Data Flow Aided Fuzzing)

Once a potential gadget chain is identified, the second stage focuses on automatically generating a valid injection object to trigger and verify its exploitability. This addresses the challenge of complex payload construction.

1. Data Flow Dependency Construction:

• JDD analyzes the call sequence within the identified gadget chain to construct data flow dependencies between the fields of the prospective injection object. This understanding is critical for knowing how manipulating one field might affect subsequent method calls.

1. Constraint Extraction:

• JDD extracts three main categories of constraints that dictate how the injection object must be structured and its fields populated:
• Class Hierarchical Relationships: These constraints guide the generation of the initial object structure. For example, if a fragment expects a SimpleEntry instance within a HashMap's table field, JDD determines that the table field must store instances of SimpleEntry types. This ensures that dynamic method invocations (equals(), toString()) correctly jump to the intended next fragments.
• Field-Related Constraints: These are conditions that ensure smooth method invocation within a fragment. An example from the talk is activeRetryCount < activeRetryMax, which might be a conditional branch that needs to be satisfied for the exploit to proceed.
• Implicit Constraints: These prevent the program from triggering exceptions (e.g., NullPointerException, ClassCastException). For instance, a field might be required to be non-null before a method is invoked on it, or a field might need to be an instance of a specific type due to a forced type cast.

1. Injection Object Constraint Description (IOCD):

• All these extracted constraints are modeled using a node-disk structure called IOCD.
• Class Nodes: Represent initialized object contents in the gadget chain. Each class node stores the class name and references to its relevant field nodes.
• Field Nodes: Represent fields within a class. Each field node stores the relevant constraints associated with that field.
• Interconnections: Class nodes are interconnected through field nodes using directed edges, indicating the hierarchical relationships between object instances. The IOCD also specifically identifies which fields store the attacker's payload and any constraints directly related to its construction.

1. Directed Fuzzing with IOCD Guidance:

• JDD uses the IOCD to enhance a directed fuzzer, generating high-quality exploit injection objects. The process involves four key steps:

1. Initial Instance Generation: JDD analyzes the class nodes to generate initial, parameterless Java instance objects based on the required class hierarchy.
2. Hierarchy Establishment: The directed edges in the IOCD are used to establish the correct hierarchical relationships among these instances.
3. Payload Field Selection: JDD selects the specific fields identified in the IOCD as related to the attack payload construction.
4. Constraint Solving and Assignment: Necessary constraints determined during static analysis are extracted, and a constraint solver is used to generate appropriate values for these fields, which are then assigned to the corresponding object fields.

• The generated object is then injected into the target deserialization method. JDD monitors whether the object successfully follows the gadget chain, reaches the sink, and triggers the expected exploit behavior. If successful, the chain is marked as exploitable. If not, JDD applies mutation strategies (guided by the IOCD to maintain structural validity and reduce uncertain mutation space) to iterate and refine the injection object.

By combining these sophisticated static analysis and guided fuzzing techniques, JDD provides a robust and efficient solution for discovering and verifying complex Java deserialization vulnerabilities.

Demo / Proof of Concept

▶ Watch: Remote code execution demonstration using runtime.exec (6:50)

While a live, real-time code demonstration was not explicitly shown in the transcript, the talk presented a highly detailed, step-by-step walk-through of a real-world zero-day gadget chain discovered by JDD. This served as a comprehensive conceptual proof-of-concept, illustrating the intricate nature of these vulnerabilities and how JDD's methodology unravels them.

The demonstrated gadget chain initiated from the put method of a HashMap object during deserialization. The core mechanism involved manipulating two SimpleEntry objects (S1 and S2) within the HashMap to share the same hash value, forcing the invocation of the equals method on a controllable key field.

The chain's progression was meticulously traced through several fragments:

1. Fragment 1: HashMap.put() -> key.equals() (triggered by hash collision). The attacker assigns the key field to a custom object, for example, an ExtremeForFSP object.
2. Fragment 2: ExtremeForFSP.equals() is invoked. The program then passes keys of S1 and S2 to this method.
3. Fragment 3: Inside ExtremeForFSP.equals(), the toString() method of the key field of S2 is invoked. The attacker assigns this key field as a JSON object.
4. Fragment 4: JSON.toString() is invoked. (The talk implicitly suggests this leads to further controlled execution, though not explicitly detailing the JSON object's internal logic here.)
5. Fragment 5-6 (Implied): The talk then jumps to demonstrating how to create the initial hash collision. It explains that by storing the same ExtremeForFSP and JSON object instances into the key and value fields of both S1 and S2, but in reversed order, the hashCode method of SimpleEntry can be manipulated to produce identical hash values, thus triggering the equals method.
6. Fragment 7: The chain eventually leads to the invocation of UnsafeDriver.reflection.
7. Fragment 8: Finally, if specific conditions are met (e.g., activeRetryCount < activeRetryMax within a conditional branch at line 64), a command injection attack is launched via Runtime.exec().

This detailed example effectively demonstrates:

• The complexity of real-world gadget chains: Spanning multiple classes and methods, involving intricate data structures (HashMap, SimpleEntry), and relying on dynamic dispatch.
• The role of specific field manipulation: How attackers control types and values of nested objects to steer execution.
• The importance of constraint satisfaction: The need to satisfy conditions like hash collisions and specific field values (activeRetryCount).
• How JDD breaks down and analyzes these chains: By identifying fragments, tracking data flow, and understanding the conditions for linking and exploitation.

The demonstration highlighted that such chains are "very difficult to detect in a limited time due to static path explosion" because dynamic invocations like Object.toString() can have hundreds of implementations. JDD's fragment-based, bottom-up approach, coupled with its ability to generate the precise injection object needed to satisfy all these conditions, is what enables it to discover and demonstrate the exploitability of such intricate, multi-fragment vulnerabilities.

Defensive Implications

▶ Watch: Key concepts: Injection Object and Gadget Chain (7:20)

The findings presented by JDD underscore the persistent and evolving nature of Java deserialization vulnerabilities, making full defense extremely challenging. The talk explicitly states that "the threat is persistent" and "very hard to defend against." This implies that simply relying on blacklisting known problematic classes is an insufficient and easily bypassable strategy.

Here are the key defensive implications derived from JDD's research:

• Blacklisting is Ineffective: JDD's discovery of alternative gadget fragments, such as those bypassing specific AbstractExtractors.compare blacklists, confirms that security through obscurity or simple exclusion lists is a losing battle. Attackers can always find new paths through the vast Java ecosystem. Defenders should not rely on blacklists as a primary defense.
• Proactive Detection is Crucial: Given the difficulty of prevention, proactive detection tools like JDD become indispensable. Developers and security teams should integrate such tools into their CI/CD pipelines and security audits to identify potential deserialization gadget chains in their applications and dependencies before they are exploited in the wild.
• Focus on Principle of Least Privilege and Input Validation: While JDD focuses on detection, the underlying principles of secure coding remain paramount. If deserialization is absolutely necessary, ensure that:
• Only trusted, signed, and integrity-checked data is deserialized.
• The ObjectInputStream is configured with a serialization filter (available since Java 9) to explicitly whitelist allowed classes for deserialization, rather than relying on blacklists.
• Input validation is performed rigorously on any data that might influence the deserialization process, even if it's within a serialized object.
• Understand Attack Surface Expansion: JDD demonstrated how replacing fragments can broaden attack surfaces (e.g., from Groovy to Hessian, FastJSON to Jackson). Defenders need to understand that the absence of a known vulnerable library does not guarantee safety if other libraries present can be chained to achieve the same malicious effect. This requires a holistic view of the application's entire dependency graph.
• Leverage Identified Gadget Fragments: The talk specifically mentions identifying "notable G fragments" that can bypass blacklists or trigger specific effects (hash collusion, toString() invocations, unsafe reflection). Security researchers and incident responders can use this knowledge to:
• Develop more sophisticated detection signatures.
• Perform targeted code reviews for these specific fragment patterns.
• Better understand how new bypasses might be constructed.
• Open-Source Tooling for the Community: The speakers emphasized that JDD is open-source. This is a significant defensive implication, as it allows security practitioners, researchers, and developers to:
• Integrate JDD into their own security analysis workflows.
• Contribute to its development and expand its capabilities.
• Gain deeper insights into the deserialization landscape without proprietary barriers.
• Continuous Monitoring and Patching: The discovery of 127 zero-day chains highlights the ongoing emergence of these vulnerabilities. Organizations must maintain robust patch management processes for all Java applications and their dependencies, as well as continuously monitor for new advisories and research.

In essence, JDD provides the tools and insights necessary to shift from a reactive, blacklist-based defense to a more proactive, analytical, and comprehensive approach to securing Java applications against deserialization attacks.

Key Takeaways

• Java deserialization vulnerabilities remain a high-impact, persistent threat, often leading to Remote Code Execution (RCE), as evidenced by their presence in OWASP Top 10 and incidents like Log4Shell.
• Traditional detection methods are hampered by "path explosion" due to dynamic method invocations and the extreme complexity of generating valid "injection objects" to trigger exploit chains.
• JDD addresses these challenges with a novel fragment-based, bottom-up gadget search to efficiently identify potential chains and data flow-aided directed fuzzing using an IOCD (Injection Object Constraint Description) to construct exploitable payloads.
• JDD significantly outperforms existing tools, detecting 19 previously unknown gadget chains on benchmarks and discovering 127 zero-day gadget chains in real-world Java applications, demonstrating its practical efficacy.
• The research provides critical insights into how attackers can broaden their attack surface or bypass existing defenses by replacing specific gadget fragments, emphasizing the inadequacy of simple blacklisting strategies.
• JDD is an open-source tool that offers a valuable resource for developers and security professionals to proactively identify and mitigate Java deserialization vulnerabilities in their applications and dependencies.

About the Speaker(s)

Dr. Xinyu Xing is an Associate Professor at Johns Hopkins University and serves as the Technical Director of the Johns Hopkins University Information Security Institute. His research focuses on cyber security and privacy issues within various domains, including the web, smartphones, and machine learning, leveraging different program analysis techniques.

Bo Li is a PhD student at Fudan University, affiliated with the Success Lab. His research interests lie primarily in program analysis, vulnerability detection, and exploitation.

The research also involved valuable contributions from collaborators, including Dr. Lelei Zhang, Dr. Yuan Zhang, and Dr. Meng, all from Fudan University.