KernJC: Automated Vulnerable Environment Generation for Linux Kernel Vulnerabilities

KernJC: Automated Vulnerable Environment Generation for Linux Kernel Vulnerabilities

Speakers: Bonan and Jaho, Curiosity Security Team, National University of Singapore

Conference: Black Hat Asia

YouTube: https://www.youtube.com/watch?v=Xmig8oWzlrk

Overview

This talk introduces KernGC, an innovative tool designed to automate the generation of vulnerable environments for Linux kernel vulnerabilities. Presented by Bonan and Jaho from the National University of Singapore's Curiosity Security Team, KernGC addresses a critical bottleneck in kernel security research: the arduous and often error-prone process of reproducing disclosed vulnerabilities. By streamlining the setup of precise kernel versions and configurations, KernGC empowers researchers, developers, and security analysts to rapidly validate, analyze, and test defenses against kernel exploits.

The significance of KernGC stems from the pervasive impact and increasing frequency of Linux kernel vulnerabilities. As the core of modern computing infrastructure, including cloud environments, containers, and virtual machines, a compromised kernel can lead to severe consequences such as privilege escalation, container escape, and even host compromise. The speakers highlight that despite the abundance of publicly available proof-of-concept (PoC) exploits, the challenge of creating the exact vulnerable environment often impedes effective analysis and mitigation, making KernGC a vital contribution to the security community.

Background

▶ Watch: Introduction to KernJC and talk agenda (0:00)

Linux kernel vulnerabilities pose a significant threat across various computing paradigms, from individual workstations to large-scale cloud deployments. The speakers illustrate this through a comprehensive attack path in a modern cloud environment. An attacker might initially compromise a user-facing application (e.g., Python or NodeJS web app) through remote command execution. From this initial foothold, the attacker's goal is often to escalate privileges and break out of isolated environments. This multi-stage escalation typically involves escaping application sandboxes, then containers, then virtual machines, ultimately aiming for control of the physical host. At nearly every step of this chain—from initial access via remote kernel vulnerabilities to container breakout, privilege escalation within a VM, and VM escape (e.g., through KVM vulnerabilities)—kernel vulnerabilities play a pivotal role. The green arrows in their architectural diagram vividly underscore the pervasive impact of these weaknesses.

Reproducing a kernel vulnerability is a fundamental step for various security tasks: understanding its root cause, assessing its severity, prioritizing patches, designing detection or mitigation mechanisms, and evaluating their effectiveness. While the security community often focuses on developing and sharing proof-of-concept exploits, less attention is given to the crucial task of preparing the vulnerable environment. This oversight is problematic because the environment setup is often complex, time-consuming, and fraught with hidden challenges.

The speakers highlight common pitfalls through real-world examples. A GitHub issue reveals a researcher struggling to reproduce a vulnerability despite following published steps, encountering issues with kernel compilation and missing features (e.g., NFQ rules). The original author's response, suggesting abstract config checks, underscores the lack of precise documentation for environment setup. A more concrete case study, CVE-2021-22555, a high-severity heap-based out-of-bounds write in the Linux kernel's netfilter subsystem, illustrates the core problems. The National Vulnerability Database (NVD) might list specific vulnerable kernel versions (e.g., 5.11.22), but a closer inspection of the patch reveals that some of these listed versions are, in fact, already patched. Attempting to trigger a vulnerability in a patched kernel is futile, analogous to "not being able to wake a person pretending to be asleep."

Beyond incorrect version claims, the intricate Kconfig mechanism of the Linux kernel presents another major hurdle. For CVE-2021-22555, enabling the vulnerable code path requires specific Kconfig options such as CONFIG_NETFILTER, CONFIG_NETFILTER_X_TABLES, and CONFIG_COMPAT. Furthermore, the proof-of-concept might rely on specific kernel features like NFQ (Netfilter Queue), which itself depends on another Kconfig option. The relationships between these configurations are complex, forming a dependency graph where enabling one option might require several others to be selected or deselected. Manually identifying all necessary Kconfig options and their dependencies, along with the correct vulnerable kernel version, transforms vulnerability reproduction into a "dirty work" involving extensive source code downloading, compilation, configuration, and debugging—a process KernGC aims to fully automate.

Key Findings

▶ Watch: Understanding the broad impact of kernel vulnerabilities (2:00)

KernGC's design is predicated on two core insights: first, the presence of a patch definitively implies the absence of a vulnerability in that specific code version; and second, kernel configurations can be effectively modeled and analyzed as a graph. Leveraging these insights, KernGC delivers several key findings and contributions:

1. Automated Vulnerable Version Identification: KernGC accurately identifies the correct vulnerable kernel version by performing patch analysis. It tests the successful application of the vulnerability's patch against a range of kernel versions, starting from NVD's claims and backward tracking. A version is deemed vulnerable if the patch applies cleanly, indicating the code is unpatched. If the patch fails (e.g., "deletion not found"), the version is already patched. This mechanism effectively bypasses the inaccuracies often found in public vulnerability databases like NVD.
2. Automated Kconfig Identification: The tool constructs a Kconfig graph representing the complex interdependencies of kernel configuration options. It then identifies all necessary configurations for a given vulnerability through reachability analysis. This process starts by collecting "direct configs" (explicitly mentioned in CVE descriptions, Makefiles, or source code) and then traverses the Kconfig graph to discover "hidden configs" (implicit dependencies). This ensures that all required features and code paths are enabled.
3. Docker-like Command-Line Interface (CLI): KernGC provides a user-friendly, Docker-like CLI for managing vulnerable kernel environments. Commands like kgc build, kgc start, kgc cp, kgc logs, and kgc rm simplify the entire lifecycle of environment creation, interaction, and cleanup, significantly reducing the operational overhead for users.
4. Extensive Validation and NVD Inaccuracy Disclosure: In an evaluation involving 66 kernel vulnerabilities, KernGC successfully reproduced all of them. Crucially, approximately half of these vulnerabilities could not be reproduced using default kernel configurations, highlighting the necessity of KernGC's config identification. Furthermore, KernGC uncovered that 4 of these 66 vulnerabilities had incorrect version claims in NVD. Expanding this analysis to the entire MVD database, KernGC identified over 100 kernel vulnerabilities with incorrect version claims in NVD, underscoring a widespread problem with public vulnerability data.
5. Categorization of Configs for Analysis: KernGC's config identification for CVE-2021-22555 demonstrated its ability to not only find all manually identified configs but also categorize them into four types, making the analysis more structured and accessible for security researchers.

These findings collectively demonstrate KernGC's effectiveness in transforming the complex and error-prone task of kernel vulnerability reproduction into an automated, reliable, and user-friendly process, while also exposing significant data quality issues in widely used vulnerability databases.

Technical Deep Dive

▶ Watch: Challenges in reproducing kernel vulnerabilities (6:00)

KernGC's architecture is a multi-stage pipeline designed to profile vulnerabilities, identify the correct kernel version, determine necessary configurations, and provision the vulnerable environment.

The process begins with Vulnerability Information Profiling. Given a CVE ID, KernGC first identifies the relevant patch commit. From this commit, it extracts the patch content (e.g., modifications like buffer flag I = 0) and the specific files affected. Beyond the patch itself, KernGC gathers descriptive information from the CVE, including the vulnerability's text description, initial version information from NVD, and related CVE/CWE data. It also maintains an internal database of kernel version lists to facilitate accurate tracking.

The next critical step is Vulnerable Version Identification, which relies on a novel patch analysis technique. NVD often provides a range of vulnerable versions, but as observed, these can be inaccurate. KernGC addresses this by first locating the latest kernel version claimed as vulnerable by NVD. It then performs backward tracking, iterating downwards through the kernel version list. For each version, KernGC attempts to apply the vulnerability's patch. If the patch applies successfully, it signifies that the code in that kernel version is unpatched and thus vulnerable. Conversely, if the patch fails to apply (e.g., with an error message like "deletion not found"), it indicates that the vulnerability has already been patched in that version. This iterative process pinpoints the earliest and latest truly vulnerable kernel versions, ensuring that the target environment contains the exact unpatched code. For instance, if the first three tested versions fail to apply the patch, but the fourth succeeds, the fourth version is selected as the target.

Once the correct vulnerable version is identified, KernGC proceeds to Required Config Identification. This is arguably the most complex part, given the intricate nature of the Linux Kconfig system. KernGC identifies necessary configurations from multiple sources, termed "direct configs":

• Description-level configs: These are Kconfig options explicitly mentioned in the CVE description or related security advisories (e.g., CONFIG_POST_TEMPER for a specific vulnerability).
• Path-level configs: Extracted from the Makefile of the affected kernel subsystem or file. For example, a Makefile might specify that a certain source file is compiled only if CONFIG_NETFILTER_X_TABLES is enabled.
• Code-level configs: Found directly within the vulnerable source code, often within conditional compilation directives like #ifdef CONFIG_COMPAT.

After collecting these initial direct configs, KernGC constructs a Kconfig graph. In this graph, each Kconfig option is a node, and the relationships between them (e.g., depends on, select, imply) form the edges. KernGC then performs reachability analysis on this graph. For each direct config, it traverses the graph to identify all its implicit dependencies and selections—the "hidden configs." For example, enabling CONFIG_NFQ might implicitly require CONFIG_NETFILTER and other related options. By merging the direct and hidden configs, KernGC generates a comprehensive list of all Kconfig options required to enable the vulnerable code path and associated features. The speakers demonstrated this process for CVE-2021-22555, where KernGC successfully identified all the manually discovered configs and categorized them for clearer analysis.

Finally, with the precise vulnerable kernel version and the complete set of required Kconfig options, KernGC moves to Environment Provisioning. It automatically downloads the specified kernel source, applies the identified configurations, compiles the kernel, and then provisions a virtual machine (VM) with this custom-built vulnerable kernel. This VM is then ready for security analysis, PoC execution, and defense mechanism testing. The entire process is orchestrated to be fully automated, freeing researchers from the manual, time-consuming, and error-prone tasks of environment setup.

Demo / Proof of Concept

▶ Watch: Real-world examples of failed vulnerability reproduction (8:00)

The speakers provided a compelling demonstration of KernGC's capabilities, using the previously discussed CVE-2021-22555 as a motivating example. The demo highlighted the simplicity and efficiency of KernGC's Docker-like command-line interface (CLI).

The process began by building the vulnerable environment:

kgc build CVE-2021-22555

This single command triggered KernGC to perform all the underlying steps: profiling the vulnerability, identifying the correct kernel version (which might differ from NVD's claims), determining all necessary Kconfig options via graph analysis, compiling the custom kernel, and packaging it into a reproducible environment.

Once the environment was built, the user could start it:

kgc start <environment_ID>

This command launched a virtual machine running the specially crafted vulnerable Linux kernel. To interact with the environment and execute a proof-of-concept (PoC) exploit, the user would compile their PoC locally and then copy it into the running VM:

kgc cp /path/to/local/poc <environment_ID>:/path/to/remote/poc

To monitor the kernel's behavior and confirm the vulnerability trigger, the user would open a separate terminal for logs:

kgc logs <environment_ID>

Then, in the original terminal, the user would attach to the running environment:

kgc attach <environment_ID>

Inside the attached terminal, the user would execute the copied PoC. Immediately, in the kgc logs terminal, evidence of the vulnerability being triggered would appear, typically in the form of a kernel panic, a sanitizer report, or other debugging output indicating the successful exploitation of the heap out-of-bounds vulnerability.

Finally, after completing the analysis, the user could easily remove the environment, ensuring a clean workspace:

kgc rm <environment_ID>

This demonstration effectively showcased how KernGC transforms a historically "dirty work" of kernel vulnerability reproduction into a "neat and clean", fully automated, and easily manageable process. The ability to quickly spin up, interact with, and tear down precise vulnerable environments is a game-changer for kernel security research.

Defensive Implications

▶ Watch: Detailed analysis of kernel patch and configuration challenges (10:00)

KernGC offers significant defensive implications for various stakeholders within the cybersecurity ecosystem:

• For Vulnerability Researchers and Exploit Developers: KernGC drastically reduces the time and effort required to set up vulnerable environments. This acceleration allows researchers to focus more on root cause analysis, developing reliable PoCs, and understanding exploit primitives, rather than struggling with environment configuration. It provides a standardized and reproducible platform for validating research findings.
• For Kernel Developers and Maintainers: The tool can assist in verifying patches. By building a pre-patch environment and confirming the vulnerability, then building a post-patch environment and confirming its absence, developers can rigorously test the effectiveness of their fixes. The detailed Kconfig insights provided by KernGC can also help them understand how specific features interact and which configurations enable potentially risky code paths.
• For Security Analysts and Incident Responders: KernGC enables rapid validation of disclosed vulnerabilities against specific kernel versions in use within an organization. Instead of relying solely on NVD claims, analysts can use KernGC to build environments and verify if their deployed kernel versions are indeed vulnerable. This capability is crucial for accurate risk assessment and prioritization of patching efforts. The finding that over 100 NVD entries contain incorrect version claims highlights the critical need for such independent verification.
• For Detection and Mitigation Developers: Building and testing security solutions (e.g., kernel-level intrusion detection systems, exploit mitigations, fuzzers) requires a reliable way to trigger vulnerabilities. KernGC provides a consistent, automated method to generate these testbeds, allowing developers to evaluate their tools against a wide range of real-world kernel vulnerabilities and ensure their defenses are effective. The ability to enable/disable KVM or other mitigations via kgc start options further aids in controlled testing.
• For Security Auditors and Compliance Teams: KernGC can be used to audit the security posture of custom kernel builds or specific configurations. By systematically reproducing known vulnerabilities under different Kconfig settings, organizations can identify potential weaknesses that might arise from non-default configurations or outdated components. The observation that roughly half of the tested vulnerabilities required non-default configurations underscores the importance of this.

In essence, KernGC demystifies and automates a complex, error-prone process, empowering defenders to be more proactive, accurate, and efficient in their fight against Linux kernel vulnerabilities.

Key Takeaways

• Kernel vulnerability reproduction is a major bottleneck: Setting up vulnerable Linux kernel environments is complex, time-consuming, and often hindered by incorrect version information and intricate Kconfig dependencies.
• NVD often contains inaccurate version claims: KernGC's analysis revealed over 100 kernel vulnerabilities in the MVD database with incorrect version claims, highlighting the unreliability of public data for precise reproduction.
• KernGC automates the entire environment setup: The tool automatically identifies correct vulnerable kernel versions through patch analysis and determines all necessary Kconfig options via a Kconfig graph and reachability analysis.
• A Docker-like CLI simplifies usage: KernGC provides a user-friendly command-line interface (kgc build, kgc start, kgc cp, kgc logs, kgc rm) for seamless management of vulnerable kernel environments.
• Many vulnerabilities require non-default configurations: KernGC demonstrated that approximately half of the 66 tested vulnerabilities could not be triggered with default kernel configurations, emphasizing the need for targeted config identification.
• KernGC enhances security research and defense: By making kernel vulnerability reproduction reliable and efficient, KernGC empowers researchers to analyze vulnerabilities faster and enables defenders to better validate, prioritize, and test their security mechanisms.

About the Speaker(s)

The talk was presented by Bonan and Jaho, who are members of the Curiosity Security Team at the National University of Singapore. Their research focuses on addressing practical challenges in kernel security, as exemplified by the development of KernGC. The team maintains a homepage where more information about their research endeavors can be found.