Reverse Engineering MicroPython Frozen Modules

Reverse Engineering MicroPython Frozen Modules

Speakers: Wesley McGrew, Senior Cyber Fellow, Martin Federal

Conference: DEF CON 32

YouTube: https://www.youtube.com/watch?v=QXa29AJqdRc

Overview

This talk, presented by Wesley McGrew at DEF CON 32, delves into the often-misunderstood security implications of MicroPython frozen modules. While MicroPython is a lightweight implementation of Python 3 designed for resource-constrained microcontrollers, its "frozen modules" feature, intended for efficiency, is frequently misused as an obfuscation technique to hide code and secrets. McGrew's presentation meticulously outlines how to reverse engineer these modules, demonstrating that the perceived security they offer is largely illusory.

The talk is particularly relevant for individuals involved in embedded device security, CTF (Capture The Flag) challenges that leverage hardware badges, and product developers using MicroPython. It highlights a critical distinction between efficiency and security, urging developers to reconsider their reliance on obscurity for protecting sensitive information. McGrew illustrates the process using a Raspberry Pi Pico, a popular and accessible platform for MicroPython development, making the concepts tangible and reproducible for the audience.

Ultimately, this presentation serves as a crucial reminder that while a feature might make code harder to access, it rarely makes it truly secure against a determined attacker. It provides a practical methodology for uncovering code and secrets thought to be hidden, reinforcing fundamental principles of secure design in the embedded space.

Background

▶ Watch: Introduction to MicroPython frozen modules reverse engineering (0:00)

MicroPython is a lean, efficient implementation of the Python 3 programming language optimized for microcontrollers with limited resources, contrasting sharply with CPython, the standard Python interpreter found on desktop systems. Unlike CPython, which manages a full operating system environment, MicroPython focuses on core language features and specific libraries for interacting with microcontroller hardware, such as GPIO (General Purpose Input/Output) pins. It includes a subset of Python's standard libraries and adds specific MicroPython-only modules tailored for embedded development. A key feature is its REPL (Read-Eval-Print Loop), allowing interactive code execution via a serial terminal, similar to a standard Python console.

Developers using MicroPython have three primary ways to deploy their code onto a device:

1. Plain text .py files: Stored on a small, emulated file system in the flash memory, these are easily readable and writable via the REPL.
2. Pre-compiled bytecode (.mpy) modules: Python code is compiled into MicroPython-specific bytecode files. These are more efficient than plain text but still reside on the file system and retain metadata that aids in their disassembly.
3. Frozen modules: This is the most efficient option. Pre-compiled bytecode modules are "frozen" directly into the MicroPython firmware image at build time. This means the code is part of the main firmware binary, loaded directly into memory without needing additional file system operations or memory allocation at runtime.

The primary purpose of frozen modules is efficiency. By embedding modules directly into the firmware, MicroPython saves memory by avoiding the need to load .py or .mpy files from a file system into RAM. This also speeds up execution as the code is immediately available. However, this feature is frequently "abused or misused" for obfuscation. Many developers, particularly in communities like badge life or CTF organizers, believe that freezing modules makes their code and embedded secrets (like passwords or API keys) inaccessible to users. Forum discussions cited by McGrew reveal a common sentiment: while not perfect, freezing modules is seen as a way to "make it harder" for others to read proprietary code or extract sensitive data because "nobody has the tools to disassemble it." This talk directly challenges that assumption.

The process of obtaining a firmware image is a prerequisite for any reverse engineering effort. While some microcontrollers might employ protection mechanisms requiring advanced techniques like glitching, platforms like the Raspberry Pi Pico often allow firmware extraction using standard tools. For instance, the Pico tool info utility can inspect a connected Pico and, in some MicroPython firmware builds, directly list the names of frozen modules. This "binary information" is optionally encoded by MicroPython firmware developers. While this makes the initial identification of frozen modules "too easy" on some Picos, the talk aims to demonstrate extraction even when such symbolic information is absent, addressing the more challenging scenario where an adversary has actively tried to remove any helpful metadata. The mpy-tool, a utility provided with MicroPython, is used for freezing modules and inspecting .mpy files, but it does not natively support direct disassembly of frozen modules from a raw firmware image.

Key Findings

▶ Watch: Three methods for storing code in MicroPython firmware (4:00)

The central finding of Wesley McGrew's talk is that MicroPython frozen modules, despite being integrated directly into the firmware and lacking conventional metadata, are not a secure mechanism for obfuscating code or hiding secrets. While the freezing process removes much of the information that standard tools like mpy-tool rely on for easy disassembly of .mpy files, the underlying bytecode structure remains intact and recoverable.

McGrew demonstrates that it is entirely feasible to:

1. Extract the raw bytecode: The compiled Python bytecode, which represents the frozen module, can be located and extracted from the larger MicroPython firmware image.
2. Reconstruct the module: Even without explicit file system metadata or symbolic information, the bytecode can be analyzed and reconstructed into a disassemblable format. This process is shown to be achievable both when some symbolic information (like module names) is incidentally available (e.g., via Pico tool info) and, critically, when such symbols have been deliberately stripped or are naturally absent, representing a more challenging and realistic adversarial scenario.
3. Disassemble and analyze the reconstructed bytecode: Once reconstructed, the bytecode can be fed into appropriate tools (or custom scripts) to reveal the original Python logic, including any embedded strings, variables, or functions.

The talk effectively debunks the notion that freezing modules provides a significant security barrier. Instead, it confirms that this feature primarily serves efficiency purposes and should never be relied upon for confidentiality. The "layer of protection" it offers is merely an obfuscation that can be bypassed with the right reverse engineering techniques, rendering any embedded secrets or proprietary logic fully exposed to an attacker with physical access to the device and the ability to extract its firmware.

Technical Deep Dive

▶ Watch: Why frozen modules hide code and secrets in CTFs (4:50)

The technical core of reverse engineering MicroPython frozen modules lies in understanding how MicroPython compiles and stores code, and then identifying and reconstructing that structure from a raw firmware image.

MicroPython operates by compiling Python source code (.py files) into a compact, custom bytecode format, which is then typically stored in .mpy files. This bytecode is interpreted by the MicroPython virtual machine. When a module is "frozen," its compiled bytecode is directly embedded into the MicroPython firmware binary during the build process. This embedding removes the file system abstraction and much of the metadata (like module headers, file names, and string tables) that would normally accompany a standalone .mpy file.

The challenge for a reverse engineer is to locate this bytecode within the larger firmware binary and then parse it without the aid of the missing metadata. The speaker outlines a methodology that tackles this in two main scenarios:

1. With Symbols (Easier Case): On platforms like the Raspberry Pi Pico, the MicroPython firmware might optionally include additional binary information that can be read by tools like Pico tool info. This information can sometimes contain a list of the names of frozen modules. While this doesn't directly provide the bytecode, knowing the module names can significantly aid in identifying where their corresponding bytecode might reside or how they are referenced within the firmware's internal structures. The speaker mentions that MicroPython firmwares can specify "additional fields" in this binary information, which can include a "list of the frozen modules." This provides a starting point, essentially a "hint" to the reverse engineer. The mpy-tool itself, used for freezing and inspecting .mpy files, contains some logic for how this information is encoded, which could be leveraged.

1. Without Symbols (Harder Case): This scenario represents the true test of reverse engineering, where no explicit module names or helpful metadata are provided by Pico tool info or similar utilities. In this case, the reverse engineer must rely on the inherent structure of MicroPython bytecode.

• Bytecode Structure: MicroPython bytecode has a distinct signature and internal structure. Although the exact details of this structure are not exhaustively detailed in the transcript, it's implied that familiarity with the MicroPython compiler's output is crucial. This includes understanding opcodes, operand types, and how constants (including strings, integers, and function pointers) are organized.
• String Pool Reconstruction: A critical aspect mentioned by McGrew is that MicroPython often uses an "index into a linked list of pools of these strings." This means that string literals used within the Python code (e.g., variable names, function names, string constants) are not embedded directly next to their usage in the bytecode. Instead, they are stored in separate "string pools," and the bytecode refers to them by an index. To reconstruct the module, the reverse engineer must first locate and reconstruct these string pools, and then correctly map the bytecode's string references back to the extracted strings. This is a common optimization technique in interpreters to save space and enable string interning.
• Identifying Bytecode Boundaries: Without explicit headers, identifying the start and end of a frozen module's bytecode within the larger firmware can be challenging. This often involves scanning the firmware for known bytecode signatures or patterns, or by analyzing references from the MicroPython interpreter's internal function tables, if available.
• Disassembly: Once the raw bytecode segment and its associated string pools are extracted, a custom disassembler (or a modified version of mpy-tool's internal disassembler) would be required to convert the bytecode back into a human-readable form, akin to Python pseudocode or assembly-like instructions. This allows for the recovery of function definitions, variable assignments, control flow, and embedded constants.

The mpy-tool is a critical component in the MicroPython ecosystem, used for compiling .py files to .mpy and for freezing modules. While it doesn't directly de-freeze modules, its internal logic for parsing and generating .mpy files and handling frozen modules provides invaluable insight into the bytecode format and the data structures involved. By studying mpy-tool's source or behavior, a reverse engineer can develop the necessary parsers to reverse the freezing process. The talk implicitly suggests that understanding the mpy-tool's mechanisms is key to building the tools needed for reconstruction.

In summary, the technical deep dive involves a multi-step process: firmware acquisition, initial analysis for symbolic hints (if any), identification of bytecode segments and string pools based on MicroPython's internal structure, and finally, custom parsing and disassembly to recover the original code logic. This process underscores that the "obfuscation" provided by frozen modules is superficial and can be overcome by understanding the underlying interpreter's design.

Demo / Proof of Concept

▶ Watch: Using Pico tool to list embedded frozen modules (6:20)

Wesley McGrew's talk includes a comprehensive demonstration of the entire reverse engineering process for MicroPython frozen modules, specifically utilizing the Raspberry Pi Pico as the target hardware. The demo serves as a practical proof of concept, illustrating each step from creation to reconstruction.

The demonstration begins by building a simple "Hello World" style module in Python. This module, likely a .py file containing basic print statements or function definitions, is chosen for its simplicity, allowing the audience to focus on the freezing and reverse engineering process rather than complex application logic.

Next, the speaker compiles this .py file into MicroPython bytecode, resulting in an .mpy file. This step showcases the intermediate representation of Python code before it's embedded. The mpy-tool would typically be used for this compilation.

The core of the demo involves freezing this pre-compiled .mpy module into a custom MicroPython firmware image for the Raspberry Pi Pico. This process involves modifying the MicroPython build system to include the module's bytecode directly within the firmware binary. The speaker explains that this is the point where most of the helpful metadata for easy disassembly is stripped away. The generated firmware is then flashed onto a Raspberry Pi Pico.

Following the firmware flashing, McGrew moves to the reverse engineering phase, demonstrating the reconstruction of the frozen module. This is performed in two distinct scenarios:

1. Reconstruction with Symbols: In this scenario, the Pico tool info utility is used to query the connected Raspberry Pi Pico. As mentioned earlier, some MicroPython firmware builds for the Pico include optional binary information that lists the names of frozen modules. The demo illustrates how this information, though not directly providing the code, can significantly simplify the reverse engineering process by giving the analyst a clear target name and potentially hinting at its location or how it's referenced internally. This allows for a more guided extraction.

1. Reconstruction Without Symbols: This is the more challenging and realistic scenario, where no such helpful metadata is available. The demo would then proceed to show how to identify the bytecode of the "Hello World" module within the raw firmware image by analyzing its structure, potentially scanning for known bytecode signatures or patterns. The critical step here would be to reconstruct the module's string pool and correctly interpret the bytecode's references to these strings. This part of the demo highlights the custom tools or scripts required to parse MicroPython's unique bytecode format and reassemble the module's components.

Finally, the extracted and reconstructed module is analyzed. The demonstration culminates in displaying the disassembled bytecode or a reconstructed approximation of the original Python source code, proving that the "Hello World" module, despite being frozen and stripped of metadata, can be successfully recovered and understood. This hands-on example powerfully illustrates that the freezing mechanism offers only a superficial layer of obfuscation, not true security.

Defensive Implications

▶ Watch: Understanding Pico tool's limitations and mpy tool (7:20)

The detailed reverse engineering process demonstrated by Wesley McGrew carries significant defensive implications for anyone developing with MicroPython, especially in security-sensitive contexts. The primary takeaway is unequivocal: MicroPython frozen modules should not be considered a security boundary for protecting sensitive information or proprietary code.

Here are the key defensive actions and considerations:

1. Do Not Rely on Obfuscation for Security: The most critical implication is that relying on the "difficulty" of extracting frozen modules for security is a false sense of protection. The talk unequivocally proves that a determined attacker with physical access to the device and the ability to extract firmware can recover the original Python code. This means any passwords, API keys, cryptographic secrets, proprietary algorithms, or sensitive business logic embedded within frozen modules are vulnerable to extraction.

1. Implement Proper Security Controls: For applications requiring genuine confidentiality or integrity, developers must employ robust security measures that go beyond mere obfuscation. This includes:

• Hardware Security Modules (HSMs) or Secure Elements: For storing cryptographic keys and performing sensitive operations. These dedicated hardware components are designed to resist physical tampering and extraction.
• Hardware Roots of Trust: To ensure the integrity of the boot process and firmware, preventing unauthorized modifications.
• Strong Encryption: If data must reside on flash, it should be encrypted with keys managed by a secure element or derived through secure means, rather than hardcoded in accessible bytecode.
• Secure Boot: To verify the authenticity and integrity of the firmware before execution.

1. Assume Firmware is Readable: Developers should operate under the assumption that all code and data stored on a microcontroller's flash memory can eventually be read by an adversary. This principle, often called "security by design," dictates that even if a firmware image is extracted, sensitive operations and data should remain secure due to underlying hardware protections or robust cryptographic implementations.

1. Educate Teams on Misconceptions: Project managers and developers need to be educated about the limitations of frozen modules. The common belief that freezing "makes it harder" to access code should be dispelled, replacing it with an understanding that it primarily offers efficiency benefits, not security.

1. CTF Design Review: For Capture The Flag (CTF) challenges that utilize MicroPython-based badges, this research implies that hiding flags or secrets within frozen modules is a weak challenge. CTF organizers should either acknowledge this as an "easy" challenge or use more sophisticated techniques (e.g., interacting with external servers, requiring complex hardware interaction, or leveraging actual hardware security features) to protect their flags.

In essence, while MicroPython's frozen modules are excellent for optimizing performance and memory usage on constrained devices, they offer no meaningful security against reverse engineering. Defenders must shift their focus from obscurity to verifiable cryptographic and hardware-based security solutions when dealing with sensitive assets on embedded systems.

Key Takeaways

• MicroPython frozen modules are for efficiency, not security: Their primary purpose is to optimize memory usage and loading times on resource-constrained microcontrollers, not to protect code or data from reverse engineering.
• Obfuscation is not security: Relying on the "difficulty" of extracting frozen modules to hide secrets (like passwords or API keys) or proprietary code is a false sense of security; it's merely obfuscation that can be bypassed.
• Frozen modules can be reverse engineered: Despite the removal of metadata during the freezing process, the underlying MicroPython bytecode and its associated string pools are recoverable from the firmware image.
• Reconstruction is feasible with and without symbols: Even without helpful symbolic information (like module names provided by Pico tool info), the bytecode structure can be analyzed and reconstructed using custom tools and an understanding of MicroPython's internal workings.
• Don't embed secrets directly: Sensitive information should never be hardcoded or directly embedded in frozen modules. For true security, leverage hardware security modules (HSMs), secure elements, or robust cryptographic practices.
• Assume firmware is readable: Adopt a "security by design" mindset where all code and data on a device's flash memory are considered accessible to a determined adversary.

About the Speaker(s)

Wesley McGrew is a Senior Cyber Fellow with Martin Federal. His expertise lies in cybersecurity, particularly in areas like reverse engineering and embedded systems security, as evidenced by his detailed research into MicroPython frozen modules. His work often involves dissecting complex systems to understand their vulnerabilities and limitations, providing valuable insights for both developers and security professionals.