Break the Wall from bottom: Automated Discovery of Protocol-Level Evasion Vulnerabilities in Web Application Firewalls

Break the Wall from bottom: Automated Discovery of Protocol-Level Evasion Vulnerabilities in Web Application Firewalls

Speakers: Qi Wang; Jianjun Chen; Zheyu Jiang; Run Guo; Ximeng Liu; Chao Zhang

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=k62qrY9LVvA

Overview

This talk, presented by Qi Wang at IEEE S&P, delves into a critical and persistent challenge in web security: the evasion of Web Application Firewalls (WAFs) through protocol-level vulnerabilities. WAFs are a cornerstone of modern web application protection, acting as a crucial defense layer against various online threats, from SQL injection to Cross-Site Scripting (XSS). However, their effectiveness hinges on their ability to accurately parse and interpret HTTP requests, a task that often proves more complex than anticipated due to the inherent ambiguities and evolving standards of the HTTP protocol.

The research highlights that while WAFs employ sophisticated rule sets to detect malicious payloads, attackers frequently bypass these defenses by exploiting discrepancies in how WAFs and backend web application frameworks parse the same HTTP request. Historically, the discovery of such protocol-level evasion techniques has been a manual, painstaking process, leading to many vulnerabilities being overlooked. This presentation introduces WAF-Menace, an innovative automated fuzzing framework designed to systematically uncover these hidden bypasses, significantly advancing the state-of-the-art in WAF security testing.

The work is highly significant for several reasons. First, it addresses a fundamental weakness in WAF design, demonstrating that even with mature and strict detection rules, a WAF can fail if it misinterprets the very structure of the incoming request. Second, by automating the discovery process, WAF-Menace provides a powerful tool for both defenders to harden their WAFs and researchers to explore the vast landscape of HTTP parsing ambiguities. The findings underscore the continuous cat-and-mouse game between attackers and defenders, emphasizing the need for WAFs to be meticulously aligned with the parsing behaviors of popular web application frameworks.

Background

▶ Watch: Introduction to WAFs and bypass challenges (0:00)

Web Application Firewalls (WAFs) have become an indispensable component of web security infrastructure, serving as the first line of defense for countless web applications. They are designed to inspect incoming HTTP/S traffic, identify malicious patterns, and block requests that could exploit vulnerabilities. Industry guidelines, such as PCI DSS and HIPAA, frequently mandate the deployment of WAFs, and their market share continues to grow due to the increasing sophistication of web-based attacks. The working principle of a WAF typically involves three main strategies: first, parsing HTTP messages to extract parameters; second, matching these parameters against a predefined set of security rules (e.g., for SQL injection or XSS); and third, taking an action (like blocking the request) if any rule matches.

However, attackers are constantly seeking ways to bypass WAFs. A common initial approach involves mutating payloads, such as using case conversion or various encoding methods, to trick simple rule-based detections. While commercial WAFs, often backed by extensive rule sets like the OWASP Core Rule Set (CRS) with over 5,000 rules, are generally robust against such basic payload mutations, a more insidious class of bypasses exists at the protocol level. This occurs when the WAF and the backend web application server interpret the same HTTP request differently. For example, a WAF might fail to recognize a JSON body if the Content-Type header is unusual (e.g., X-Whatever-Json), allowing a malicious payload to bypass detection despite strict SQL injection rules.

The concept of protocol-level evasion has been discussed for years, even presented at conferences like Black Hat USA. Yet, the vast majority of discoveries in this domain have been manual, relying on expert knowledge and painstaking reverse-engineering of WAF and web server parsing logic. This manual approach is inherently inefficient and prone to overlooking subtle parsing discrepancies. Previous research, such as WAF-A-MoLE, has explored using adversarial machine learning to craft SQL injection payloads that bypass WAFs. However, such approaches are often language-specific; a SQL injection bypass crafted for a database query cannot be directly adapted to bypass a Cross-Site Scripting (XSS) attack, for instance, due to fundamental grammatical differences between SQL and JavaScript. The core problem remains that a WAF's ability to correctly detect malicious payloads is fundamentally dependent on its ability to correctly parse HTTP parameters, aligning with how the actual web application framework processes them. If this foundational parsing step is flawed or inconsistent, even the most advanced detection rules become irrelevant.

Key Findings

▶ Watch: Uncovering protocol-level evasion vulnerabilities (2:10)

The research presented introduces WAF-Menace, a novel, automated fuzzing framework designed to systematically uncover protocol-level WAF evasion vulnerabilities. This framework addresses the critical gap left by manual discovery methods, demonstrating that fuzzing, a highly effective technique for finding memory bugs, can be adapted to efficiently mine for WAF bypasses.

The key findings and contributions of this work are:

• Automated Discovery Framework (WAF-Menace): The development and implementation of WAF-Menace, a black-box fuzzing framework guided by code coverage, which can automatically identify WAF evasion vulnerabilities by exploiting HTTP parsing discrepancies between WAFs and web application frameworks.
• Three Protocol-Level Evasion Tactics: Through manual analysis of the discovered evasion cases, the researchers summarized three distinct and novel tactics for bypassing WAFs at the protocol level:

1. Malformed Parameter Structures: Attackers craft HTTP requests with syntactically malformed parameters that the WAF fails to parse correctly, while the backend web application framework is tolerant and extracts the intended malicious payload.
2. Parameter Type Confusion: Attackers manipulate HTTP headers or structures to mislead the WAF into misidentifying the content type or parameter structure, causing it to apply incorrect parsing logic or rules. The web application, however, correctly identifies and processes the parameters.
3. Feature Spot Gap: Attackers leverage deprecated, extended, or non-standard HTTP features that are supported by the web application framework but are either unknown or not correctly processed by the WAF.

• Extensive Vulnerability Discovery: WAF-Menace successfully identified a staggering 3,011 unique evasion samples across a comprehensive evaluation matrix of 40 WAFs and 20 popular open-source web application frameworks. This broad coverage demonstrates the widespread prevalence of these vulnerabilities.
• Widespread Parsing Inconsistencies: The evaluation revealed two critical facts:
• All 20 tested web application frameworks were found to accept some form of non-standard or non-regular HTTP requests, indicating a degree of parsing tolerance that attackers can exploit.
• The majority of the 40 tested WAFs could be bypassed with specific, crafted HTTP requests, highlighting a significant security blind spot.
• Vendor Reporting and Fixes: The discovered vulnerabilities were responsibly disclosed to affected vendors. As of the presentation, 11 vendors had received reports, 9 had confirmed the issues, and 8 had already released fixes, demonstrating the practical impact and urgency of these findings.

These findings collectively underscore a fundamental flaw in the current WAF defense paradigm, where the assumption of consistent HTTP parsing between defense and application layers is often violated, paving the way for sophisticated bypasses.

Technical Deep Dive

▶ Watch: Three challenges for automated WAF evasion discovery (3:15)

The core challenge addressed by WAF-Menace is the automated discovery of protocol-level WAF evasion. Unlike traditional memory bugs that cause crashes, WAF evasion is a "silent bug" – the WAF simply fails to block a malicious request, forwarding it to the backend without error. To tackle this, the researchers identified and solved three main technical challenges.

The first challenge is to efficiently generate and mutate testing requests. For correctness, the test inputs must consistently contain the original malicious payloads. For efficiency, the search space for mutations should be constrained to closely resemble legitimate HTTP messages, reducing irrelevant testing. WAF-Menace addresses this by using grammar-based fuzzing with Abstract Syntax Trees (ASTs). Instead of raw messages, the fuzzer operates on ASTs, allowing for precise control. Malicious payload nodes within the AST are "frozen," ensuring they remain intact throughout mutation, thus preserving the correctness of the attack. The AST structure also limits the search space by guiding mutations according to HTTP grammar rules. Mutations occur at two levels:

• Grammar Level: Duplicating, deleting, or removing non-leaf nodes in the AST to alter the overall HTTP structure.
• Byte Level: Deleting or adding bytes within leaf nodes (raw strings) and encoding leaf node content, but crucially, not within the frozen payload nodes.

The second challenge is to efficiently test black-box WAFs. Most commercial WAFs are closed-source, making traditional white-box fuzzing approaches (which rely on code instrumentation) impossible. Existing black-box fuzzers often generate inputs blindly, which is inefficient. WAF-Menace leverages the availability of open-source HTTP parsers in many web developer frameworks. It uses a hybrid fuzzing strategy:

• White-box Testing Phase: An initial phase where WAF-Menace fuzzes HTTP requests against open-source web APIs. During this phase, it collects code coverage information from these parsers.
• Coverage-Guided Black-box Testing: The collected code coverage guides the fuzzing process, allowing WAF-Menace to prioritize and generate test requests that are more likely to trigger different parsing paths in the WAF, as both WAFs and web APIs are implemented to parse HTTP requests. Requests that are successfully parsed by the web API (indicating they are syntactically plausible) are then forwarded to the black-box WAFs.

The third challenge is to automatically detect WAF evasion cases. As WAF evasion doesn't cause crashes, a new detection mechanism is required. WAF-Menace employs a two-pronged validation system:

• WAF Validators: These components monitor whether a request sent to the WAF is blocked or forwarded. A successful bypass means the WAF forwards the request, typically returning a specific status code or the exact forwarded HTTP request in its response.
• Web API Validators: These are deployed on the backend web application server and are designed to extract the parameters that the web API actually uses from the forwarded request.

An evasion case is successfully detected when the WAF validator passes (request is forwarded, not blocked) AND the Web API validator passes (the web API correctly extracts and processes the original malicious payload). This two-step validation is crucial for excluding false positives, especially those caused by WAFs normalizing requests before forwarding.

The overall WAF-Menace framework integrates these components:

1. Generator: Based on ABNF grammars extracted from official HTTP standards (e.g., RFCs), it builds initial ASTs. It starts from a symbol (e.g., body) and recursively derives symbols until reaching terminal nodes (raw strings), forming a fully expanded AST representing an HTTP request.
2. Mutator: Takes an AST and applies grammar-level or byte-level mutations, ensuring payload nodes are preserved. For instance, it might duplicate subtrees (e.g., a body part), delete bytes in boundary nodes, or encode leaf nodes.
3. Web Executors (White-box testing): Sends mutated requests to open-source web APIs (e.g., Flask, Django). It collects code coverage from these parsers and prioritizes promising requests in the fuzzing corpus.
4. WAF Validators (Black-box testing): Sends valid HTTP requests (those successfully parsed by web APIs) to the target WAFs. It checks for specific WAF block pages or status codes and captures the exact request forwarded by the WAF.
5. Web API Validators (Black-box testing): Receives the WAF-forwarded request and attempts to extract parameters. This confirms if the malicious payload is indeed reaching the application layer correctly.
6. Evasion Sample Centrifuge: A crucial post-processing component that minimizes and verifies discovered evasion samples. It iteratively removes "useless" nodes from a successful evasion AST, re-validating the bypass with both WAF and Web API validators. This process helps create minimal, unique evasion samples and eliminates redundant parts of the request, making the vulnerabilities easier to analyze and patch. All verified evasion samples are stored for further analysis and reporting.

Demo / Proof of Concept

▶ Watch: Request generation and mutation using ASTs (6:05)

While the talk did not feature a live demo, the speaker provided concrete examples illustrating each of the three discovered protocol-level WAF evasion tactics. These examples serve as powerful proof-of-concepts, demonstrating how WAF-Menace uncovers subtle parsing discrepancies that lead to bypasses.

1. Parameter Type Confusion:

This tactic involves misleading the WAF about the type of parameters being sent, causing it to apply incorrect parsing logic.

• Example 1: Fake Content-Type Header: An attacker sends a request where the WAF believes the body is XML due to an early Content-Type header, but the actual web application (e.g., a Flask API) uses the last Content-Type header (e.g., application/json) to determine the body type. This discrepancy allows a SQL injection payload within the JSON body to bypass the WAF, even if the WAF has strict SQL injection rules for XML parsing.
• Example 2: Alibaba Cloud WAF and File Parameters: An attacker crafted a request that made the Alibaba Cloud WAF believe it was processing a file upload. By adding specific headers like Content-Transfer-Encoding and a filename parameter within the Content-Disposition header, the WAF was misled into treating the body as a file, and thus not applying its normal parameter validation. However, a PHP-based web application would parse this request as normal parameters because it interprets line separators differently, recognizing two distinct headers and no filename parameter in the Content-Disposition. This allowed a SQL injection attack to succeed against the PHP application, bypassing the Alibaba Cloud WAF.

2. Malformed Parameter Structures:

Here, attackers construct data that is syntactically malformed according to strict HTTP standards, but which the WAF fails to parse correctly, while the more tolerant web application framework successfully extracts the intended parameters.

• Example 1: Cloudflare WAF and Malformed Boundary: The researchers found that crafting a boundary parameter with a malformed quote (or "quarter" as per transcript, likely a typo for "quote" or "malformed token") caused the Cloudflare WAF to fail in correctly parsing the HTTP body. In contrast, most web APIs were able to correctly parse the body despite the non-standard boundary, allowing malicious payloads to pass through.
• Example 2: Fortinet WAF and PHP Tolerance: PHP-based APIs were found to be remarkably tolerant of malformed HTTP messages. An attacker could delete the closing boundary and form-data tokens within the Content-Disposition header. While the Fortinet WAF failed to parse this request correctly and thus missed the malicious payload, the PHP-based API still managed to extract the correct parameters. This specific tactic enabled attackers to launch SQL injection or command injection attacks against PHP-based web applications, bypassing the Fortinet WAF.

3. Feature Spot Gap:

This tactic exploits differences in support for deprecated, extended, or non-standard HTTP features between WAFs and web application frameworks.

• Example 1: Quoted-Printable Encoding and Go APIs: The Content-Transfer-Encoding: quoted-printable header is deprecated according to RFC 7878, which states that senders should not generate parts with this header. However, many Go-based APIs still support and decode this encoding. Attackers could encode their malicious payloads using quoted-printable encoding. Since WAFs often adhere to the latest RFCs and do not process this deprecated encoding, they would miss the payload, while the Go-based application would correctly decode and execute it, leading to SQL injection or command injection.
• Example 2: UTF-7 Charset and Django APIs: Django APIs were found to support the charset parameter in Content-Type headers, even when the Content-Type was application/x-www-form-urlencoded. Most WAFs, however, do not parse the charset parameter in this specific Content-Type context. This allowed attackers to encode their malicious payloads using UTF-7 encoding. The Microsoft Azure WAF, for instance, failed to correctly parse the UTF-7 encoded payload, while the Django application would decode the HTTP body using the specified charset parameter, enabling injection attacks.

These detailed examples demonstrate the practical exploitability of the discovered vulnerabilities and underscore the necessity for WAFs to meticulously align their parsing logic with the diverse and sometimes non-standard behaviors of real-world web application frameworks.

Defensive Implications

▶ Watch: WAF and WebAP validators for evasion detection (8:15)

The findings from WAF-Menace have profound implications for web application security, particularly for WAF vendors, implementers, and application developers. The existence of over 3,000 evasion cases across mainstream WAFs and web frameworks indicates a systemic problem that requires a multi-faceted defensive strategy.

1. WAF Parsing Logic Hardening: WAF vendors must urgently review and enhance their HTTP parsing engines. The primary takeaway is that WAFs need to parse HTTP requests exactly as the backend web application frameworks do, including their tolerance for malformed requests and support for deprecated or extended features. This requires continuous research and development to keep pace with the evolving and often ambiguous HTTP specifications and their diverse implementations. WAFs should not assume strict adherence to standards if popular frameworks deviate.
2. Comprehensive Fuzzing for WAFs: WAF vendors should adopt advanced fuzzing techniques, similar to WAF-Menace, as a standard part of their quality assurance and security testing pipelines. This includes grammar-based, coverage-guided fuzzing against a wide array of HTTP parsing scenarios, explicitly looking for discrepancies with how various web server and application frameworks (e.g., PHP, Python's Flask/Django, Go, Node.js) interpret requests.
3. Regular Updates and Rule Set Refinement: Beyond parsing logic, WAF rule sets need continuous refinement. While the problem lies at the parsing layer, the knowledge gained from these evasion tactics can inform more robust rules that anticipate and detect these protocol-level manipulations. WAFs should be capable of handling edge cases like multiple Content-Type headers or unexpected Content-Transfer-Encoding values.
4. Application-Layer Input Validation: Developers should not solely rely on WAFs for protection. Robust, context-aware input validation, sanitization, and output encoding at the application layer remain critical. If a malicious payload bypasses the WAF, strong application-layer defenses can still prevent exploitation. This includes validating Content-Type, charset, and other HTTP header parameters that could influence how the application processes the request body.
5. Awareness of Web Framework Parsing Peculiarities: Web application developers need to be acutely aware of how their chosen framework (e.g., PHP, Django, Flask, Go's HTTP server) handles non-standard, malformed, or deprecated HTTP features. Overly tolerant parsing behavior by frameworks, while sometimes intended for backward compatibility, can create dangerous blind spots when paired with WAFs that are less tolerant or interpret standards differently. Developers should prioritize patching or configuring frameworks to be less permissive where security is paramount.
6. Layered Security Approach: The findings reinforce the importance of a layered security architecture. A WAF is a critical layer, but it is not infallible. Combining WAFs with other security controls like API gateways, strong authentication/authorization, regular vulnerability scanning, and secure coding practices provides a more resilient defense posture.
7. Standardization and Clarity: The broader security community, including standards bodies, could benefit from clearer, more prescriptive guidelines on HTTP parsing behavior, especially for ambiguous or deprecated features, to minimize discrepancies between implementations.

In essence, the research serves as a stark reminder that the security of a WAF is only as strong as its understanding of the underlying protocol and its alignment with the applications it protects.

Key Takeaways

• Protocol-level WAF evasion is a widespread and critical vulnerability: WAFs often fail to protect against attacks when their HTTP parsing logic differs from that of the backend web application framework.
• Automated fuzzing is highly effective for discovery: The WAF-Menace framework demonstrates that grammar-based, coverage-guided fuzzing can systematically and efficiently uncover thousands of these elusive protocol-level bypasses.
• Three primary evasion tactics exist: Attackers exploit Malformed Parameter Structures, Parameter Type Confusion, and Feature Spot Gaps between WAFs and web applications.
• WAFs must align parsing with web frameworks: To be effective, WAFs need to precisely mimic the parsing behavior of diverse web application frameworks, including their tolerance for non-standard or deprecated HTTP features.
• Widespread vulnerabilities confirmed: WAF-Menace identified over 3,000 unique evasion samples across 40 WAFs and 20 popular web application frameworks, leading to confirmed fixes from multiple vendors.
• Defensive strategies must evolve: WAF vendors need to improve parsing robustness, and application developers must implement strong input validation, recognizing that WAFs are not a silver bullet.

About the Speaker(s)

The talk "Break the Wall from bottom: Automated Discovery of Protocol-Level Evasion Vulnerabilities in Web Application Firewalls" was presented by Qi Wang, who introduced himself as "G1 or you can call me AI." He delivered the presentation on behalf of a research team that includes Jianjun Chen, Zheyu Jiang, Run Guo, Ximeng Liu, and Chao Zhang. This collaborative effort from the researchers highlights their work in systematically identifying and understanding protocol-level WAF evasion techniques, contributing significant advancements to web security research.