AiIAM: Transforming the Democratized AWS IAM...

AiIAM: Transforming the Democratized AWS IAM...

Speakers: Anthony Scheller, Jorge L Gomez

Conference: BSidesSF 2024

YouTube: https://www.youtube.com/watch?v=B58hDAysxh4

Overview

This talk introduces Vapor Lock, an innovative open-source project formerly known as AiIAM, designed to tackle the pervasive challenge of managing Identity and Access Management (IAM) policies in large, democratized cloud environments. Presented by Anthony Scheller, Head of Security Engineering at StubHub, and Jorge L Gomez, Staff Security Engineer and Tech Lead at Twilio, Vapor Lock leverages large language models (LLMs) and control plane log analysis to address the principle of least privilege problem, particularly for service-to-service IAM policy creation. The core problem Vapor Lock aims to solve is the inherent complexity and error-proneness of crafting precise IAM policies, often leading to overly permissive configurations like "star for action, star for permission," which significantly increases an organization's attack surface.

The speakers highlight that while existing Cloud Security Posture Management (CSPM) tools are effective for reactive identification of overly permissive policies, there remains a critical gap in proactive, "shift-left" solutions that empower developers to create secure policies from the outset. Vapor Lock fills this void by offering a self-service platform that allows engineers to generate new, right-sized IAM policies based on natural language descriptions or to resize existing policies based on actual usage patterns derived from cloud logs. This approach not only enhances developer velocity by streamlining policy creation but also significantly improves an organization's security posture by enforcing the principle of least privilege, thereby reducing the risk of data breaches stemming from misconfigured access controls.

The presentation delves into the architectural components of Vapor Lock, its two primary workflows—LLM-driven policy generation and deterministic policy right-sizing—and demonstrates its user-friendly interface and API capabilities. By providing a practical, extensible, and open-source solution, Vapor Lock represents a significant step forward in transforming how organizations manage IAM at scale, moving towards a more secure and efficient cloud operational model.

Background

▶ Watch: Problem statement: Overly complex IAM policies leading to misconfigurations a... (02:15)

The landscape of cloud security is continuously evolving, with Identity and Access Management (IAM) standing as a foundational pillar. However, managing IAM at scale, particularly in dynamic cloud environments like AWS and GCP, presents significant challenges. Jorge Gomez recounts a common scenario where an audit revealed an IAM policy with "star for action, star for permission," a clear indicator of the complexity and frustration engineers face when trying to configure precise access controls. This complexity is exacerbated by the ever-growing number of cloud services, the intricate permutations of permissions, and the need to incorporate conditional statements while adhering to compliance requirements. Such misconfigurations are consistently cited as a top vector for data breaches, underscoring the critical need for better solutions.

Organizations typically adopt one of two strategies for IAM governance: centralized or democratized. In a centralized model, a dedicated team (often security or platform engineering) is responsible for reviewing and approving all IAM policies. While this approach ensures standardization and adherence to the principle of least privilege, it often becomes a bottleneck, significantly impeding developer velocity, especially in rapidly scaling organizations. Conversely, the democratized approach empowers individual engineering teams to build and own their services, including their IAM policies. This model boosts developer velocity but introduces inconsistencies in security baselines, as not all teams possess the same level of security maturity or discipline. Anthony Scheller notes that organizations poised for expansion invariably gravitate towards a democratized model, making it imperative for security practitioners to architect controls that accommodate this decentralized reality.

Mature cloud security postures in democratized environments typically incorporate several key processes and technologies:

• Infrastructure as Code (IaC) Pipelines: Moving all infrastructure management through code (e.g., Terraform) ensures repeatable, maintainable infrastructure and allows security teams to embed "security by default" controls into reusable templates.
• Policy as Code (PaC) Checks: Tools like HashiCorp Sentinel, CFGuard, Conftest, and OPA are integrated into IaC pipelines to validate configurations against predefined security criteria. For instance, an IAM role module might have explicit permission boundaries and denies, but developers can still create their own policies within these guardrails, subject to PaC checks.
• Organization/Tenant Layer IAM Protection: Services like AWS Service Control Policies (SCPs) allow organizations to define deny policies at a higher level, ensuring that no matter what permissions are granted in child accounts, certain actions are universally prohibited.
• Routine Permission Audits: Security teams conduct regular audits using cloud service provider control plane analysis (e.g., CloudTrail logs) to identify overly permissive or unused permissions.
• Developer Education: In a decentralized model, educating developers on the gravity of their configurations and the importance of secure infrastructure is paramount.
• SaaS Cloud Security Posture Management (CSPM) Tools: Products like Wiz, Orca, and Prisma Cloud are widely used to identify common misconfigurations (e.g., externally exposed S3 buckets, open EC2 security groups). More recently, these tools have added functionality to detect overly permissive IAM permissions based on usage patterns or identify unused IAM roles and policies.

While CSPM tools are invaluable for a reactive approach—identifying issues after infrastructure and policies are deployed—Scheller and Gomez identified a crucial opportunity for a shift-left approach. They argue that there's a need to enable developers to generate better IAM policies proactively, before deployment, rather than solely relying on post-deployment detection. This realization formed the genesis of Vapor Lock, aiming to complement existing CSPM solutions by providing a self-service mechanism for developers to create least-privilege policies from the start.

Key Findings

▶ Watch: Introduction of Vapor Lock as a 'shift-left' solution for proactive IAM polic... (10:15)

Vapor Lock's development yielded several significant findings and contributions to the field of cloud IAM security:

• Effective Hybrid Approach to Least Privilege: The project successfully demonstrates a powerful hybrid strategy for achieving the principle of least privilege. It combines the generative capabilities of Large Language Models (LLMs) for proactive policy creation with the deterministic accuracy of control plane log analysis for reactive policy right-sizing. This dual approach addresses both the initial policy generation challenge and the ongoing maintenance of least privilege.
• LLMs for Proactive Policy Generation: A key finding is the viability of using LLMs to generate new, least-privilege IAM policies from natural language descriptions. This "shift-left" capability empowers developers to quickly obtain functional and secure policies without deep IAM expertise, directly addressing the "star for action, star for permission" problem by making precise policy creation accessible.
• Deterministic Right-Sizing via CloudTrail Analysis: Vapor Lock confirms the effectiveness of leveraging AWS IAM Access Analyzer in conjunction with CloudTrail logs for deterministic right-sizing of existing policies. By analyzing actual usage over a specified period (e.g., 90 days), the tool can accurately identify and remove unused permissions, ensuring policies reflect only necessary access.
• Mitigation of LLM Hallucinations: The integration of AWS IAM Access Analyzer into the LLM-driven workflow to validate generated permissions is a critical finding. This step effectively mitigates the risk of LLM "hallucinations" (generating non-existent or invalid permissions), ensuring that the policies returned to users are not only least-privilege but also functionally correct and valid within the AWS ecosystem.
• Developer Empowerment and Velocity: By providing a self-service platform with both a user-friendly UI and a robust API, Vapor Lock demonstrates that security can be integrated into the development workflow without sacrificing developer velocity. Engineers can generate or resize policies quickly, fostering greater adoption of security best practices.
• Extensible and Open-Source Design: The architecture of Vapor Lock is designed for extensibility, allowing for the relatively low-effort addition of new cloud services and, in the future, support for other cloud providers (GCP, Azure) and even Kubernetes RBAC. Its open-source nature encourages community involvement and continuous improvement.
• Anonymization for LLM Interaction: The practice of sending dummy ARNs to the LLM instead of actual resource identifiers highlights a practical security measure for interacting with external AI services, preventing the inadvertent exposure of sensitive infrastructure details.

These findings collectively underscore Vapor Lock's potential to significantly enhance cloud security postures by making least-privilege IAM policy management more automated, accessible, and reliable for development teams.

Technical Deep Dive

▶ Watch: Demonstration of LLM-based policy creation using natural language descriptors... (14:00)

Vapor Lock is engineered as a cloud-native solution comprising several interconnected components, designed for deployment on container orchestration platforms like Kubernetes via Helm or Docker Compose. The architecture is modular, facilitating both user interaction and complex asynchronous processing.

At its core, Vapor Lock consists of:

• Vapor Lock Frontend: Built with Next.js, this component provides the user interface (UI) and acts as an API router, offering a developer-friendly dashboard.
• Vapor Lock Backend API: This is the orchestrator, responsible for managing the various processes. It is built using a framework that supports Swagger documentation, indicating a robust API design (likely FastAPI as mentioned in the demo).
• Asynchronous Job Processing: For tasks that require longer execution times, Vapor Lock employs Celery as a distributed task queue with a worker node, and Redis serves as the state manager backend for Celery, storing job status and results.

The system supports two primary workflows: Create Policy (LLM-driven generation) and Resize Policy (deterministic right-sizing).

Create Policy Workflow (LLM-driven Generation)

This workflow focuses on proactively generating new IAM policies based on natural language descriptions, leveraging the power of large language models.

1. Client Request: A client (either through the Next.js UI or directly via the API) makes a request to the Vapor Lock API. The request includes parameters such as the desired policy name, the source AWS service (e.g., Lambda, EC2), destination AWS services (e.g., SNS topic), natural language permissions (e.g., "read," "describe," "write," "admin"), an optional Amazon Resource Name (ARN) for precision, and the desired output format (JSON, HCL/Terraform, CloudFormation).
2. Prompt Interpolation: The Vapor Lock Backend API interpolates these parameters into a structured prompt suitable for an LLM.
3. LLM Interaction: This prompt is then sent to OpenAI, specifically utilizing the GPT 3.5 Turbo model. A crucial security measure here is that Vapor Lock sends a dummy version of the ARN to the LLM, crafting it in the same format but without exposing the actual sensitive resource identifier.
4. Policy Generation: OpenAI returns a generated IAM policy document.
5. Validation and Scrubbing: To address potential LLM "hallucinations" (where the model might generate non-existent or invalid permissions), Vapor Lock sends the generated policy document to AWS IAM Access Analyzer. Access Analyzer validates the permissions against actual AWS service definitions. If any permissions are found to be invalid or non-existent, Vapor Lock's backend scrubs these particular actions out of the policy.
6. Formatting and Response: The validated policy undergoes additional processing, is formatted into the client's desired output (JSON, HCL for Terraform, or YAML/JSON for CloudFormation), and then sent back to the client.

Resize Policy Workflow (Deterministic Right-Sizing)

This workflow focuses on reactively right-sizing existing IAM policies by analyzing actual usage patterns, ensuring adherence to the principle of least privilege.

1. Client Request: A client (UI or API) sends a request to the Vapor Lock API, providing the principal ARN of the IAM entity to be analyzed and the desired output format.
2. Asynchronous Job Registration: The Vapor Lock API forwards this request to the Celery worker node. The worker registers the job and immediately returns a unique request ID to the client. This allows the client to continue polling for the job's status without blocking.
3. CloudTrail Log Analysis: The Celery worker initiates a request to AWS IAM Access Analyzer. Access Analyzer then queries CloudTrail logs for the specified principal over the last 90 days. It identifies all permissions that the principal has effectively used during this period.
4. Policy Refinement: Access Analyzer determines which permissions were not used and effectively strips them away, returning a new, right-sized IAM policy that includes only the permissions that have been actively utilized.
5. State Management: Once Access Analyzer completes its analysis and returns the refined policy, the Celery worker stores this policy in the Redis cache, associated with the unique request ID.
6. Polling and Retrieval: The client, using the previously received request ID, continuously polls a third API endpoint (get_resize_permission). The Vapor Lock API retrieves the policy from Redis once it's available, formats it, and sends it back to the client.

Supported Output Formats

Vapor Lock provides flexibility in how policies are returned:

• JSON: The default and most common format for AWS IAM policies.
• HCL (Terraform): For seamless integration with Infrastructure as Code workflows using Terraform.
• CloudFormation: For users preferring AWS's native infrastructure templating service.

Extensibility

A core design principle for Vapor Lock is extensibility. The architecture is built to allow for relatively low-effort additions of new services to its IAM service catalog. This foresight aims to accommodate the ever-expanding suite of cloud services and facilitate community contributions.

Demo / Proof of Concept

▶ Watch: Demonstration of deterministic policy right-sizing by analyzing CloudTrail lo... (15:30)

The demonstration of Vapor Lock showcased its functionality through both its web-based user interface (UI) and its programmatic API, accessible via Swagger documentation. The goal was to highlight the ease of use and the practical application of its policy generation and right-sizing capabilities.

The UI, built with Next.js, presents a clean dashboard with two main options: "Create Policy" and "Resize Policy."

1. Create Policy (LLM-driven Generation):

• Users navigate to the "Create Policy" section, which presents a web form.
• Input Fields:
• Policy Name: A user-defined name for the new policy.
• Source Service: A dropdown allows selection of the AWS service that will act as the principal (e.g., Lambda, EC2).
• Natural Language Descriptors: Users can select desired permissions using intuitive terms like "list," "subscribe," "read," "describe," "write," or "admin." The system determines the underlying AWS IAM actions.
• Destination Services: Users can add multiple destination AWS services (e.g., an SNS topic) that the source service needs to interact with.
• ARN Toggle: An option to provide a specific Amazon Resource Name (ARN) for the destination resource. If no ARN is supplied, Vapor Lock uses the service name and provides an ARN placeholder in the output. If an ARN is provided, the system extrapolates the service names from it.
• Generation: After filling in the details, clicking the "Generate Policy" button triggers the LLM workflow.
• Output: The generated least-privilege policy is displayed, with options to view it in JSON, Terraform (HCL), or CloudFormation (YAML) formats. The demo specifically showed an example of a Lambda function needing "list" and "subscribe" permissions against an SNS topic.

2. Resize Policy (Deterministic Right-Sizing):

• Users navigate to the "Resize Policy" section, which features a simpler form.
• Input Field:
• Principal ARN: The user provides only the ARN of an existing AWS IAM principal (e.g., an IAM role or user). The speaker explicitly mentioned that the AWS account ARN used in the demo no longer exists, for security reasons.
• Generation: Clicking "Generate Policy" initiates the deterministic workflow, querying AWS IAM Access Analyzer and CloudTrail logs.
• Output: The right-sized policy, containing only the permissions actually used by the principal over the last 90 days, is returned. Like the "Create Policy" output, it can be viewed in JSON, Terraform, or CloudFormation.
• User Experience: The demo also highlighted the inclusion of light and dark themes, emphasizing the focus on developer experience even for an MVP.

API Walkthrough (Swagger Docs):

The backend API, built with a framework that generates Swagger documentation, provides programmatic access to Vapor Lock's functionalities.

• create_policy Endpoint (POST): This endpoint mirrors the "Create Policy" UI form. It accepts a JSON payload specifying the source AWS service, destination AWS service or ARN, desired actions, and output format. It returns the generated IAM policy.
• request_right_size_permissions Endpoint (POST): This endpoint corresponds to the "Resize Policy" functionality. It takes the principal ARN and output format. Due to the asynchronous nature of CloudTrail log analysis, it immediately returns a request ID.
• get_resize_permission Endpoint (GET): This endpoint is used to poll for the completion of a right-sizing job. Users provide the request ID, and once the job is complete, the endpoint returns the right-sized IAM policy in the specified format.

The demo effectively illustrated Vapor Lock's ability to simplify complex IAM policy management, providing both a user-friendly interface for quick policy generation and a robust API for integration into automated CI/CD pipelines.

Defensive Implications

▶ Watch: Future plans: Reinforcement Learning from Human Feedback (RLHF) for LLM fine-... (23:30)

Vapor Lock offers several significant defensive implications for organizations striving to enhance their cloud security posture and adhere to the principle of least privilege:

• Proactive Reduction of Attack Surface: By enabling developers to generate least-privilege IAM policies from the outset, Vapor Lock implements a crucial "shift-left" security strategy. This proactively reduces the attack surface by preventing the deployment of overly permissive policies, which are a primary vector for data breaches.
• Mitigation of Misconfigurations: The tool directly addresses the common problem of overly permissive IAM policies (e.g., "star for action, star for permission") by automating the creation of precise, context-aware policies. This significantly minimizes human error and the complexity associated with manual policy crafting, leading to fewer misconfigurations.
• Enhanced Compliance and Governance: Adhering to the principle of least privilege is a fundamental requirement for many regulatory compliance frameworks. Vapor Lock facilitates this by providing mechanisms to ensure that IAM entities only have the permissions necessary for their intended function, thereby strengthening an organization's compliance posture.
• Improved Developer Velocity with Security: Instead of acting as a bottleneck, Vapor Lock empowers developers to create secure infrastructure quickly and independently. By offering a self-service platform and API, it integrates security into the development workflow, fostering a culture where security is an enabler rather than an impediment to innovation.
• Reactive Remediation and Optimization: The "Resize Policy" feature provides a powerful reactive capability. It allows security teams and developers to audit existing policies against actual usage patterns from CloudTrail logs. This helps identify and remediate overly permissive policies in deployed environments, continuously optimizing the security posture of cloud resources.
• Robustness Against LLM Hallucinations: The integration of AWS IAM Access Analyzer to validate LLM-generated policies is a critical defensive measure. This step ensures that even though an LLM generates the initial policy, any non-existent or invalid permissions are scrubbed, preventing the deployment of non-functional or potentially exploitable access controls.
• Data Anonymization for External AI: The practice of sending dummy ARNs to the LLM for policy generation is a prudent security measure. It prevents sensitive organizational resource identifiers from being exposed to external AI services, mitigating potential data leakage risks associated with third-party model interactions.
• Standardization and Auditability: By supporting output in formats like JSON, Terraform HCL, and CloudFormation, Vapor Lock promotes the standardization of IAM policy definitions. This standardization, especially when integrated into IaC pipelines, improves the auditability and maintainability of access controls across the cloud environment.
• Complementary to CSPM Tools: Vapor Lock is designed to complement existing Cloud Security Posture Management (CSPM) tools rather than replace them. While CSPMs excel at reactive detection, Vapor Lock provides the proactive "shift-left" capability, creating a more comprehensive defense-in-depth strategy for IAM.

Key Takeaways

• Addressing IAM Complexity: Vapor Lock directly tackles the significant challenge of creating and managing least-privilege IAM policies in complex, democratized cloud environments, aiming to eliminate overly permissive configurations.
• Hybrid Approach to Policy Management: The tool uniquely combines proactive, LLM-based policy generation from natural language descriptions for new services with reactive, deterministic analysis of CloudTrail logs for right-sizing existing policies.
• Shift-Left Security Empowerment: Vapor Lock enables a crucial "shift-left" in cloud security by providing developers with a self-service platform to generate secure, least-privilege policies before deployment, complementing existing reactive CSPM solutions.
• Developer-Friendly Design: With both an intuitive Next.js UI and a robust API (documented via Swagger), Vapor Lock prioritizes developer experience, facilitating easy adoption and integration into CI/CD pipelines.
• Robust and Extensible Architecture: The system leverages a modern cloud-native stack including Next.js, Fast API, Celery, and Redis, and integrates with AWS IAM Access Analyzer and OpenAI (GPT 3.5 Turbo) for core functionalities, with a design focused on future extensibility.
• Future Enhancements: Planned improvements include implementing reinforcement learning from human feedback for LLM fine-tuning, adding authentication/authorization mechanisms, supporting alternative LLMs, and expanding support to other cloud providers (GCP, Azure) and Kubernetes RBAC.

About the Speaker(s)

Anthony Scheller is the Head of Security Engineering at StubHub. With over a decade of experience in security engineering, Anthony began his career in offensive security consulting at PWC. He later moved to Hulu, where he was instrumental in building out their cloud and network security engineering function. More recently, he worked alongside Jorge Gomez at Twilio as a Staff Engineer on the cloud security team.

Jorge L Gomez is a Staff Security Engineer and Tech Lead in Twilio's Cloud Security Engineering department. He has accumulated 15 years of experience operating as a security engineer or architect, primarily across diverse industries including energy, tech, and cloud infrastructure. Jorge met Anthony about a year prior to this talk, and their shared interest in solving the complexities of overly permissive IAM policies in cloud providers like AWS and GCP led to the brainstorming and development of Vapor Lock.