Heard you liked access, so we built Access to...

Heard you liked access, so we built Access to...

Speakers: Peter Collins, Elisa Guerrant

Conference: BSidesSF 2024

YouTube: https://www.youtube.com/watch?v=g4JZtHNfau8

Overview

This talk, "Heard you liked access, so we built Access to...", presented by Peter Collins and Elisa Guerrant from Discord, details their journey in building a new internal access control system for Discord employees. The presentation outlines the critical need for such a system, the guiding principles behind its design, the technical architecture, and its impact on Discord's security posture and operational efficiency. The system, aptly named Access, aims to provide an intuitive, transparent, centralized, and secure platform for managing employee access to various internal resources and applications.

The motivation for developing Access stems from the pervasive issue of human-related factors in data breaches. Citing Verizon's 2023 Data Breach Investigations Report, the speakers highlight that 74% of all breaches involve the human element, with privileged misuse and stolen user credentials being primary threats. Discord's initiative to build Access is a strategic response to these challenges, designed to enforce least privilege, enhance auditability, and streamline access management processes, ultimately contributing to a more mature security and business environment.

Beyond merely addressing security vulnerabilities, Access was designed with a strong emphasis on improving business velocity and fostering a positive security culture. By empowering employees and managers with self-service capabilities and transparent access information, the system reduces reliance on centralized IT or security teams for routine access requests, thereby unblocking individuals to perform their jobs more quickly and securely. The talk culminates with the announcement that Access has been open-sourced, inviting community contributions and offering a robust solution for other organizations facing similar access management complexities.

Background

▶ Watch: Introduction of top-level Role-Based Access Control (RBAC) as a core design p... (05:00)

The necessity for a robust internal access control system at Discord was underscored by industry statistics, particularly the Verizon 2023 Data Breach Investigations Report, which identified the human element in 74% of all breaches, with privileged misuse and stolen user credentials as significant contributors. Recognizing this, Discord embarked on a project to develop a system that would not only mitigate these risks but also align with their broader security and business objectives.

The guiding principles for Discord's internal access control system were meticulously defined to ensure a comprehensive and effective solution:

• Intuitive: The system needed to be user-friendly, understandable, and accessible to all employees, not just technical staff. This meant providing a web user interface for easily viewing current and historical permissions, and simplifying the process for requesting additional access, avoiding reliance on command-line utilities or GitOps flows.
• Transparent and Discoverable: A core belief was that empowering employees to solve their own permission issues safely would increase business velocity and security. The system aimed to allow individuals to answer questions like "What permissions do I have?", "What permissions does my teammate have?", "Did I recently lose access?", and "How do I request access?". Managers also needed capabilities to easily grant temporary or new team member access.
• Centralized: The goal was to create a "One-Stop Shop" to replace disparate, out-of-band access request mechanisms such as IT helpdesk tickets, chat channels, or emails, which are difficult to audit. The system needed to integrate seamlessly with Discord's identity provider, Okta, for single sign-on and authorization, leveraging network effects to become more useful with increased adoption.
• Secure: Security was paramount, focusing on least privilege and auditability. This involved implementing Role-Based Access Control (RBAC) to provide standard sets of permissions for common workflows, enforcing time-bounded expiring access to automatically remove unused permissions, and maintaining comprehensive audit trails for current and historical permissions to meet and exceed compliance objectives.

A key architectural decision was the adoption of Role-Based Access Control (RBAC). This approach manages permissions based on users' shared job functions rather than individual teams or other criteria, making it clearer why someone has access to specific resources and simplifying onboarding, project transfers, and team changes. For instance, an "Engineer" role would grant access to GitHub and engineering documentation, while a "Manager" role would grant access to a specific Google Group for managers, with both roles inheriting general "All Employees" access to resources like email.

Discord also aimed for top-level RBAC, where roles are managed at the identity provider level and propagate across various services. While generic nested groups could achieve this, they often complicate management and decrease transparency. The team sought a more intuitive and manageable solution.

The decision to build a new tool rather than buy an existing SaaS or open-source solution was made after a thorough evaluation of the market. While many existing solutions offered compatibility with Okta and other identity providers (like LDAP or Entra ID), and some even supported time-bounded expiring access, significant deficiencies were identified:

• Few provided an intuitive employee-facing view for self-service access management, often limiting such capabilities to administrator panels.
• Most lacked robust support for top-level RBAC, requiring extensive manual management of users into individual groups for different applications.
• Many struggled with delegating administration of groups in an opinionated and intuitive manner, often centralizing control with IT or security teams.
• While some supported routing access requests, the mechanisms were often not standardized or intuitive for regular employees.
• Integration with Discord's internal chat solution for notifications was also a specific requirement that off-the-shelf products did not natively support, though custom integrations were often possible.

Ultimately, no single existing solution checked all of Discord's highly valued goals, particularly around top-level RBAC, employee transparency and discoverability, and opinionated delegated administration with intuitive access request routing. This led Discord to conclude that building their own system, Access, was the most effective path to meet their specific needs and accelerate business operations.

Key Findings

▶ Watch: Explanation of the core group types: Vanilla, Role, and App Groups, forming t... (11:00)

The development and internal deployment of Access at Discord yielded several significant findings and contributions:

• Successful Internal Deployment: Access has been running internally at Discord for over a year, demonstrating its stability and effectiveness in a real-world, large-scale environment.
• Improved Operational Efficiency: The system dramatically reduced the time required for user access reviews from days to hours, significantly cutting down on back-and-forth communication and manual spreadsheet management.
• Centralized Access Management: Access successfully centralized all access requests, eliminating the previous fragmented and out-of-band methods (IT helpdesk, chat channels, emails). This streamlining has improved auditability and consistency.
• Reduced "Confused Deputy" Problem: By routing access requests to the most appropriate individuals—such as team leads for role groups or service owners for application groups—the system ensures that approvals are made by those with the most context. This has led to less "rubber stamping" and more informed decisions, reducing the risk associated with administrators approving requests without full understanding.
• Enhanced Security Posture: Access facilitated the reduction of highly privileged access permissions. For sensitive and critical groups, it became easier to audit justifications for access and enforce time limits, preventing standing access where not required and promoting a culture of least privilege.
• Open-Source Contribution: Discord has open-sourced Access, making its codebase available to the wider security community. This allows other organizations to leverage Discord's work, contribute to its development, and adapt it to their specific environments, fostering a collaborative approach to internal access management challenges.
• Extensible Architecture: The system was designed with a plugin architecture for notifications and conditional access request approvals, demonstrating its flexibility and adaptability to various organizational needs and third-party integrations beyond Discord's specific environment.

Technical Deep Dive

▶ Watch: Discussion of delegated group/app ownership and how it addresses the "confuse... (13:00)

Access is built upon a robust technical foundation designed for scalability, flexibility, and ease of use. At its core, it's a Python Flask application for the backend, paired with a React, TypeScript, and Material UI frontend, forming a single-page application with a REST API. For data persistence, PostgreSQL is used in production, with SQLite serving development environments. The system is packaged as a Docker container, enabling deployment on platforms like Kubernetes (Discord's choice) or other container orchestration systems.

The system's foundational elements include:

• Users: Representing employees, each user profile displays details like title and reporting structure, along with their group memberships and ownerships.
• Groups: Access defines three types of groups:
• Vanilla Standalone Groups: Used primarily during the initial migration phase, offering basic group functionality.
• Role Groups: These are central to the RBAC model, representing job functions (e.g., "Software Engineer," "Data Scientist"). They can be members or owners of other groups, abstracting permissions based on an individual's role.
• App Groups: These map directly to specific permissions within an application or service. An app group can be tied to any Okta compatible third-party SaaS application or first-party service, allowing centralized management of diverse application permissions.
• Apps: An "app" in Access is essentially a collection of role groups or app groups, representing a specific application or service whose access is managed by the system.

Access implements several unique features to achieve its goals:

• Direct User Assignments vs. Role Assignments: While direct assignment of users to groups is supported (primarily for initial migration), the system strongly encourages role-based access control. Users are assigned to role groups, and these role groups are then assigned to app groups, inheriting permissions. This ensures that permissions are managed based on job function rather than individual users, simplifying management and promoting consistency.
• Access Requests: Employees can request access to any group (as a member or owner) for a specified duration, providing a mandatory reason. This centralized request mechanism replaces ad-hoc processes.
• Group and App Ownership: A critical feature is the delegation of ownership for groups and apps. Owners can manage membership, ownership, and metadata (title, description). Crucially, they are responsible for approving or denying access requests. This design directly combats the "confused deputy" problem by ensuring that approval decisions are made by individuals with the most context about the group's purpose and the requester's need, leading to faster resolution times and more secure outcomes.
• Time-Bounded Access: Access enforces time limits on access grants, automatically revoking permissions after a set duration. This is invaluable for compliance (e.g., periodic access reviews every 90 days) and for reducing the accumulation of unused permissions over time, ensuring that employees' access is "right-sized" to their current job functions.
• Audit Views: The system provides comprehensive, company-wide audit views showing historical permissions for users, groups, and roles. These views are transparent and discoverable by anyone, aiding in troubleshooting historical access changes and fulfilling record-keeping requirements. Details such as membership justification, access duration, and the identity of the adder/remover are all recorded.
• Group and App Tagging: Tags can be applied to groups and apps to label similar entities and, more powerfully, to enforce constraints. These constraints can include requiring a membership justification, setting a default membership time limit, or even preventing owners from managing their own access for highly sensitive groups. Tags applied to an app automatically propagate to its constituent app groups, though individual groups can also be tagged.
• Expiring Access & Bulk Renewal: To address the challenge of managing time-bounded access at scale, Access offers a centralized page displaying all expiring access across the company. Owners can filter this view to see expiring access for groups they manage and perform bulk renewals, while individuals can view their own expiring access and initiate new requests directly from the page. This feature significantly reduces the likelihood of business disruptions due due to unexpected access loss.
• Notifications: A flexible plugin system handles notifications, with Discord's internal implementation leveraging their own platform. This system alerts owners of new access requests, notifies requesters of decisions, and provides timely warnings to both owners and individuals about expiring access. The plugin architecture allows for compatibility with other chat solutions, email, or SMS.

Under the hood, Access integrates tightly with Okta. Group changes (adding/removing members) are synced with Okta on demand for immediate actions. Additionally, a periodic cron job runs every 15 minutes to sync the state from Access to Okta, ensuring consistency, resolving potential rate limit issues, and enforcing expiring access revocations.

The system's extensibility is a key design principle. Beyond the notification plugin, Access introduces a new plugin interface for conditionally approving or denying access requests automatically based on custom Python logic. This allows for advanced automation, such as calling out to other services for policy checks or hardcoding specific groups/roles for automatic approval under certain conditions.

Demo / Proof of Concept

▶ Watch: Overview of the technical stack (Python Flask, React, Postgres, Docker/K8s) a... (21:00)

While the talk did not feature a live, interactive demonstration, the speakers effectively conveyed the functionality and user experience of Access through a series of detailed screenshots and conceptual diagrams. These visual aids illustrated key aspects of the system, including:

• A user's profile page, showing their title, reporting structure, and group memberships.
• The RBAC diagram, clarifying the relationship between users, roles, and resources.
• An example of an access request form, highlighting the fields for duration and justification.
• A user audit view, demonstrating the historical record of access changes, including who added/removed access, the justification, and duration.
• A tag page, showing applied constraints and the groups/apps associated with a tag.
• The expiring access page for owners, illustrating how they can view and bulk renew access for multiple users.
• Examples of Discord notifications for access requests and expiring access.

The speakers also mentioned that a blog post accompanying the open-source release of Access includes screen recordings, which provide a more dynamic view of the application in action. This combination of static screenshots and external video resources serves as the primary proof of concept for the system's capabilities.

Defensive Implications

▶ Watch: Q&A discussion on extending the system to support other Identity Providers (I... (25:00)

The implementation of Access at Discord carries significant defensive implications, enhancing the organization's security posture across multiple vectors:

• Reduced Attack Surface from Human Element: By directly addressing the "human element" in breaches (74% according to Verizon's 2023 DBIR), Access minimizes risks associated with privileged misuse and stolen credentials. Enforcing least privilege and time-bounded access reduces the window of opportunity for attackers to exploit compromised accounts.
• Stronger Least Privilege Enforcement: The RBAC model, coupled with time-bounded expiring access, ensures that users only have the permissions necessary for their current job functions. Automatic revocation of unused permissions prevents privilege creep, a common issue where users accumulate excessive access over time, making them high-value targets.
• Enhanced Auditability and Compliance: Comprehensive audit views provide an immutable record of all access changes, including justifications, durations, and responsible parties. This transparency is crucial for meeting and exceeding compliance objectives (e.g., SOC 2, ISO 27001) and significantly simplifies internal and external audits. Periodic access reviews, implicitly enforced by time-bounded access, further strengthen compliance.
• Mitigation of "Confused Deputy" Risk: Delegating access approval to team leads and service owners, who possess the most context about specific groups and applications, drastically reduces the likelihood of inappropriate access grants. This targeted approval process ensures that access decisions are informed and aligned with business needs, preventing the "rubber stamping" that can lead to over-privileged accounts.
• Centralized Control and Visibility: Consolidating all access requests and management into a single system eliminates shadow IT and unmanaged access pathways. This centralization provides security teams with a holistic view of access across the organization, enabling better policy enforcement and risk assessment.
• Proactive Security Posture: Notifications for expiring access proactively alert users and owners, preventing accidental loss of critical access and encouraging timely reviews. This reduces business disruptions and ensures that security controls are actively maintained rather than reactively addressed after an incident.
• Scalable Security Culture: By making the "secure way the easy way," Access fosters a security-conscious culture. Its intuitive design and transparent nature empower employees to manage their own access securely, reducing friction and promoting a positive relationship between security and productivity.
• Reduced Risk of Lateral Movement: By limiting standing access to sensitive resources and enforcing time-bound permissions, Access makes it harder for an attacker who has compromised an initial account to move laterally within the network, as their access to other systems would likely be temporary or non-existent.
• Extensibility for Future Security Needs: The plugin architecture for conditional approvals allows organizations to integrate custom security logic, such as calling out to other services for real-time risk assessments or enforcing specific policies based on device state or other attributes, further strengthening defensive capabilities.

Key Takeaways

• Human Element is Critical in Breaches: A significant majority (74%) of data breaches involve human factors like privileged misuse and stolen credentials, underscoring the urgent need for robust internal access control systems.
• Core Principles for Access Control: Effective access management systems must be intuitive, transparent, centralized, and secure, balancing ease of use with strong security practices like least privilege and comprehensive auditability.
• The Power of Top-Level RBAC: Implementing Role-Based Access Control (RBAC), especially managed at a top level (e.g., via the Identity Provider), simplifies access management, enhances clarity, and streamlines onboarding and team changes across an organization.
• Strategic Build vs. Buy Decisions: Existing SaaS and open-source solutions often fall short on critical features such as employee self-service views, opinionated delegated administration, and true top-level RBAC, justifying custom development for organizations with specific, high-value requirements.
• Essential Features for Modern Access Management: Key functionalities like time-bounded access, delegated group/app ownership (to combat the "confused deputy" problem), comprehensive audit views, and a flexible notification system are crucial for both security efficacy and user adoption.
• Open-Source for Community Benefit: Discord's decision to open-source Access provides a valuable framework and solution for other organizations grappling with similar internal access management challenges, fostering collaboration and shared security improvements.

About the Speaker(s)

Peter Collins is a Senior Staff Security Engineer at Discord, where he focuses on cloud security and identity and access management within the platform security team. His professional philosophy centers on developing software that makes the secure path the easiest path for users. As a senior staff engineer, Peter strives to cultivate a security culture that is transparent, scalable, empathetic, and positive.

Elisa Guerrant is a Security Engineer at Discord, working alongside Peter on the platform security engineering team. Her work has primarily involved malware prevention and access management. Elisa's professional goals are to enhance the intuitiveness of security practices and to make computers and applications safer for all users.