Securing Citizen Developers: A New Opportunity to Build Safe Applications

Securing Citizen Developers: A New Opportunity to Build Safe Applications

Speakers: Kayla Underoffer, Lead Security Engineer, CTO Office, Zenity

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=3KVKEYqHUcQ

Overview

In "Securing Citizen Developers: A New Opportunity to Build Safe Applications," Kayla Underoffer, Lead Security Engineer in the CTO office at Zenity, addresses a burgeoning and often overlooked security challenge: the rapid proliferation of applications built by non-IT professionals using low-code/no-code (LCNC) platforms. This talk posits that while traditional "shift left" security initiatives have struggled to achieve their full potential within professional development teams, the rise of citizen development presents a "new hope" for integrating security earlier and more effectively into the application lifecycle. Underoffer highlights the unique risks inherent in this paradigm and, crucially, outlines a pragmatic, platform-centric strategy for mitigating them.

The presentation delves into the specific vulnerabilities introduced when business users, often experts in their functional domains but not in software development or security, construct critical applications that interact with sensitive business data. It provides concrete examples of common security misconfigurations in LCNC environments, ranging from authorization misuse to the mishandling of secrets and the emergence of novel AI-specific runtime risks. Ultimately, Underoffer argues that the contained nature and inherent automation capabilities of LCNC platforms offer a distinct advantage for implementing holistic security controls, enabling organizations to achieve significant risk reduction even with limited resources.

This talk is highly relevant for security professionals, IT leaders, and business stakeholders grappling with the expansion of shadow IT and the increasing reliance on LCNC solutions. It provides a foundational understanding of the risks and a actionable framework for establishing effective security governance in these rapidly evolving ecosystems. By embracing the "new hope" of citizen development, organizations can transform a potential security blind spot into an opportunity for building inherently more secure applications from the ground up.

Background

▶ Watch: Introduction and talk agenda (0:00)

The journey to securing citizen development applications begins with a critical look at the historical challenges faced by shift left security initiatives in traditional software development. As defined by Underoffer, shift left aims to identify vulnerabilities as early as possible in the software development lifecycle (SDLC) to reduce security debt in production. Despite this clear goal, two primary strategies—embedding security engineers directly into development teams or assigning security responsibilities to developers—have not consistently yielded desired outcomes.

Underoffer attributes these struggles to two main themes: a mismatch of expectations and an overwhelming volume of vulnerabilities. Security teams often fail to integrate security metrics into developer performance, leading to a disconnect between security goals and development incentives. Furthermore, the sheer volume of identified vulnerabilities, often numbering in the millions, coupled with a high rate of perceived false positives (e.g., 99 million false positives out of 100 million vulnerabilities from a developer's perspective), overwhelms both security and development teams. With limited security personnel (e.g., five security team members) acting as blockers to faster code shipments, shift left has frequently become an adversarial rather than collaborative process.

This context sets the stage for citizen development, which introduces a new audience and technology but with the same overarching goal: creating more secure applications. Citizen development, a term popularized by Microsoft, involves non-IT trained employees building production-level software using low-code/no-code (LCNC) platforms. These platforms provide "building blocks" or "Legos" that allow business users—experts in finance, HR, or marketing, but not IT or security—to create complex automations, workflows, customer-facing applications, and even AI agents.

The scale of this phenomenon is staggering. Zenity's 2024 enterprise state of low-code/no-code report found that, on average, organizations are seeing over 79,000 applications being built by business users in these environments. These applications inherently interact with critical business data, making them prime targets for exploitation. Crucially, the citizen development "SDLC" is fundamentally different from traditional models. Business users "envision and create" without formal processes, iteration, or, most significantly, the involvement of IT or security stakeholders. This lack of traditional governance and the rapid, unconstrained deployment of data-rich applications create a vast and often invisible attack surface, which Underoffer refers to as "pure shadow land."

Key Findings

▶ Watch: Introducing citizen development: a new hope for security (4:00)

The core findings of this talk revolve around the specific security risks introduced by citizen development and the unique opportunities for mitigation within LCNC environments. Underoffer categorizes these risks into build-time risks and runtime risks, emphasizing that choices made during the build phase directly influence the severity of runtime vulnerabilities.

From Zenity's 2024 report, the top three build-time risks identified in citizen development environments are:

1. Authorization Misuse: This occurs when an application or agent is configured to operate with excessive privileges, often inheriting the permissions of its creator, leading to privilege escalation or unintended access for other users.
2. Authentication Failures: Applications are deployed without proper authentication mechanisms, allowing unauthenticated or publicly accessible interaction with sensitive data or functions.
3. Data and Secrets Handling: Hard-coding sensitive information like API keys or credentials directly into application logic, a common anti-pattern in traditional development that becomes even more prevalent and challenging to address with non-technical builders.

Beyond these foundational application security risks, the emergence of AI agents within LCNC platforms introduces a new class of runtime risks:

• Prompt Injection: Malicious inputs designed to manipulate the AI agent's behavior, either directly or indirectly.
• Curious AI: AI models exhibiting unintended behaviors, such as hallucinations or sophisticated lying, which can lead to misinformation or inappropriate actions.
• Sensitive Data Exfiltration: AI agents inadvertently or maliciously exposing sensitive information they process or access.
• Jailbreaking: Circumventing the AI agent's intended safeguards to achieve unauthorized functionality.

A critical insight is that while runtime risks manifest during operation, their severity and likelihood are often determined by build-time choices. For instance, an agent configured with an overly permissive data access policy at build time creates a larger attack surface for prompt injection or data exfiltration at runtime. This interconnectedness underscores the importance of addressing security at the earliest possible stage in the citizen development process, even if that process is informal.

Technical Deep Dive

▶ Watch: Scale of citizen development and critical data exposure (5:20)

Underoffer illustrates the key findings with concrete examples of how these risks manifest in real-world LCNC applications, often using simplified graphical representations to highlight the vulnerable configurations.

Authorization Misuse is exemplified by a "credit approval agent" built by a user named Chris. This agent is connected to various data sources and actions, including an interaction with Salesforce. The critical vulnerability arises from a build-time choice: Chris configures the agent's action to "use my credentials" (i.e., Chris's credentials) whenever it is invoked. While this might be acceptable if Chris is the sole user, the moment this agent is shared with others, every user interacting with the agent effectively acts as Chris in Salesforce. This leads to privilege escalation for unauthorized users, as they gain access and perform actions far beyond their own permissions by proxy through Chris's agent. This common misconfiguration highlights how easily default settings or convenience can compromise the principle of least privilege.

Authentication Failures are demonstrated through a "hackerbot v2" agent. This agent, despite its suspicious name, is connected to an "internal SharePoint," typically a repository for critical business data. The core issue is that the agent is configured to be publicly available and unauthenticated. This means anyone with the link to interact with the agent, whether internal or external, can access and potentially manipulate the critical data it's connected to. Underoffer emphasizes that while some public-facing applications might legitimately be unauthenticated, the context of internal business data usually dictates strict authentication requirements. The ease with which an LCNC builder can deploy an unauthenticated agent connected to sensitive resources presents a significant and immediate risk.

Data and Secrets Handling is presented with a "simple flow" that mirrors a classic vulnerability in traditional development. This flow, also built by Chris, incorporates a hard-coded plain text secret. This secret, perhaps an API key or a database credential, is embedded directly into the application's logic. If the application's source (or even its configuration) is ever exposed, this secret is immediately compromised. The challenge here is not just the technical vulnerability, which is well-understood by professional developers, but the educational gap. Convincing a finance or HR professional, who may not grasp the implications of hard-coding secrets, to avoid this practice requires a different approach than educating a seasoned software engineer.

Finally, Underoffer provides an example of a runtime risk involving an "email customer support agent." This agent is characterized as autonomous, triggered by an incoming email based on a pre-defined flow. The critical build-time choice that amplifies its runtime risk is the decision to "do not filter the from field" when processing or sending emails. This lack of input validation means the autonomous agent will process emails from any sender without verification. This broadens the attack surface significantly, making the agent susceptible to various attacks, including social engineering, spam, or even more sophisticated prompt injection if the agent interacts with an LLM. This scenario perfectly illustrates how a seemingly innocuous build-time configuration can dramatically increase the potential for malicious activity during the agent's autonomous operation.

These examples collectively underscore that LCNC platforms, while simplifying development, do not inherently simplify security. Instead, they shift the burden of security configuration to non-expert users, making traditional application security principles more critical than ever, albeit in a new technological context.

Demo / Proof of Concept

▶ Watch: Citizen development 'SDLC': business owns all, security absent (6:45)

While Kayla Underoffer's talk does not feature a live demonstration or a new proof of concept developed specifically for the presentation, it effectively uses illustrative visual aids to simulate the demonstration of vulnerabilities. The "Technical Deep Dive" section of the talk relies on simplified graphical representations of LCNC application configurations to visually explain how specific build-time choices lead to security risks like authorization misuse, authentication failures, and hard-coded secrets. These diagrams serve as clear, conceptual proofs of concept, showing the logical flow and vulnerable points within typical citizen-developed applications and AI agents.

Furthermore, Underoffer references a highly relevant BlueHat talk delivered by Microsoft and Zenity, which details Microsoft's own journey in securing their internal LCNC environment. This external reference serves as a real-world case study and a robust proof of concept for the strategies discussed, demonstrating that significant risk reduction is achievable. The BlueHat talk, as highlighted by Underoffer, delves into the shared responsibility model and the successful implementation of "silent remediation campaigns" using the platform's native capabilities, providing concrete evidence of the feasibility and effectiveness of the proposed defensive measures.

Defensive Implications

▶ Watch: Categorizing security risks in citizen development applications (8:00)

The unique characteristics of LCNC environments — their contained nature, inherent automation, and focus on business innovation platforms — offer distinct opportunities for holistic risk reduction. Underoffer outlines a four-pronged strategy: Visibility, Risk Assessment, Remediation, and Prevention.

1. Visibility: This is the foundational step, addressing the pervasive issue of "shadow IT" in LCNC. As Underoffer notes, these environments are often "pure shadow land," with an average 39.1% year-over-year growth further obscuring the landscape. To gain visibility, organizations must:

• Lean on Platform Capabilities: Utilize the native inventory features of enterprise LCNC platforms to identify "who is building what and where."
• Third-Party Tools: Supplement platform capabilities with specialized third-party tools designed for LCNC security posture management.
• Data-Centric Approach: Start by identifying the most critical data (e.g., finance, R&D) and then map the environments and applications that interact with it.
• Set Precedent: Establish from the outset that inventory is mandatory. "If you build it, you will document it." This proactive approach avoids costly retroactive data collection.

2. Risk Assessment: While established toolsets are less mature than in traditional vulnerability management, effective risk assessment is still possible:

• Focus on Build-Time Choices: Analyze "how" applications are built, examining the configurations and choices made by citizen developers.
• Automated Scanning: Due to the scale of these environments (thousands of applications), automated scanning is crucial for identifying common misconfigurations. Manual penetration testing is not yet widely applicable or scaled for this domain.
• Leverage Ecosystem Tools: Though small, the ecosystem of third-party LCNC security tools is growing and should be utilized.

3. Remediation: This is where LCNC platforms offer significant advantages, particularly the feasibility of silent remediation:

• Automated Correction: For certain vulnerabilities, security teams can directly intervene and fix issues without requiring action from the citizen developer. Examples include:
• Removing sensitive data from logs.
• Changing authentication settings (e.g., making an unauthenticated agent authenticated).
• Adjusting access settings (e.g., disallowing guest access at an environmental level).
• Platform Guardrails: Leverage the platform's native capabilities to set these guardrails and perform automated remediation actions.
• Manual Campaigns: For more complex issues or those requiring maker-specific context, traditional manual remediation campaigns will still be necessary, similar to those in professional development environments.

4. Prevention: The ultimate goal is to prevent vulnerabilities from being introduced in the first place:

• Environmental-Level Guardrails: Implement top-down security policies and configurations at the platform or environment level. This ensures that default choices are secure, rather than leaving critical decisions to individual makers.
• Automate Prevention: Utilize the inherent automation capabilities of LCNC platforms to enforce guardrails, validate configurations, and integrate prevention strategies directly into the development workflow.
• Education and Culture:
• Contextual Communication: When performing silent remediation, inform the maker about the change, explain why it was necessary, and provide guidance on how to make the correct choice themselves in the future.
• Top-Down Standards: Establish a clear security culture from leadership, emphasizing that security is a shared responsibility within these environments.
• Collaborate with Business: Unlike the adversarial relationship sometimes seen in traditional shift left, citizen development offers an opportunity for security to collaborate with business users, creating collective security standards.
• Meet Makers Where They Are: Avoid traditional IT ticketing systems (Jira, ServiceNow). Instead, use automated email campaigns, speak the language of the platform, and provide visual aids (e.g., screenshots of fixes) to make security guidance accessible and actionable for non-technical users.

Underoffer cites a powerful success story: a Microsoft team, in collaboration with Zenity, managed to reduce risk by 90% within four months using a team of fewer than five people by implementing these strategies within their own LCNC environment. This demonstrates that with the right approach and leveraging platform capabilities, securing citizen development is not only possible but highly efficient. The talk concludes by urging organizations to assess their current visibility, risk assessment, remediation, and prevention strategies for LCNC, reinforcing that there is "always hope" to build more secure applications in this space.

Key Takeaways

• Citizen Development is a Growing, Critical Attack Surface: Non-IT professionals are rapidly building thousands of production-level applications with LCNC platforms, inherently interacting with critical business data, often without traditional security oversight.
• New Risks Demand New Strategies: The "shift left" challenges in professional development are amplified in citizen development, requiring a tailored approach to address unique build-time (authorization misuse, authentication failures, secrets handling) and runtime (AI prompt injection, data exfiltration) risks.
• Visibility is Paramount: Organizations must gain comprehensive visibility into their LCNC environments, which often exist as "shadow IT," by leveraging platform capabilities, third-party tools, and a data-centric inventory approach.
• Platform-Centric Remediation and Prevention are Key: The contained nature and automation features of LCNC platforms enable effective strategies like "silent remediation" (automated fixes by security teams) and top-down environmental guardrails to prevent vulnerabilities.
• Educate and Collaborate with Makers: Successful security in citizen development requires meeting business users where they are, communicating in their language, providing clear, actionable guidance (e.g., screenshots), and fostering a collaborative security culture rather than an adversarial one.
• Significant Risk Reduction is Achievable: Real-world examples demonstrate that even small security teams can achieve substantial risk reduction (e.g., 90% in four months) by strategically applying these principles and leveraging platform automation.

About the Speaker(s)

Kayla Underoffer is a Lead Security Engineer in the CTO office at Zenity. She is a passionate advocate for security in emerging technology spaces, particularly enthusiastic about the topics discussed at conferences like VulnCon. Her work focuses on understanding and mitigating security risks introduced by new development paradigms, such as citizen development and low-code/no-code platforms.