Bots can Snoop: Uncovering and Mitigating Privacy Risks of Bots in Group Chats

Bots can Snoop: Uncovering and Mitigating Privacy Risks of Bots in Group Chats

Speakers: Kai-Hsiang Chou

Conference: USENIX Security

YouTube: https://www.youtube.com/watch?v=EDypI3WyK0I

Overview

This talk, presented by Kai-Hsiang Chou, delves into the often-overlooked privacy implications of integrating chatbots into group messaging platforms. Titled "Bots can Snoop: Uncovering and Mitigating Privacy Risks of Bots in Group Chats," the research highlights how commonly deployed chatbots are frequently over-privileged, gaining access to sensitive user data far beyond what is necessary for their intended functions. This over-privilege manifests in two critical ways: chatbots reading messages irrelevant to their purpose, and their ability to identify message senders, potentially enabling cross-group user tracking.

The talk underscores a fundamental tension in modern messaging: the desire for enhanced functionality through bots versus the imperative to maintain user privacy, particularly within end-to-end encrypted (E2EE) environments. While many platforms offer robust E2EE for peer-to-peer or group chats, the introduction of chatbots often compromises these guarantees or fails to implement crucial privacy features like selective message access and sender anonymity. The presented research not only identifies these pervasive vulnerabilities across popular platforms but also proposes a novel cryptographic protocol, SnoopGuard, designed to reconcile functionality with robust privacy protection in group chat settings.

The significance of this work is profound, given the widespread adoption of chatbots. As of 2021, Discord alone hosted approximately 430,000 active chatbots, estimated to be present in 30% of all groups, while Telegram boasted over 10 million. The privacy risks associated with these ubiquitous digital assistants are substantial, ranging from inadvertent data leakage to potential malicious tracking. By exposing these vulnerabilities and offering a practical, protocol-level solution, this research provides a critical roadmap for platform developers and users alike to foster a more secure and privacy-respecting group messaging ecosystem.

Background

▶ Watch: Introduction: Chatbots' privacy risks in group chats (0:00)

The proliferation of chatbots has dramatically reshaped the landscape of digital communication, particularly within group messaging applications. Platforms like Discord and Telegram serve as prime examples of this integration, with hundreds of thousands to millions of active chatbots facilitating various tasks, from scheduling and moderation to answering queries. While these bots enhance user experience and group functionality, their integration often comes at a significant privacy cost, largely due to their over-privileged access to group chat data.

The core privacy problems identified stem from two primary issues:

1. Over-privileged Message Access: Many chatbots are designed or configured to read every message within a group, irrespective of its relevance to their function. For instance, a facial detection chatbot might have access to all messages, including those that do not contain images or URLs. A case study involving a Q&A bot on Discord revealed it was reading far more messages than necessary for its operation. This broad access creates a large attack surface and increases the risk of sensitive information leakage.
2. Sender Identity Leakage: Chatbots frequently learn the identity of the user who sends a message, even when this information is not required for their task. This capability enables bots, particularly malicious ones, to track users across different groups. Analysis of a Telegram conversation dataset revealed that 3.6% of users encountered the same bot in multiple groups, underscoring the potential for persistent user tracking and profiling.

To address these issues, the research proposes two critical privacy properties for chatbot integration:

• Selective Message Access: A mechanism to limit a chatbot's access exclusively to messages that are directly relevant to its purpose.
• Sender Anonymity: A method to hide the identity of the message sender from the chatbot, preventing cross-group user tracking.

Ideally, these properties should be provided without compromising end-to-end encryption (E2EE), a cornerstone of secure messaging. However, a survey of popular messaging platforms—including WhatsApp, Viber, Signal, Line, Telegram, Discord, Slack, and Keybase—revealed a significant gap in their ability to simultaneously deliver E2EE, selective message access, and sender anonymity.

• WhatsApp, Viber, Signal (User Bots): These platforms, when using automated user accounts as bots, preserve E2EE. However, they offer neither selective message access nor sender anonymity, meaning bots see everything and everyone.
• Line: While Line supports E2EE for group chats, adding a chatbot explicitly disables encryption for the entire group. Even then, it fails to provide selective message access or sender anonymity.
• Telegram and Discord: These platforms offer some form of selective message access (e.g., through specific bot commands or message types) but do not support E2EE for groups with bots.
• Slack: Slack provides sender anonymity (messages from bots can be generic, or bot interactions are command-based), but it does not offer E2EE for group communications.
• Keybase: Keybase stands out as the only platform in the study that supports both E2EE and selective message access. However, its design does not incorporate sender anonymity, meaning bots can still identify who sent a message.

The difficulty in achieving these features alongside E2EE primarily stems from the intricacies of group key agreement (GKA) schemes. E2EE relies on shared secret keys, which ideally should rotate frequently to enhance security. However, existing GKA solutions face three main challenges in the context of bots:

1. Selective Message Access with Key Rotation: If group keys rotate, a bot might miss key updates, rendering it unable to decrypt messages intended for it. Conversely, if keys do not rotate, the bot might decrypt messages not meant for it. The confidentiality of a message should not depend on whether its ciphertext is hidden from the chatbot.
2. Sender Anonymity with Key Updates: Most common GKA protocols reveal who initiated a key update, thereby leaking sender identity. Achieving sender anonymity requires updating keys without disclosing the updater's identity.

These challenges highlight a critical need for new cryptographic protocols that can seamlessly integrate privacy-preserving bot functionality into E2EE group messaging environments.

Key Findings

▶ Watch: Analysis of existing platforms' bot privacy features (2:40)

The research presented in "Bots can Snoop" uncovers several critical findings regarding the state of chatbot privacy in group messaging and proposes a foundational solution:

1. Pervasive Over-Privilege and Privacy Risks: The study definitively demonstrates that chatbots, due to their widespread adoption and often unrestricted access, pose significant privacy risks in group chats. They are frequently over-privileged, meaning they can read messages irrelevant to their function and identify senders, enabling potential user tracking across multiple groups. For example, 3.6% of users in the Telegram dataset encountered the same bot in different groups, highlighting the potential for persistent profiling.
2. Failure of Current Platforms: A comprehensive survey of popular messaging platforms (WhatsApp, Viber, Signal, Line, Telegram, Discord, Slack, Keybase) reveals a consistent failure to simultaneously provide end-to-end encryption (E2EE), selective message access, and sender anonymity when chatbots are present. Most platforms either disable E2EE upon bot integration, lack selective access, or fail to anonymize senders. Keybase was the sole exception providing E2EE and selective access, but still lacked sender anonymity.
3. Fundamental Protocol Challenges: The core difficulty in achieving these properties while maintaining E2EE is rooted in existing group key agreement (GKA) schemes. Mechanisms for key rotation, essential for forward secrecy, conflict with the need for selective access (bots might miss keys or decrypt unintended messages). Similarly, most GKA protocols inherently leak the identity of participants who initiate key updates, undermining sender anonymity.
4. SnoopGuard: A Novel Cryptographic Solution: The most significant contribution is the introduction of SnoopGuard, a secure group messaging protocol. SnoopGuard is designed to address the identified challenges by integrating selective message access and sender anonymity directly into E2EE group chats. It is built upon existing tree-based continuous group key agreement (CGKA) schemes, making it compatible with modern protocols like MLS (Message Layer Security) and Sender Keys.
5. The Compressed Multiloot Tree (CMRT): At the heart of SnoopGuard is the Compressed Multiloot Tree (CMRT), a novel GKA structure. CMRT extends tree-based CGKA by strategically isolating chatbots from the main user key tree. This design allows for independent key derivation for each bot and anonymizes user key updates from the bots' perspective, effectively delivering both selective message access and sender anonymity without compromising E2EE.
6. Minimal Performance Overhead: Empirical evaluation of SnoopGuard, integrated with MLS, demonstrates that the protocol introduces only minimal overhead for operations like adding a chatbot or sending messages. While its theoretical complexity for sending messages (O(log n + m)) is higher than some alternatives (e.g., Sender Keys' O(1)), it is deemed acceptable given that most groups typically contain only a small number of chatbots.
7. Dependence on Trigger Functions: The effectiveness of SnoopGuard, particularly for selective message access, hinges on the reliability of trigger functions that determine message relevance. A poorly designed or malicious trigger function could label all messages as relevant, effectively bypassing SnoopGuard's protections. This highlights a critical non-protocol-level challenge that requires robust vetting mechanisms.

These findings collectively paint a clear picture of existing privacy deficiencies in group chat bot integrations and offer a robust, cryptographically sound path forward.

Technical Deep Dive

▶ Watch: Introducing SnoopGuard protocol and Compressed Multiloot Tree (CMRT) (4:45)

SnoopGuard is presented as a secure messaging protocol specifically designed to enable selective message access and sender anonymity within end-to-end encrypted (E2EE) group chats, building on modern tree-based continuous group key agreement (CGKA) schemes. The protocol aims to be easily integratable into existing frameworks like the Sender Keys protocol and the Message Layer Security (MLS) protocol.

The core technical innovation of SnoopGuard is the Compressed Multiloot Tree (CMRT). This novel group key agreement structure extends traditional tree-based CGKA to accommodate chatbots in a privacy-preserving manner.

Compressed Multiloot Tree (CMRT) Architecture

1. Regular User Structure: When a group consists solely of regular users, the CMRT functions identically to the underlying CGKA scheme. Users are represented as leaves in a cryptographic tree, and the group establishes a common group key, typically denoted as G, at the root of this tree.
2. Adding a Chatbot (C1): When a chatbot (e.g., C1) is introduced to the group, it is not integrated directly into the existing CGKA tree as a regular user. Instead, SnoopGuard creates a small, separate tree specifically for C1. This small tree has two leaves:

• One leaf is a copy of the root of the main user CGKA tree (G).
• The other leaf is the chatbot itself (C1).

These two leaves then derive a shared root key, denoted as S1, using the standard CGKA process.

1. User's View vs. Chatbot's View:

• Regular users maintain a complete view of the entire tree structure, including the main group key G, the chatbot's shared root S1, and the chatbot C1.
• The chatbot's view is strictly limited. It can only "see" its own small tree. Crucially, its only information about the larger user CGKA tree is the copied root node G. Because the chatbot only observes the root node G and not the internal structure or individual leaf nodes of the user tree, any key updates made by individual users within the main tree are effectively anonymized from the chatbot's perspective. This mechanism is what achieves sender anonymity.

1. Adding Multiple Chatbots (C2): If a second chatbot (C2) is added, the process is replicated. Another distinct small tree is created, connecting a copy of the main user tree's root (G) with C2, and deriving a shared root S2. This design ensures that each chatbot maintains its own independent state and keying material, preventing interference between them. This isolation is fundamental to enabling selective message access, as individual chatbots can be granted access to specific keys without affecting others.

Message Sending Mechanism

The CMRT's structure facilitates selective message sharing and sender anonymity during communication:

• User Sending a Message to Chatbot C1:

1. The user first updates the main user CGKA tree. This action produces a new group root key, G'.
2. This update then propagates upward into the small tree associated with C1, resulting in a new shared root key S1'.
3. The user can then encrypt the message using S1', a key known only to the users and chatbot C1.
4. Crucially, the state and keys for chatbot C2 (and any other chatbots) remain completely unchanged. C2 is unaware of both the message and the key update intended for C1, thus ensuring selective message access.

• User Sending a Message to Chatbot C2 (after C1):

1. The user again updates the main CGKA tree, producing a new group key G''.
2. Because the user retains a copy of the previous relevant group key G (the one from the last time C2 might have received an update), they can propagate this new key update specifically to C2's small tree.
3. This results in a new shared root key S2'' for C2.
4. The user then encrypts the message using S2''.
5. Throughout this process, chatbot C1 remains completely unaware of both the message and the key update for C2, reinforcing selective access.

• Chatbot Sending a Message:

1. If chatbot C1 wishes to send a message to the group, it updates its own small tree, deriving a new key S1''.
2. It then uses S1'' to encrypt the message.
3. Because each chatbot has its own distinct root and tree, the tree of chatbot C2 remains unaffected.
4. Furthermore, users can handle the key update originating from C1 without requiring C1 to know the intricate structure of the user's subtree, preserving the privacy of the user group.

Performance Evaluation

SnoopGuard's performance was evaluated both theoretically and empirically.

• Theoretical Complexity (n users, m chatbots):
• Adding a chatbot: Requires O(n + m) operations. This involves all users and existing chatbots participating in the key agreement process to integrate the new bot.
• Sending a message: Requires O(log n + m) operations. This complexity arises from navigating the main user tree (log n) and potentially updating keys for m chatbots if a message is sent to all of them.
• Comparison with Existing Protocols:
• MLS (Message Layer Security): Adding a chatbot O(n + m), sending/receiving messages O(log n + m).
• Sender Keys Protocol: Adding a chatbot O(n + m), sending/receiving messages O(1).

Compared to MLS, SnoopGuard exhibits similar complexity. However, for sending messages, SnoopGuard's O(log n + m) is higher than Sender Keys' O(1). The speakers acknowledge this, but argue it is generally not a significant issue because most groups typically do not have a large number of chatbots (m is small).

• Empirical Performance: The researchers implemented SnoopGuard, integrated with MLS, to measure its practical performance. The results showed that both adding a chatbot and sending a message introduced only minimal overhead to the overall protocol operations. This suggests that the theoretical complexity, while sometimes higher, translates to acceptable real-world performance.

Limitations

A significant limitation of SnoopGuard highlighted in the talk is its dependence on the trigger function. The effectiveness of selective message access hinges on how accurately and reliably the trigger function can identify whether a message is relevant to a specific chatbot's purpose. If this function is poorly designed, contains vulnerabilities, or is maliciously manipulated to label every message as relevant, a chatbot could effectively bypass SnoopGuard's protections and regain unrestricted access. The speakers suggest that addressing this issue may fall outside the scope of the messaging protocol itself, proposing vetting processes (manual or automated) to ensure trigger functions behave as intended and cannot be abused.

Demo / Proof of Concept

▶ Watch: CMRT: How chatbots are added and isolated securely (5:45)

While the talk did not feature a live, interactive demonstration for the audience, the practical feasibility and performance of SnoopGuard were thoroughly validated through its implementation and empirical evaluation. The researchers integrated SnoopGuard with the Message Layer Security (MLS) protocol, a modern standard for group key agreement, to measure its real-world overhead.

This implementation served as the proof of concept, demonstrating that SnoopGuard's novel Compressed Multiloot Tree (CMRT) structure could be successfully integrated into an existing, robust group messaging framework. The empirical measurements focused on the total time required for all clients to perform key operations: adding a chatbot to a group and sending a message. The results, presented in figures during the talk, indicated that these operations introduced only minimal overhead to the underlying MLS protocol. This practical validation confirms that SnoopGuard is not just a theoretical construct but a viable solution that can be deployed without significantly degrading the performance of E2EE group messaging applications.

Defensive Implications

▶ Watch: SnoopGuard's theoretical and empirical performance evaluation (9:15)

The findings and proposed solution of "Bots can Snoop" carry substantial implications for various stakeholders in the digital communication ecosystem:

For Messaging Platform Developers:

• Prioritize Privacy-by-Design for Bots: Platform architects must move beyond merely integrating chatbots for functionality and embed selective message access and sender anonymity as core requirements, alongside end-to-end encryption (E2EE). This means adopting or adapting protocols like SnoopGuard.
• Implement Robust Group Key Agreement: Invest in modern, flexible CGKA schemes that can accommodate the unique privacy needs of bots without compromising E2EE. The MLS protocol is a strong candidate for such integration.
• Develop Bot Vetting Processes: Recognizing the vulnerability introduced by trigger functions, platforms must establish rigorous vetting procedures for chatbots. This could involve automated code analysis, sandboxing, and manual reviews to ensure trigger functions are not malicious or overly permissive, and that they adhere to a principle of least privilege.
• Transparent Permission Models: Design user interfaces that clearly communicate the permissions a chatbot has, what data it can access, and for what purpose. Users should have granular control over these permissions.

For Chatbot Developers:

• Adhere to Least Privilege: Design chatbots to request and utilize only the absolute minimum amount of data and permissions necessary for their intended function. Avoid broad message access or sender identification unless explicitly required and justified.
• Secure Trigger Functions: Implement trigger functions with security and privacy in mind. Ensure they are robust against manipulation and accurately determine message relevance. Consider open-sourcing or making trigger logic auditable where appropriate.
• Educate Users: Provide clear documentation on how your bot handles user data, what privacy features it supports, and how users can manage its permissions.

For End Users:

• Be Aware of Bot Permissions: Understand that adding a chatbot to a group chat can significantly alter its privacy posture. Be cautious about the bots you invite and verify their reputation.
• Question Over-Privileged Bots: If a chatbot seems to request excessive permissions or access messages unrelated to its stated purpose, exercise caution or avoid using it.
• Advocate for Privacy Features: Support platforms that prioritize user privacy and offer features like selective message access and sender anonymity for bots. Provide feedback to developers about your privacy concerns.

For Security Researchers and Academics:

• Advance Usability Research: Further explore how to design intuitive interfaces that make complex privacy features (like selective message access and sender anonymity) comprehensible and manageable for average users.
• Develop Adaptive Permission Models: Investigate more dynamic and context-aware permission frameworks, potentially leveraging natural language processing (NLP) to help users make informed decisions about bot access in evolving group dynamics.
• Study User Perceptions: Conduct further research into user perceptions of privacy in group chat settings with bots, as most existing work focuses on one-on-one interactions. Understanding user expectations and concerns is crucial for designing effective and adoptable privacy solutions.

By adopting these defensive strategies across the ecosystem, the risks identified by "Bots can Snoop" can be significantly mitigated, paving the way for a future where chatbot functionality and user privacy can coexist harmoniously.

Key Takeaways

• Chatbots in group messaging platforms are frequently over-privileged, leading to significant privacy risks such as access to irrelevant messages and leakage of sender identities, enabling potential user tracking across groups.
• Current popular messaging platforms generally fail to simultaneously provide end-to-end encryption (E2EE), selective message access, and sender anonymity when bots are integrated, often compromising E2EE or lacking key privacy features.
• SnoopGuard is a novel secure group messaging protocol that successfully integrates selective message access and sender anonymity into E2EE group chats, addressing a critical gap in existing solutions.
• The core of SnoopGuard is the Compressed Multiloot Tree (CMRT), which extends tree-based continuous group key agreement by creating isolated key agreement structures for bots, thereby anonymizing user key updates and enabling targeted message sharing.
• While SnoopGuard introduces only minimal performance overhead when integrated with protocols like MLS, its effectiveness in enforcing selective message access is critically dependent on the security and reliability of trigger functions.
• Future research and development must focus on improving the usability of privacy features, developing adaptive permission models for bots, and better understanding user perceptions of privacy in complex group chat environments.

About the Speaker(s)

Kai-Hsiang Chou presented the talk "Bots can Snoop: Uncovering and Mitigating Privacy Risks of Bots in Group Chats" at USENIX Security. He is the lead author of the paper on which this presentation is based, a collaborative work with Leean Wang, Jonathan Wayin, and other colleagues. His research focuses on enhancing privacy and security within modern communication platforms, particularly in the context of integrating third-party services like chatbots into encrypted environments.