HoloConnect AI - From Space to Biohacking

HoloConnect AI - From Space to Biohacking

Speakers: Dr. Fernando De La Peña Llaca, AXA

Conference: DEF CON

YouTube: https://www.youtube.com/watch?v=NVr-rO4aJL4

Overview

In a captivating DEF CON presentation, Dr. Fernando De La Peña Llaca from AXA introduced HoloConnect AI, a groundbreaking system designed to provide real-time holographic presence and AI-driven assistance, even in the most challenging and disconnected environments. The talk underscored a pivotal shift from traditional cloud-dependent AI solutions to robust, edge-based systems capable of operating entirely offline. Dr. De La Peña Llaca, representing AXA Aerospace and AXT, a contractor for NASA and the Department of Defense, framed HoloConnect AI as a solution "built for bad days," emphasizing its resilience against communication failures, network jamming, and privacy threats.

The core innovation lies in its ability to project a three-dimensional holographic representation of a person or an AI agent, coupled with localized artificial intelligence that can "see," "understand," and "interact" with the user in real-time. This goes far beyond conventional video calls or chatbot interactions, aiming for a true sense of "presence." The technology's origins in developing two-way holographic teleportation to the International Space Station highlight its pedigree in overcoming extreme latency and unreliable network conditions, making it uniquely suited for critical applications ranging from remote medical guidance to cybersecurity and hospitality.

The talk not only showcased the technical prowess of HoloConnect AI but also emphasized a strong commitment to security and privacy by design. With features like air-gapped operation, on-device data processing, and a transparent approach to vulnerability disclosure, AXA positions HoloConnect AI as a trustworthy and deployable solution for sensitive operations. The presentation at DEF CON served as both an introduction to this advanced technology and an open invitation to the security community to scrutinize and fortify its defenses, pushing the boundaries of what is possible with secure, autonomous edge AI.

Background

▶ Watch: Introduction: HoloConnect AI for critical communication delays. (0:00)

The genesis of HoloConnect AI stems from a fundamental challenge in communication: overcoming vast distances and unreliable networks, particularly evident in space exploration. Dr. De La Peña Llaca vividly illustrated this with the example of communicating with someone on Mars, where a simple "hello" could take between 25 to 44 minutes to receive a response due to the speed of light. Such delays are not merely inconvenient; in critical scenarios like a medical emergency, they are a matter of life or death. The prospect of a complete communication blackout further exacerbates these risks, highlighting the urgent need for autonomous, real-time decision-making capabilities that do not rely on constant, stable connectivity to a distant cloud.

AXA Aerospace and AXT, as contractors for NASA and the Department of Defense, have been at the forefront of addressing these challenges. Their prior work involved pioneering two-way holographic teleportation to the International Space Station (ISS). This monumental achievement, recognized by NASA with awards such as the Technology of the Year and Exceptional Software Award, demonstrated the feasibility of transmitting a 3D representation of a person across immense distances and at high speeds—the ISS travels at 17,500 miles per hour (28,800 kilometers per hour). This foundational work established a "space-grade communications" standard, proving that holographic presence could survive "bad days" characterized by significant latency, packet loss, and an inherently insecure operating environment.

In parallel with these space-grade communication advancements, the prevailing landscape of artificial intelligence has largely centered on cloud-based processing. While powerful, this model introduces inherent vulnerabilities: reliance on internet connectivity, potential exposure of sensitive data to third-party cloud providers, and the inability to function during network outages or in air-gapped environments. Furthermore, most AI assistants, like Alexa or Siri, primarily "listen" and respond based on audio cues and predefined commands, lacking true visual understanding or contextual awareness. HoloConnect AI emerged from the confluence of these challenges, seeking to merge the immersive experience of holographic presence with the security and autonomy of local, edge-based AI, thereby addressing critical gaps in communication, real-time assistance, and data privacy.

Key Findings

▶ Watch: Core features: 3D presence, local AI, low bandwidth, air-gapped. (3:00)

HoloConnect AI presents a paradigm shift in how AI-driven assistance and remote presence are delivered, moving beyond conventional limitations. The core innovation lies in its ability to deliver 3D presence, projecting a three-dimensional person or AI agent rather than a flat video feed. This immersive experience is powered by local AI at the edge, ensuring that critical decisions are made instantaneously and entirely without the cloud. This fundamental architectural choice underpins several key findings:

Firstly, the system is trustable by design. AXA has implemented robust security measures, including signed models, a comprehensive Software Bill of Materials (SBOM), and on-device logs. This ensures transparency, verifiability, and an auditable trail for all AI operations, a crucial factor for sensitive applications. The architecture supports modular model packs, allowing updates without compromising control or requiring full system overhauls.

Secondly, HoloConnect AI is engineered for extreme resilience and efficiency. It operates with remarkably low bandwidth requirements, needing only 5 megabits per second (Mbps) and below. This makes it viable in environments where high-speed internet is scarce or non-existent. Furthermore, its air-gapped mode allows it to function where cloud connectivity is explicitly disallowed for security reasons. The system also incorporates a jamming-aware mesh network capability, ensuring rapid recovery and continued operation even when radio frequency (RF) interference becomes severe. This suite of features collectively enables "space-grade communications" that are robust, secure, and independent.

Thirdly, the AI agent itself demonstrates advanced contextual understanding. Unlike typical voice assistants, HoloConnect AI "really can see you." It processes depth information, maintains eye contact, and provides hand guidance, indicating sophisticated computer vision capabilities. This allows the AI to understand the context of its environment and the user's actions, responding to commands heard locally with a level of engagement akin to human interaction.

Finally, the technology boasts remarkable device agnosticism, supporting 2,800 devices and counting. This broad compatibility extends across cell phones, tablets, computers, and various VR, AR, and XR headsets, making it highly adaptable and accessible. Beyond its initial space applications, the commercial entity, AXATH, is actively deploying HoloConnect AI in diverse fields such as medicine (for real-time physician consultations), hospitality (for concierge services), and even cybersecurity (for guiding users on potential threats). This broad applicability underscores its versatility and potential to transform multiple industries by providing secure, intelligent, and immersive assistance.

Technical Deep Dive

▶ Watch: HoloConnect AI "sees" and understands context (video demo). (5:55)

HoloConnect AI represents a sophisticated convergence of advanced optics, artificial intelligence, and resilient networking, all optimized for edge deployment. The fundamental technical differentiator is its 3D presence capability. Unlike traditional video conferencing that transmits flat, 2D images, HoloConnect AI captures and renders a three-dimensional representation of a person or an AI agent. While the specific holographic projection technology wasn't detailed in the talk, the speaker mentioned "holo connector AI" machines, available in "table size" and "human size," that operate "no goggles, no glasses." This implies a form of volumetric display or projection that creates a perception of depth and realism without requiring specialized headgear for the end-user, enhancing the sense of true presence and interaction.

The core of HoloConnect AI's intelligence resides in its local AI at the edge. This architecture bypasses the need for constant cloud connectivity, allowing all computational processing and decision-making to occur on specialized local hardware. The speaker explicitly mentioned the use of Nvidia AI computers for this purpose, indicating powerful, specialized GPUs are employed to handle the intensive demands of real-time 3D rendering, computer vision, and natural language processing. By keeping AI models and data on-device, latency is drastically reduced, enabling instantaneous responses critical for scenarios like medical emergencies.

Security and trustworthiness are paramount, reflected in the system's trustable by design principles. This includes:

• Signed models: AI models are cryptographically signed, ensuring their integrity and authenticity. This prevents tampering and guarantees that only approved, verified models are executed on the device.
• Software Bill of Materials (SBOM): A comprehensive SBOM provides a transparent list of all components, libraries, and dependencies used in the HoloConnect AI software. This allows for detailed supply chain security analysis and rapid identification of vulnerabilities in underlying components.
• On-device logs: All operational logs and interactions are stored locally, providing an auditable trail without exposing sensitive information to external cloud services. This also supports post-incident analysis.
• Cloud blocked by default: A stringent security posture ensures that, by default, HoloConnect AI systems do not attempt to connect to external cloud services. This minimizes the attack surface and prevents accidental data exfiltration.
• Wipe to a known good image between sessions: For sensitive deployments, the system can be configured to revert to a pristine, verified state after each operational session, eliminating persistent malware or configuration drift.

The communication backbone leverages space-grade communications principles. This includes end-to-end encryption for all data in transit, mutual authorizations to ensure only approved entities can communicate, and the use of allow-listed routes to restrict network traffic to known, secure pathways. The system is designed for full auditability of its communication channels. Crucially, it boasts a jamming-aware mesh networking capability. This implies dynamic frequency hopping, adaptive routing, and signal processing techniques that allow the system to detect and rapidly recover from deliberate radio frequency interference, maintaining connectivity even in contested electromagnetic environments. The ability to operate on 5 Mbps and below bandwidth further underscores its resilience, making it suitable for satellite links or heavily constrained terrestrial networks.

The AI's ability to "see and understand context" indicates advanced computer vision and spatial awareness algorithms. This involves not just object recognition but also depth perception, pose estimation, and potentially semantic scene understanding. For instance, in the medical scenario, the AI would need to identify a wound, the application of gauze, and the user's hand movements to provide accurate, real-time guidance. The "hand guidance" feature suggests the AI can track human gestures and provide visual overlays or spoken instructions based on those movements. Furthermore, the ability to "hear commands locally" confirms on-device natural language processing (NLP), allowing voice commands to be processed without transmitting audio data to the cloud, enhancing both speed and privacy. The device-agnostic nature of the platform, supporting 2,800+ devices, suggests a highly modular software architecture and flexible rendering pipeline, capable of adapting its output to various display technologies, from phone screens to immersive VR/AR headsets.

Demo / Proof of Concept

▶ Watch: Security: on-device logs, cloud blocked, safer to hack. (7:15)

The presentation incorporated several compelling demonstrations and proofs of concept to illustrate HoloConnect AI's capabilities and its commitment to security.

The most impactful demonstration was a simulated medical emergency scenario set on Mars. The speaker dramatized an astronaut, "Houston, I've been hit. I'm bleeding out. I can't wait." In this dire situation, the astronaut activates HoloConnect AI. The AI agent immediately responds, providing critical, real-time instructions: "Apply direct pressure to the wound with the gauze. You're doing great." This simulation effectively highlighted the system's core value proposition: providing autonomous, AI-driven medical guidance when human help is 200 million kilometers away and seconds matter, all without any cloud connection. It was a powerful visualization of "no cloud, no connection, just our AI working locally to guide a medical procedure in real time."

Beyond this dramatic scenario, a short video clip provided a more direct look at the HoloConnector AI's contextual understanding. The video showed a person holding a model of an aircraft, likely a military plane, and asking the AI, "Hey, can you see this?" The HoloConnector AI responded with remarkable accuracy: "The image shows a person holding a model of an aircraft, likely a military plane with an emphasis on its detailed design, including multiple engines." This demonstrated the AI's ability to not only "see" and identify objects but also to infer details and context from the visual input, moving beyond simple object recognition to a more nuanced understanding of the scene. The speaker emphasized, "You're not talking to a speaker. You're meeting someone. Holo Connector AI sees you. It understands."

Crucially, for a DEF CON audience, Dr. De La Peña Llaca extended an explicit invitation to the security community to "hack" HoloConnect AI. He provided QR codes leading to:

1. The HoloConnect AI app itself, usable on any device.
2. A collection of YouTube videos showcasing the app's functionality.
3. A dedicated platform specifically for security researchers to "hack" the system and report vulnerabilities.
4. Contact information for AXA.

The speaker directly encouraged attendees to focus their efforts on the third QR code, stating, "Just hack the third one." He detailed a structured program for security research, starting with a sandbox environment that allows for streaming, simple thread model swaps, camera feed manipulation, and transport spoof tests. This can then be extended to an SDK, using the "same tools you use in production." AXA committed to shipping fixes for reported issues and crediting researchers for their discoveries, emphasizing coordinated disclosure and the publication of "fixes and lessons learned." This open, transparent approach to security validation serves as a practical proof of concept for the system's robustness and AXA's commitment to building a truly secure product.

Defensive Implications

▶ Watch: Future vision: overlaying human holograms onto robots for trust. (11:15)

HoloConnect AI presents a compelling blueprint for how secure and resilient AI systems can be architected, offering significant defensive implications for a wide range of applications, particularly in critical infrastructure and sensitive data environments.

The most prominent defensive implication is the decentralization of AI processing to the edge. By operating "no cloud, no connection," HoloConnect AI drastically reduces the attack surface commonly associated with cloud-dependent AI. Data, especially sensitive personal health information (PHI) or proprietary business data, remains on the device, mitigating risks of data breaches during transit or at rest in third-party cloud data centers. The speaker explicitly stated, "no PHI, synthetic or de-identified data," "logs stay on device," and "cloud is blocked by default," which are critical privacy-enhancing features. For defenders, this means less reliance on external network security and cloud provider assurances, and more control over data sovereignty.

The trust by design principles are foundational for robust defense. The use of signed models ensures that only authenticated and verified AI models are executed, preventing malicious model injection or tampering. This is crucial for maintaining the integrity of AI-driven decisions, especially in critical applications like medical guidance. The provision of a Software Bill of Materials (SBOM) offers unparalleled transparency into the software supply chain. Defenders can use SBOMs to proactively identify and patch vulnerabilities in third-party components, a common vector for sophisticated attacks. On-device logs provide an immutable audit trail for forensic analysis without external exposure, aiding in incident response and compliance. Furthermore, the ability to wipe to a known good image between sessions offers a powerful recovery mechanism against persistent threats, ensuring system integrity for each new operational period.

For environments prone to communication disruption, such as military operations, disaster relief, or remote industrial sites, the jamming-aware mesh and low bandwidth (5 Mbps and below) capabilities are game-changers. Defenders can deploy HoloConnect AI in contested or degraded network environments, knowing that the system is engineered to maintain communication and AI functionality even under severe RF interference or limited connectivity. This resilience is critical for maintaining command and control, providing real-time intelligence, or delivering emergency services when conventional networks fail.

The speaker also highlighted a direct cybersecurity application: a German company is using HoloConnect AI to guide users on cybersecurity practices, asking "is this safe? can guide me what I'm doing wrong? maybe I have been hack." This suggests an AI-powered security assistant that can visually analyze a user's environment or digital actions and provide real-time, context-aware advice or alerts. Such an agent could potentially guide users through incident response steps, identify phishing attempts, or flag suspicious activity, acting as an intelligent, always-on security guardian.

Finally, AXA's commitment to coordinated disclosure and a security research program is a strong defensive posture. By openly inviting the DEF CON community to "hack" the system and offering credit for discovered vulnerabilities, AXA is proactively hardening its product. This collaborative approach, combined with the promise to "publish fixes and lessons learned," fosters a more secure ecosystem and builds trust with the security community, which is essential for any system intended for critical, real-world deployment.

Key Takeaways

• Edge-Native AI for Critical Operations: HoloConnect AI fundamentally shifts AI processing from the cloud to the edge, enabling real-time, offline operation crucial for environments with extreme latency, unreliable connectivity, or strict air-gapped requirements, as demonstrated by its space heritage.
• True 3D Presence and Contextual Understanding: The system delivers more than just video; it creates a 3D holographic presence with an AI that can "see," "understand context," and provide "hand guidance," offering a level of interaction far beyond traditional voice assistants.
• Security and Privacy by Design: HoloConnect AI is built with robust defensive features, including signed AI models, a comprehensive Software Bill of Materials (SBOM), on-device data logs, and cloud blocked by default, ensuring data sovereignty and system integrity.
• Resilience in Degraded Environments: Engineered for "bad days," the technology utilizes a jamming-aware mesh and operates on ultra-low bandwidth (5 Mbps and below), making it highly resilient to network disruptions and capable of maintaining functionality in contested electromagnetic environments.
• Broad Applicability Beyond Space: While originating from NASA projects, HoloConnect AI has diverse commercial applications in medical (real-time guidance), hospitality (AI concierge), and cybersecurity (intelligent security assistant), showcasing its versatility.
• Open and Collaborative Security Posture: AXA actively encourages security research through a sandbox, SDK, and coordinated disclosure program, inviting the hacking community to identify vulnerabilities and contribute to the system's robust hardening.

About the Speaker(s)

Dr. Fernando De La Peña Llaca is a distinguished figure from AXA, a company deeply involved in advanced technology development. He represents AXA Aerospace and AXT, both of which are proud contractors for prestigious organizations such as NASA and the Department of Defense. Dr. De La Peña Llaca's team is credited with a groundbreaking achievement: the development of two-way holographic teleportation to the International Space Station (ISS). This pioneering work earned them significant recognition from NASA, including the coveted NASA Technology of the Year award by the NASA director and the Exceptional Software award. His expertise lies in creating resilient, cutting-edge communication and AI solutions designed to operate in extreme and challenging environments. More recently, Dr. De La Peña Llaca indicated that AXA Aerospace's government work has spun off into AXATH, a commercial entity now bringing these advanced technologies to broader markets, including the medical field, hospitality, and business-to-enterprise sectors. He is driven by a vision to build secure, autonomous AI solutions that can truly see, understand, and interact with users, pushing the boundaries of human-AI collaboration.