From Virtual Touch to Tesla Command: Unlocking Unauthenticated Control Chains From Smart Glasses for Vehicle Takeover

From Virtual Touch to Tesla Command: Unlocking Unauthenticated Control Chains From Smart Glasses for Vehicle Takeover

Speakers: Xingli Zhang, PhD Student, University of Louisiana at Lafayette; Yazhou Tu; Yan Long; Liqun Shan; Mohamed A Elsaadani; Kevin Fu

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=5_NGOKfUolg

Overview

This talk, presented by Xingli Zhang and collaborators at IEEE S&P, unveils a novel and concerning attack vector that allows an attacker to manipulate a Tesla vehicle without the owner's interaction or authentication, starting from a seemingly innocuous pair of smart glasses. The research demonstrates how a chain of vulnerabilities, spanning physical hardware, software automation tools, and integrated APIs, can be exploited to achieve unauthorized control over a connected vehicle. This work is significant because it highlights the often-overlooked security implications of increasingly interconnected consumer electronics and the transitive trust relationships that form between them.

The core of the attack lies in leveraging Electromagnetic Interference (EMI) to trigger the voice assistant on smart glasses, which then, through pre-configured automation shortcuts on a paired smartphone, sends critical commands to a Tesla car. The researchers successfully bypassed the typical requirement for phone unlocking or direct user authentication, showcasing a real-world, low-cost, and rapid attack that could have serious safety and privacy consequences. This presentation serves as a critical wake-up call for manufacturers and users alike regarding the complex security perimeter of modern smart ecosystems.

The findings are particularly relevant in an era where smart devices are becoming ubiquitous, and automation tools are simplifying complex tasks across various platforms. As more devices join these interconnected systems, the potential for unforeseen control chains and exploitable weak links multiplies. This talk not only identifies such a chain but also meticulously dissects its components, demonstrating a complete end-to-end takeover and proposing concrete defensive strategies.

Background

▶ Watch: Unauthenticated Tesla control via smart glasses (0:00)

The proliferation of consumer electronics, particularly smart glasses, has led to devices with enhanced functionalities, greater user accessibility, and compatibility with a wide array of other gadgets. These advancements, while convenient, inadvertently create complex control chains where devices are linked through various communication protocols and software integrations. A common feature in many smart glasses, such as the Razer Anzu used as an example in this research, is the integration of capacitive touch sensors for intuitive controls like changing music, managing calls, and activating voice assistants.

Previous research has established that strong electromagnetic signals can induce a "force touch" event on capacitive touch sensors. This phenomenon forms the initial physical layer of the attack vector explored in this talk. Capacitive touch sensing works by detecting changes in capacitance when a conductive object, like a human finger, approaches or touches the sensor. Microcontrollers measure the charge time of a capacitor to infer these touch events. The researchers built upon this understanding, investigating whether specific EMI signals could reliably mimic a touch, thereby activating the voice assistant on smart glasses without physical contact.

Complementing this hardware-level vulnerability, the research also leverages automation tools like Apple Shortcuts (on iOS) and IFTTT (If This Then That, often used on Android). These tools are designed to simplify repetitive tasks by allowing users to create automated action chains. For instance, a user might configure a shortcut to "unlock my car" via a voice command. Crucially, these automation tools can be activated through voice assistants and, once configured, often bypass the need for explicit phone unlocking or additional authentication for certain actions, especially if the device is already paired and trusted. Tesla's decision to make its APIs publicly available in October 2023 further facilitates integration with third-party applications and automation platforms, expanding the potential attack surface. This combination of physical EMI vulnerabilities and software automation bypasses creates the "unauthenticated control chain" at the heart of the presented research.

Key Findings

▶ Watch: Proving Emi attack on capacitive touch with prototype (4:00)

The research presents several critical findings that collectively demonstrate the feasibility and robustness of the "Virtual Touch to Tesla Command" attack:

1. EMI-Induced Voice Assistant Activation: The primary discovery is the ability to reliably activate the voice assistant on smart glasses (specifically Razer Anzu and Huawei X Gentle Monster Eyewear II Plus) using carefully calibrated Electromagnetic Interference (EMI) signals. This circumvents the need for physical touch, enabling an attacker to trigger the voice assistant from a distance. The researchers found effective frequencies within the 100 MHz to 1,000 MHz range, with a 10 MHz interval scan identifying optimal points.
2. Bypassing Phone Authentication via Automation Tools: The talk unequivocally demonstrates that once the voice assistant on the smart glasses is activated, automation tools (Apple Shortcuts on iOS and IFTTT on Android) can be exploited to send commands to a paired Tesla vehicle, even when the smartphone remains locked. This is a crucial bypass of a fundamental security layer, as these tools, when configured, inherit a level of transitive trust from the initial pairing, allowing critical actions without further user verification.
3. End-to-End Unauthenticated Vehicle Control: The researchers successfully combined these two vulnerabilities to create a complete, unauthenticated control chain. An attacker can initiate the process by injecting EMI into smart glasses, activating the voice assistant, which then triggers an automation shortcut on a locked phone to send commands (e.g., "unlock Tesla") to the vehicle. The attack process was shown to be remarkably fast, completing in approximately seven seconds.
4. High Success Rates and Real-World Applicability: In both indoor laboratory evaluations and outdoor real-world tests, the attack achieved nearly 100% success rates. The outdoor experiments, conducted in various typical parking environments (coffee shops, supercharger stations, supermarkets, university lots), confirm the practicality and effectiveness of the attack under diverse ambient conditions.
5. Low-Cost Attack Apparatus: The attack can be executed using readily available and inexpensive equipment. The portable electromagnetic signal generator based on an oscillating circuit and a mini speaker, used for delivering voice commands, cost approximately $28 and $19 respectively, highlighting the low barrier to entry for potential attackers.
6. Robustness Against Noise: The attack's performance was evaluated under various noise levels, including white noise, human conversation, and traffic noise, ranging from 17 dBA up to 90 dBA. Even at noise levels significantly higher than typical urban street noise (73.4 dBA), the attack maintained high success rates, demonstrating its resilience in real-world noisy environments.

These findings collectively underscore a significant security vulnerability arising from the complex interplay of physical device vulnerabilities and software automation, posing a tangible threat to connected vehicle security.

Technical Deep Dive

▶ Watch: Bypassing phone lock using automation tools (Shortcuts/IFTTT) (6:00)

The attack described in the talk is a sophisticated multi-stage process, chaining together vulnerabilities in hardware (smart glasses), software (smartphone automation), and API integration (Tesla).

1. EMI Injection to Smart Glasses:

The first stage involves activating the voice assistant on smart glasses using Electromagnetic Interference (EMI). The researchers focused on smart glasses like the Razer Anzu and Huawei X Gentle Monster Eyewear II Plus, which utilize capacitive touch sensors for interaction. These sensors detect touch events by measuring changes in capacitance when a finger approaches, which in turn affects the charge time of a capacitor.

To identify the most effective frequencies for generating EMI coupling, the team scanned frequencies from 100 MHz to 1,000 MHz with 10 MHz intervals. The setup included a directional antenna, an amplifier, and a signal generator. For deeper understanding, they tore down the smart glasses and built a prototype capacitive touch sensing circuit using an Arduino. By observing the charge time of the capacitor under both finger touch and EMI signals, they confirmed that EMI signals indeed extended the charge time, mimicking a touch event.

To increase the attack distance and intensity, an oscillating circuit was developed. This circuit consisted of a bipolar junction transistor (BJT) as its gate device, a DC power supply, an antenna, and a three-point capacitor oscillating circuit. This specialized circuit generated higher-intensity EMI signals, making the attack feasible from a practical distance. The EMI signal effectively mimicked a two-second press or double-tap, which are the standard gestures to activate the voice assistant on these smart glasses.

2. Bypassing Authentication with Automation Tools:

Once the voice assistant on the smart glasses is activated, the next challenge is to bypass the smartphone's lock screen and authentication requirements to issue commands to the Tesla. This is achieved through automation tools: Apple Shortcuts for iOS and IFTTT (If This Then That) for Android.

• Apple Shortcuts (iOS): For iPhones, once the official Tesla app is installed, corresponding shortcuts are automatically integrated into the iOS system. Users can create personalized shortcuts or automate specific tasks. The attack leverages a pre-configured shortcut (e.g., "Unlock Tesla") that the voice assistant can trigger directly. Crucially, if the phone is already paired with the smart glasses and the shortcut is set up, the voice assistant can execute the command even if the phone screen is locked, bypassing Face ID or Touch ID. Apple Shortcuts offers over 300 built-in actions and integrates with many popular apps.

• IFTTT (Android): For Android phones, the process involves linking the user's Tesla account with a third-party application like Teslemetry. This generates an access token in Tesla. An IFTTT applet is then configured to execute a web hook that utilizes Teslemetry's APIs, which in turn communicate with the official Tesla API to interact with the car. IFTTT provides over 650 available services. Similar to iOS, once this chain is established, a voice command received by the phone (via the smart glasses' voice assistant) can trigger the IFTTT applet, sending commands to the Tesla without requiring the phone to be unlocked.

The researchers note that Tesla made its APIs publicly available in October 2023, further enabling such integrations and expanding the scope for third-party applications and automated control chains. The combined effect of EMI triggering the voice assistant and automation tools bypassing authentication creates a potent, unauthenticated path to vehicle control.

Demo / Proof of Concept

▶ Watch: Outdoor attack scenario and low-cost devices (9:00)

The talk included a compelling demonstration of the full attack chain, showcasing its practicality and speed in real-world scenarios.

Attack Scenario: The researchers envisioned a common scenario: a victim driver parks their Tesla in front of a coffee shop, takes off their smart glasses, leaves them on the car's dashboard, and steps into the coffee shop. The smartphone remains connected to the smart glasses, and the glasses are typically left in an "on" mode unless intentionally powered down.

Attack Devices: The entire attack was executed using two low-cost, portable devices:

1. A portable electromagnetic signal generator, based on the oscillating circuit developed by the researchers.
2. A mini speaker, used to play the voice command for the smart glasses' assistant.

These devices cost approximately $28 and $19 respectively and are easily procurable from online retailers like Amazon or Walmart, highlighting the low barrier to entry for potential attackers.

Attack Execution: The demonstration video showed the attack being completed in a mere seven seconds. The sequence involved:

1. The attacker positioning the EMI generator near the smart glasses on the dashboard.
2. The EMI signal activating the smart glasses' voice assistant.
3. The mini speaker playing a pre-recorded voice command (e.g., "Unlock Tesla").
4. The voice command being processed by the paired, locked smartphone via automation tools.
5. The Tesla receiving and executing the "unlock" command.

Real-World Testing: To validate the attack's performance under diverse conditions, the researchers conducted outdoor experiments at four typical urban sites:

• A parking lot in front of a coffee shop.
• A Supercharger station.
• A parking lot in front of a supermarket.
• A relatively quiet university parking site.

For each site, they tested the "unlock Tesla" command 10 times using Huawei X Gentle Monster Eyewear II Plus glasses paired with an iPhone. The success rates across all these diverse environments were consistently high, approaching 100%.

Noise Level Evaluation: Recognizing that ambient noise could affect voice command recognition, the team also evaluated the attack's performance under controlled noise levels. They used two full-range speakers placed in front of the car to generate various types of noise: white noise, high-fidelity recordings of human conversation, and traffic noise, all downloaded online. They gradually increased the sound level from 17 dBA up to 90 dBA, in increments of 5 dBA. The experiments showed that even at noise levels significantly exceeding typical urban street noise (which averages 73.4 dBA in New York City, according to 2015 statistics), the attack maintained high success rates, demonstrating its robustness and practical viability in noisy real-world settings.

Defensive Implications

▶ Watch: 7-second Tesla takeover demo and high success rates (10:00)

The detailed analysis of this control chain attack provides crucial insights for developers, manufacturers, and users to bolster security. The researchers emphasize that simply disabling voice assistants or adding blanket authentication steps can compromise accessibility, especially for routine activities. Instead, a more nuanced approach is required:

1. Anomaly-Based Authentication: Instead of constant authentication, systems should implement mechanisms to detect anomalous events. For example, if a "car unlock" command is received when the user's phone is geographically far from the vehicle, the system should trigger an additional authentication challenge or deny the command. This helps balance security with user convenience.

1. Enhanced Human Feature Detection for Smart Glasses: To verify the bona fide usage of smart glasses and prevent EMI-induced activations, passive verification modules can be integrated:

• Skin Detection: Modules to detect the presence of human skin could verify that the glasses are being worn.
• Airflow Change Detection: Sensors could detect changes in airflow around the ear, indicating physical presence.
• IMU Sensors for Liveness Detection: The Inertial Measurement Unit (IMU) sensors already present in many smart glasses could be used to detect subtle movements indicative of being worn by a living person.
• Simple Infrared Modules: An infrared module could detect the presence of an object (like a face) between two frames, acting as a basic presence detector.

1. Advanced Voice Verification: For voice assistants, integrating modules capable of distinguishing between bonafide human voices and synthetic or recorded voices would be a significant defense. This would prevent attackers from using mini speakers with pre-recorded commands.

1. Challenges in EMI Shielding: The researchers acknowledge that completely shielding capacitive touch sensors against EMI is challenging. Capacitive touch sensors inherently require non-conductive external material around the touch area, which can make effective electromagnetic shielding difficult without compromising design or functionality. This suggests that software and system-level defenses might be more practical than purely physical shielding for this specific vulnerability.

1. Stricter Security Standards for Smartphone and App Ecosystems: Smartphone system and app designers, particularly those managing automation tools and integrations, must adopt stricter and more comprehensive testing standards.

• New Functionality & Device Connection: Every new function or device connection point should undergo rigorous security evaluation.
• Third-Party Risk Assessment: For third-party applications and services integrating with critical systems (like vehicle APIs), vendors should be required to perform self-assessments and provide proof of their security measures before being allowed to join the control chain. Phone companies have a role in enforcing these standards.

In essence, defensive strategies must move beyond isolated component security to address the transitive trust and interconnectedness of modern smart ecosystems. This involves a layered approach combining anomaly detection, advanced biometrics, and stringent security evaluations at every point of integration.

Key Takeaways

• Transitive Trust is a Critical Vulnerability: The attack highlights how trust relationships between seemingly disparate devices (smart glasses, smartphone, car) can be chained together, creating an unauthenticated path to critical system control.
• Physical Layer Attacks are Real: Electromagnetic Interference (EMI) can reliably trigger capacitive touch sensors on smart glasses, activating voice assistants without physical interaction, demonstrating the importance of physical layer security.
• Automation Tools Can Bypass Authentication: Smartphone automation tools (Apple Shortcuts, IFTTT) can be exploited to execute sensitive commands from voice assistants, even when the phone is locked, effectively bypassing primary authentication mechanisms.
• Connected Ecosystems Expand Attack Surface: The increasing integration of smart devices and the public availability of APIs (e.g., Tesla's) create complex control chains that can be exploited, making security assessments of the entire ecosystem crucial.
• Low-Cost & Rapid Attacks are Possible: The demonstrated attack is achievable with inexpensive, off-the-shelf equipment and can be completed in a matter of seconds, making it a highly practical threat.
• Layered Defenses are Essential: Mitigating such complex attacks requires a multi-faceted defense strategy, including anomaly detection, advanced biometric verification for voice assistants, and stricter security vetting for all connected devices and third-party integrations.

About the Speaker(s)

The lead presenter for this research was Xingli Zhang, a PhD student from the University of Louisiana at Lafayette. His work, as demonstrated in this talk, focuses on the security implications of emerging consumer electronic products and their interconnectedness within automated control chains. He, along with his collaborators Yazhou Tu, Yan Long, Liqun Shan, Mohamed A Elsaadani, and Kevin Fu, contributed to this detailed study on the security vulnerabilities arising from the integration of smart glasses, smartphones, and connected vehicles.