Istio: The Past, Present and Future of the Project and Communi... Lin Sun, Louis Ryan & Raymond Wong

Istio: The Past, Present and Future of the Project and Community

Speakers: Lin Sun, Head of Open Source, Solo.io; Louis Ryan, CTO, Solo.io; Raymond Wong, Senior Architect, Forbes

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=poBOYc_EkpA

Overview

This talk provides a comprehensive journey through the evolution of Istio, the popular open-source service mesh, from its inception to its current state and exciting future developments. Presented by Istio founder Louis Ryan, CNCF TOC member Lin Sun, and early adopter Raymond Wong from Forbes, the session critically evaluates Istio's original goal of application workload transparency, acknowledging past shortcomings, particularly with the traditional sidecar model. The core focus is on the groundbreaking Istio Ambient Mode, a revolutionary architecture designed to overcome the operational complexities and performance bottlenecks associated with sidecars, offering a more transparent and efficient service mesh experience.

The speakers highlight Istio's significant milestone of joining the Cloud Native Computing Foundation (CNCF) in 2022, a move that bolstered community contributions and adoption. Raymond Wong shares invaluable real-world insights from Forbes' successful migration to ambient mode, detailing the challenges and substantial benefits encountered by a large enterprise. The talk also casts an eye toward the future, revealing upcoming features that address critical enterprise needs, such as enhanced support for AI/ML inference workloads, native integration with Windows environments, and advanced control mechanisms for the new waypoint proxies. This session is crucial for anyone involved in cloud-native application deployment, offering a deep dive into how Istio is adapting to meet the evolving demands of modern distributed systems.

Background

▶ Watch: Introduction and Istio's past, present, future (0:00)

When Istio was first conceived eight years ago, its ambitious goal was to be completely transparent to application workloads. However, as Louis Ryan candidly admits, this goal was largely unmet for a significant period due to the reliance on the sidecar model. In the sidecar architecture, an Envoy proxy runs alongside each application container within the same pod. While effective for traffic interception and policy enforcement, this model introduced considerable operational overhead. Every time an Envoy CVE (Common Vulnerabilities and Exposures) required an upgrade, application pods had to be restarted, leading to downtime and disruption. Furthermore, sidecars could interfere with mutating webhook injections, sometimes preventing pods from starting, and added unnecessary overhead to pipelines like Argo Workflows.

A pivotal moment in Istio's history was its donation to the CNCF in 2022. This move addressed concerns about Google's ownership of the project's copyright, which had previously deterred some adopters and contributors. With the CNCF stewardship, Istio began to aggressively challenge its own design, leading to the development of ambient mode. This new architecture was specifically designed to fulfill the original promise of transparency and mitigate the operational burdens of sidecars.

Forbes, represented by Raymond Wong, provides a compelling real-world context for Istio's evolution. They had been running Istio since before the pandemic, starting with version 1.4 on GKE (Google Kubernetes Engine) across seven clusters (three production, three non-production, one test), managing over 500 services. Forbes initially adopted Istio for several key benefits:

• Shared ingress: Reducing the need for a load balancer per application, saving costs.
• Automated certificate management: Integrating with cert-manager to handle certificate rotation, eliminating manual effort.
• Canary deployments: Enabling weighted traffic routing to new deployments for phased rollouts and early detection of issues.
• Mutual TLS (mTLS): Out-of-the-box encryption for service-to-service communication.
• Observability: Leveraging Kiali and Prometheus metrics for troubleshooting.
• Multicluster service mesh: Although initially implemented for active-active east-west regions, this feature was later rolled back to save costs, highlighting the practical considerations of enterprise adoption.

Forbes' experience underscores the need for a solution like ambient mode, which promised to retain the benefits of Istio while drastically simplifying its operation and reducing resource consumption.

Key Findings

▶ Watch: Istio joins CNCF in 2022 (2:15)

The core innovation presented is Istio Ambient Mode, which significantly re-architects how the service mesh operates to achieve greater transparency and efficiency. Key findings include:

• Two-Layer Architecture for Transparency: Ambient mode introduces a novel two-layer architecture. A ztunnel operates at Layer 4 (L4) on each node, handling mTLS and basic authorization for all pods on that node without requiring sidecars. For advanced Layer 7 (L7) functionalities, waypoint proxies are deployed, acting as dedicated Envoy proxies that can be scoped to namespaces, services, or clusters. This separation allows for granular control and eliminates the need for per-pod sidecars.
• Superior Performance: Performance testing using Iperf demonstrated that ambient mode achieves the highest TCP throughput compared to other service mesh implementations. Crucially, Istio (in both sidecar and ambient modes) and Linkerd were the only projects shown to secure traffic using mTLS even when the source and destination pods are on the same node, a critical security consideration often overlooked.
• Real-World Operational Benefits: Forbes' successful migration of over 500 services across seven clusters to ambient mode validated its operational advantages. The most significant benefit cited was the elimination of application restarts for Istio upgrades or Envoy CVE patches, drastically reducing operational overhead and improving application availability. Raymond Wong highlighted the ease of migration once the initial setup was complete, despite being an early adopter.
• Future-Proofing for Emerging Workloads: Istio is actively evolving to support new and demanding use cases. A significant development is the integration of an AI inference extension into the Kubernetes Gateway API, with Google contributing an Istio implementation. This addresses the unique routing challenges of stateful AI models (e.g., LLM chat sessions), using Envoy's xproc for look-aside load balancing.
• Expanded Platform Reach: Microsoft's contribution of native ambient mesh support for Windows environments demonstrates Istio's commitment to supporting diverse enterprise platforms beyond Linux and Kubernetes. This extends the benefits of transparent traffic capture and security to Windows workloads.
• Enhanced Control over Waypoints: Future releases will provide extensive control over waypoint proxies, including autoscaling, resource allocation, placement (affinity/anti-affinity, zonal topology), rollout strategies, and the ability to bundle extensions like WebAssembly (Wasm) or Lua filters. This increased flexibility allows users to optimize waypoints for specific performance and security requirements.
• Improved Egress Management: The user experience for configuring egress traffic has been radically improved, making it significantly easier to control and secure outbound connections using waypoint proxies and ServiceEntry configurations. This empowers organizations to manage external dependencies more effectively.
• Composability and Innovation: The layered, composable nature of ambient mode, particularly its reliance on the Gateway API, fosters independent innovation. The keynote announcement of K Gateway as an alternate waypoint implementation exemplifies how the ecosystem can extend Istio's capabilities, allowing users to choose the best gateway for their needs while retaining Istio as the mesh.

Technical Deep Dive

▶ Watch: Ambient Mode's two-layer architecture explained (3:15)

Istio Ambient Mode represents a fundamental architectural shift designed to deliver the benefits of a service mesh without the operational burden of sidecars. The core of this innovation lies in its two-layer architecture:

Layer 4 (ztunnel)

The foundational layer of ambient mode is the ztunnel. This component is a lightweight, purpose-built proxy that runs as a daemonset on each node within the Kubernetes cluster. Its primary responsibilities are:

• Mutual TLS (mTLS): The ztunnel establishes and enforces mTLS for all traffic originating from or destined for pods on its node. This ensures that all in-cluster communication is encrypted in transit, regardless of whether the communicating pods are on the same node or different nodes. This addresses a critical security gap where same-node traffic might otherwise bypass encryption.
• Simple Authorization Policy Enforcement: The ztunnel can enforce basic Layer 4 authorization policies, ensuring that only authorized traffic can flow between services.
• Node-Scoped Operation: Critically, the ztunnel operates at the node level, serving all application pods on that node. This eliminates the need for a separate proxy container within each application pod, significantly reducing resource consumption and simplifying application deployment.
• Transparent Traffic Capture: The ztunnel transparently intercepts and secures L4 traffic, making the service mesh infrastructure invisible to the application itself.

Layer 7 (Waypoint Proxy)

For advanced traffic management and policy enforcement at Layer 7, ambient mode introduces waypoint proxies. These are essentially dedicated Envoy proxies that are deployed as separate Kubernetes deployments.

• Advanced L7 Functions: Waypoint proxies provide the rich set of L7 features that Istio is known for, including:
• Traffic shifting and splitting: Enabling Canary deployments, A/B testing, and blue/green deployments.
• Traffic resiliency: Implementing timeouts, retries, circuit breaking, and fault injection.
• Rich Authorization Policies: Enforcing fine-grained access control based on HTTP headers, URLs, methods, and other L7 attributes.
• Layer 7 Observability: Generating detailed metrics, logs, and traces for application traffic, crucial for monitoring and troubleshooting.
• Flexible Scoping: Unlike sidecars tied to individual pods, waypoint proxies can be deployed at various scopes:
• Per-namespace: A single waypoint proxy can serve all services within a specific namespace.
• Per-service: A waypoint can be dedicated to a particular service.
• Multiple namespaces or cluster-wide: A common waypoint can be configured to manage traffic across a broader scope, as demonstrated by Forbes.
• On-Demand Deployment: Waypoint proxies are only deployed when L7 functionalities are required, further optimizing resource usage.

Forbes' Migration to Ambient Mode

Raymond Wong detailed Forbes' practical, phased approach to migrating from the sidecar model to ambient mode:

1. Switch to Kubernetes Gateway API: The first crucial step was to adopt the Kubernetes Gateway API, the "next generation" standard for ingress and load balancing. This involved deploying the Gateway API CRDs and configuring new Gateway and HTTPRoute resources. Forbes used YAML and Kustomize for this configuration.
2. Upgrade and Install Istio Ambient: Forbes upgraded their Istio installation, leveraging an Ansible playbook that now uses Helm and the ambient wrapper chart. Specific GKE platform values were configured, and the Istio CNI was installed into a custom istio-system namespace, requiring an additional resource quota.
3. Set Up Waypoints: Initially, waypoints were deployed per namespace. However, Forbes optimized this by creating a common waypoint in their istio-ingress namespace, similar to how they managed traditional ingress gateways.
4. Update Namespace Labels: This is where the "magic happens" for ambient mode. Forbes implemented a phased rollout using YTT templating to apply three essential labels to namespaces:

• istio.io/dataplane-mode=ambient
• istio.io/rev=default (or a custom revision)
• istio.io/gateway-api-active=true

An additional label was mentioned for integrating waypoints with ingress.

1. Restart Applications (One Last Time): After applying the labels, applications needed one final restart to remove the injected sidecars and allow the ztunnel to take over L4 traffic handling.
2. Validate and Clean Up: Validation commands from Istio docs were used to ensure proper functionality. Old virtual services and gateways were then cleaned up. Raymond also mentioned a new Istio CLI migration tool that provides guided steps and compatibility checks, which would have been useful during their early adoption.

Migration Gotchas and Resolutions

Forbes encountered several challenges during their migration:

• GKE Gateway API Conflicts: Using GKE's native Gateway features alongside manually installed Gateway API CRDs led to Argo CD sync loops due to conflicting definitions. Google's CRDs were noted to be slightly behind the latest open-source versions.
• Gateway API Listener Limits: The Gateway API has a limit of 64 listeners per gateway. Forbes hit this limit, necessitating the creation of additional gateways to support more endpoints.
• External DNS Risks: Misconfiguring External DNS with registry-no-op could potentially wipe out entire DNS zones. Careful configuration and specific external DNS zones with prefixes were required.
• Multicluster Ambient Support: At the time of migration, multicluster service mesh with ambient was not fully supported, but it was announced as an upcoming feature.
• Fully Qualified Names: When moving Istio objects, using fully qualified names (e.g., service.namespace.svc.cluster.local) was sometimes necessary.
• Stuck Pods: Older ambient versions occasionally experienced stuck pods, resolved by restarting the CNI.
• Vault Injection Conflicts: Initial issues connecting to Vault during the injection process were resolved by recreating the Kubernetes cluster and using data plane v2 (a Cilium fork).

Future Technical Directions

The talk outlined several key future developments:

• AI Inference Extension for Gateway API: To support the unique demands of AI/ML inference workloads, particularly stateful ones like LLM chat sessions, a Gateway API inference extension is being developed. Google has contributed an Istio implementation that uses Envoy's x-referral-proc (xproc) extension as a "look-aside load balancer." This allows an external process to dynamically determine the optimal backend for a request based on session state, ensuring requests for a specific chat session are consistently routed to the correct GPU-backed instance. Istio plans to abstract xproc into a more general-purpose API.
• Native Windows Support: Microsoft has contributed an implementation of ambient mesh that runs natively on Windows. This addresses the differences in traffic capture mechanisms on Windows compared to Linux, extending Istio's security and management benefits to Windows-based applications running in Kubernetes or on VMs.
• Advanced Waypoint Control: Future releases will introduce extensive configurations for waypoint proxies, including:
• Autoscaling: Dynamic scaling of waypoints based on traffic.
• Resource Management: Fine-grained control over CPU and memory limits.
• Placement Control: Using affinity/anti-affinity rules and zonal topology to optimize waypoint location relative to applications.
• Rollout Control: Sophisticated strategies for updating waypoints.
• Extension Bundling: The ability to package Wasm or Lua filters directly into waypoint images for custom logic.
• Simplified Egress Configuration: The user experience (UX) for configuring egress traffic is being radically improved. By leveraging waypoint proxies and the ServiceEntry resource, controlling outbound connections to external services (e.g., third-party SaaS, LLMs, GitHub, S3) will become significantly easier, offering the same level of control as ingress traffic.
• Ecosystem Composability: The ambient model's reliance on the Gateway API as a standard integration point promotes a highly composable ecosystem. This allows for alternative waypoint implementations, such as the newly announced K Gateway project, to seamlessly integrate with Istio. This composability is expected to accelerate innovation and offer users more choice in their service mesh components.

Demo / Proof of Concept

▶ Watch: Forbes' reasons for adopting Istio (6:10)

Lin Sun presented a live demonstration of Istio Ambient Mode in action, showcasing its Layer 7 observability capabilities without the need for sidecars. The demo involved a local large language model (LLM) integrated with an application running within an Istio ambient mesh.

During the demonstration, Lin used her phone camera to capture audience gestures, which were then fed to the LLM. The LLM successfully analyzed the mood, reporting it as "positive and interactive." This real-time interaction highlighted the application's functionality.

Crucially, the demo leveraged Kiali, Istio's observability tool, to visualize the traffic flow. The Kiali dashboard showed how requests traversed the ingress gateway, reached the demo application, then routed through the waypoint proxy, and finally connected to the external large language model. Lin specifically pointed out the detailed Layer 7 metrics and insights provided by ambient mode, which were instrumental in debugging an earlier failure she experienced. This illustrated the power of ambient's observability features in identifying and resolving issues without the complexity of managing sidecar-based metrics. The ability to track traffic through the waypoint to an external service demonstrated the comprehensive visibility offered by the new architecture.

Defensive Implications

▶ Watch: Forbes' step-by-step Istio Ambient migration (8:50)

The shift to Istio Ambient Mode and its future developments carry significant defensive implications for organizations securing their cloud-native environments:

• Reduced Attack Surface and Operational Security: By eliminating sidecars, ambient mode drastically reduces the number of running proxy instances per pod. This directly translates to a smaller attack surface. Furthermore, the most significant operational benefit – the elimination of application restarts for Envoy CVE patches or Istio upgrades – means that security updates can be applied to the mesh infrastructure (ztunnels and waypoints) independently of application deployments, leading to faster patching cycles and improved overall security posture without application disruption. This reduces the window of vulnerability.
• Ubiquitous L4 Security with ztunnel: The node-local ztunnel ensures that mTLS and basic authorization are enforced for all L4 traffic, including communication between pods on the same node. This is a critical security enhancement, as same-node traffic often bypasses traditional network firewalls or ingress/egress controls, leaving it vulnerable to compromise. The ztunnel guarantees encryption in transit at the earliest possible point, establishing a zero-trust foundation.
• Centralized and Granular L7 Policy Enforcement: Waypoint proxies enable centralized and highly granular L7 security policies. Defenders can define rich authorization rules, traffic filtering, and rate limiting at the namespace, service, or even cluster level. This provides a single point of control for enforcing security posture across applications, making it easier to audit, manage, and scale security policies compared to managing individual sidecar configurations. This centralization also simplifies the enforcement of compliance requirements.
• Secure Routing for Stateful AI/ML Workloads: The upcoming Gateway API inference extension is vital for securing sensitive AI/ML deployments. By providing a dedicated, secure routing layer that can handle the stateful nature of AI models (e.g., maintaining chat sessions on specific GPUs), organizations can ensure data integrity, prevent unauthorized model access, and protect against inference attacks. The use of Envoy's xproc for intelligent, policy-driven routing is a powerful defensive mechanism for these critical workloads.
• Extending Security to Windows Environments: The native support for ambient mesh on Windows platforms is a significant win for enterprises with mixed operating system environments. This allows organizations to apply the same robust security controls – mTLS, authorization, and observability – to their Windows-based applications, eliminating security blind spots and ensuring consistent policy enforcement across their entire Kubernetes estate.
• Enhanced Egress Control: The improved egress UX empowers security teams to gain much tighter control over outbound traffic. This allows for explicit whitelisting of external dependencies, monitoring of data exfiltration attempts, and enforcement of policies for communication with third-party SaaS providers or external APIs. This capability is crucial for preventing data breaches and maintaining compliance.
• Observability for Threat Detection: The detailed L7 observability provided by ambient mode, as demonstrated with Kiali, remains a powerful defensive tool. By visualizing traffic flows, identifying anomalies, and collecting metrics, security teams can detect suspicious activity, troubleshoot security incidents, and gain insights into potential compromises more effectively than with less transparent architectures.
• Reduced Operational Friction for Security Updates: The ability to upgrade the mesh without restarting applications not only improves availability but also reduces the operational friction often associated with applying security patches. This encourages faster adoption of security updates, minimizing exposure to known vulnerabilities.

Key Takeaways

• Istio Ambient Mode Eliminates Sidecar Overhead: The new architecture significantly reduces operational complexity, resource consumption, and the need for application restarts during mesh upgrades or Envoy CVE patching, fulfilling Istio's original goal of transparency.
• Two-Layer Architecture for Optimized Security and Performance: Ambient mode's ztunnel provides efficient L4 mTLS and basic authorization at the node level, while flexible waypoint proxies deliver advanced L7 traffic management and policy enforcement on demand, securing traffic even on the same node.
• Kubernetes Gateway API is the Future Standard: Adoption of the Gateway API is crucial for modern ingress and service mesh management, offering a unified, extensible, and future-proof control plane for traffic routing.
• Istio Actively Innovates for Emerging Workloads: The project is rapidly evolving to support critical enterprise needs like AI/ML inference with a dedicated Gateway API extension and extending its reach to diverse platforms with native Windows support.
• Enhanced Control and Composability Drive Innovation: Future features will provide extensive control over waypoint proxies, and the ambient model's composability (e.g., with K Gateway) fosters ecosystem innovation and greater choice for users.
• Real-World Adoption Validates Benefits: Forbes' successful migration demonstrates tangible benefits in terms of reduced operational overhead, improved security posture, and cost savings for large-scale enterprise deployments.

About the Speaker(s)

Lin Sun is the Head of Open Source at Solo.io, a prominent contributor to the cloud-native ecosystem. She is also a CNCF Ambassador and a member of the CNCF Technical Oversight Committee (TOC), playing a significant role in guiding the direction of cloud-native projects.

Louis Ryan is the CTO at Solo.io and one of the original founders of the Istio project. He has been deeply involved in Istio's development since its very beginning, bringing extensive historical context and architectural insight to the discussion.

Raymond Wong is a Senior Architect at Forbes, a leading business journalism company. This KubeCon EU talk marked his first-ever conference presentation, where he shared valuable real-world experiences as an early adopter and implementer of Istio Ambient Mode.