SlideShare a Scribd company logo
Better Traffic Control
with Envoy
Mark McBride
1/31/2018
Why Care about Traffic Control
Generaliza5ons: Kubernetes leads to a bunch of good things.
• Crea5ng new services is easier.
• Deploying new service versions is easier.
• Deploying smaller services is easier.
Why Care about Traffic Control
But the good things aren’t free.
• New code needs to be (safely!) integrated with your request flow.
• Addi5onal abstrac5ons have < 100% reliability.
• Longer call chains introduce more chances for failure.
Goals of Traffic Control
• Resilience
• Distributed systems are never “up”1. Dealing with failures should be
straighTorward.
• Rou5ng
• Introducing a new code to the call chain is a common opera5on. It
should be straighTorward.
1. Charity Majors, hYps://opensource.com/ar5cle/17/7/state-systems-administra5on

Recommended for you

Helm - Package Manager for Kubernetes
Helm - Package Manager for KubernetesHelm - Package Manager for Kubernetes
Helm - Package Manager for Kubernetes

Helm is a package manager for Kubernetes. It helps streamline installing and managing applications. This session covers prerequisites for Helm, which include a basic understanding of containers and Kubernetes along with its architecture. It also covers the limitations that come with running deployments using the kubectl binary, Helm's architecture, templating with it and finally ends on a note highlighting the difference between versions 2 and 3.

helmkubernetescontainer
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide

Kubernetes is an open-source tool for managing containerized workloads and services. It allows for deploying, maintaining, and scaling applications across clusters of servers. Kubernetes operates at the container level to automate tasks like deployment, availability, and load balancing. It uses a master-slave architecture with a master node controlling multiple worker nodes that host application pods, which are groups of containers that share resources. Kubernetes provides benefits like self-healing, high availability, simplified maintenance, and automatic scaling of containerized applications.

kubernetesdockercloud computing
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...

If you have an existing Java monolith, you know you must take care making changes to it or altering it in any negative way. Often times these monoliths are very valuable to the business and generate a lot of revenue. At the same time, since it’s difficult to make changes to the monolith it’s desirable to move to a microservices architecture. Unfortunately you cannot just do a big-bang migration to a greenfield architecture and will have to incrementally adopt microservices. In this talk, we’ll look at using Gloo proxy which is based on Envoy Proxy and GraphQL to do surgical, function-level traffic control and API aggregation to safely migrate your monolith to microservices and serverless functions.

envoy proxyservice meshgraphql
The Setup
• Create scenarios using augmented Envoy examples
• Use wrk to drive load against the system and measure results
• Curl, because no demo is complete without some curl
• A preview of envoy-tools to observe Envoy stats directly
Control Requires Visibility
• Making unobservable changes is not advised.
• Envoy comes with great tools out of the box.
• Stats on listeners, clusters, protocols, and more.
• An admin server for direct observa5on and control.
• envoy-tools (coming soon!) – a repository of tools that provide a more
approachable interface.
The Examples
Adding Reality to Examples
• Add configurable latency and success rate

Recommended for you

Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)

AWSKRUG 판교 2019.06.05 Kubernetes Internals (Kubernetes 해부하기) - Understanding Kubernetes Components -- Understanding Kube-APIServer -- Understanding Kube-Scheduler -- Understanding Kube-Controller-Manager -- Understanding Kube-Proxy -- Understanding DNS - Understanding Kubernetes Networking -- Understanding Pod Networking -- Understanding Service Networking

kubernetesk8sinternals
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-inNews And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in

EWERK and ENA contributed developers to the Tungsten Fabric SDN Special Interest Group. The goals were to integrate the widely adopted Tungsten Fabric into Apache CloudStack to provide advanced routing, security, and high performance networking features. This would allow features like static NAT, load balancing, firewalling, and encapsulated VM traffic between hosts using MPLS or VXLAN. The integration work added a Tungsten Fabric provider to CloudStack, allowed creating VMs with Tungsten networks, and added network policy and source NAT/port forwarding features. More testing is needed with the community.

apache cloudstackcloudstackiaas
Deep dive into Kubernetes Networking
Deep dive into Kubernetes NetworkingDeep dive into Kubernetes Networking
Deep dive into Kubernetes Networking

The document provides an overview of Kubernetes networking concepts including single pod networking, pod to pod communication, service discovery and load balancing, external access patterns, network policies, Istio service mesh, multi-cluster networking, and best practices. It covers topics such as pod IP addressing, communication approaches like L2, L3, overlays, services, ingress controllers, network policies, multi-cluster use cases and deployment options.

kubernetesnetworkingdocker
Adding Reality to Examples
Retries
Envoy supports retry policies aYached to
routes
• Select error codes to retry on.
• Configure 5meouts for each retry.
• Configure number of retries.
Retries
• No failures!
Retries—a Closer Look
Failures when calling service1
No failures returned to client

Recommended for you

Persistent Storage with Containers with Kubernetes & OpenShift
Persistent Storage with Containers with Kubernetes & OpenShiftPersistent Storage with Containers with Kubernetes & OpenShift
Persistent Storage with Containers with Kubernetes & OpenShift

Manually configuring mounts for containers to various network storage platforms and services is tedious and time consuming. OpenShift and Kubernetes provides a rich library of volume plugins that allow authors of containerized applications (Pods) to declaratively specify what the storage requirements for the containers are so that OpenShift can dynamically provision and allocate the storage assets for the specified containers. As the author of the Kubernetes Persistent Volume specification, I will provide an overview of how Persistent Volume plugins work in OpenShift, demo block storage and file storage volume plugins and close with the Red Hat storage roadmap. Presented at LinuxCon/ContainerCon by Mark Turansky, Principal Software Engineer, Red Hat Mark Turansky is a Principal Software Engineer at Red Hat and a full-time contributor to the Kubernetes Project. Mark is the author of the Kubernetes Persistent Volume specification and a member of the Red Hat OpenShift Engineering team.

linuxconred hatkubernetes
Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!

Google DevFest2019 Presentation at Infosys Campus Bangalore. Application deployment in Kubernetes with Helm is demo'ed in Google Kubernetes Engine (GKE). This is an introductory session on Helm. Several references are given in it to further explore helm3 as it is in Beta state now.

kubernetes applicationk8s application deploymentkubernetes helm
OVN - Basics and deep dive
OVN - Basics and deep diveOVN - Basics and deep dive
OVN - Basics and deep dive

OVN provides virtual networking capabilities for Open vSwitch including logical switches, routers, security groups, and ACLs. It uses OVSDB to configure OVN components and provides native integration with OpenStack Neutron. OVN's architecture includes a northbound database for logical network definitions, a southbound database for physical mappings, and daemons like ovn-northd and ovn-controller that translate between the databases.

ovnovsopenstack
Safe Retries
• Usually you don’t want to retry all
requests.
• Side effects are important to consider.
• Atomicity is important to consider.
• Computa5onal expense is important to
consider.
• Add more routes, and configure retries
accordingly.
Load Shedding
• Some5mes you get more traffic than you can handle.
• Envoy supports request limits on a per-cluster basis.
• Envoy also supports two priority groups, allowing you to save slots for
important traffic.
Without Circuit Breakers
Failures are fine, but 99% latency is slowwwwww
as requests just back up
Also, POST requests are totally offline because
we’re swamped with GETs
Without Circuit Breakers
Retries overflow, which is slow

Recommended for you

Introduction to Kafka Streams
Introduction to Kafka StreamsIntroduction to Kafka Streams
Introduction to Kafka Streams

Kafka Streams is a new stream processing library natively integrated with Kafka. It has a very low barrier to entry, easy operationalization, and a natural DSL for writing stream processing applications. As such it is the most convenient yet scalable option to analyze, transform, or otherwise process data that is backed by Kafka. We will provide the audience with an overview of Kafka Streams including its design and API, typical use cases, code examples, and an outlook of its upcoming roadmap. We will also compare Kafka Streams' light-weight library approach with heavier, framework-based tools such as Spark Streaming or Storm, which require you to understand and operate a whole different infrastructure for processing real-time data in Kafka.

stream processingkafka
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes

Those are the slides that were used to give an introduction to Kubernetes at the Nardoz Berlin Meetup on the 2018-06-28.

kubernetesawsdocker
GitOps - Operation By Pull Request
GitOps - Operation By Pull RequestGitOps - Operation By Pull Request
GitOps - Operation By Pull Request

Presentation given at Cloud Native Copenhagen, Cloud Native Aalborg, and Cloud Native Aarhus in December 2020

gitopscloudnative
With Circuit Breakers
Gobs of failures, but p99 latency is s5ll good.
Also, POST requests are available.
Also, we told clients to back off with the
x-envoy-overloaded response header.
With Circuit Breakers
Pending requests overflow, which is fast!
An Overview of Rou5ng
• Endpoint metadata for richer rou5ng primi5ves
• Probabilis5c distribu5on of traffic across mul5ple clusters
• 1% of traffic to my-great-rewrite, 99% to legacy
• 1% of traffic to v2 of my service, 99% to v1
• Header based rou5ng to cluster subsets
• If “x-canary” is set route to endpoints with a version label of v2
• Priority rou5ng, which we saw in the circuit breaking example
• Zone aware rou5ng
Traffic Shioing the Hard Way
• Mul5ple clusters
• Mul5ple routes

Recommended for you

Linux Networking Explained
Linux Networking ExplainedLinux Networking Explained
Linux Networking Explained

Linux offers an extensive selection of programmable and configurable networking components from traditional bridges, encryption, to container optimized layer 2/3 devices, link aggregation, tunneling, several classification and filtering languages all the way up to full SDN components. This talk will provide an overview of many Linux networking components covering the Linux bridge, IPVLAN, MACVLAN, MACVTAP, Bonding/Team, OVS, classification & queueing, tunnel types, hidden routing tricks, IPSec, VTI, VRF and many others.

linux networking kernel network
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링

- 발표자: 고려대학교 임재민 - 자료: https://www.slideshare.net/openstack_kr/openinfra-days-korea-2018-track-4-grafana-openstack

grafanaopenstackopeninfra days korea 2018
ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리

ksqlDB로 실시간 데이터 변환 및 스트림 처리

ksqldbconfluentcloud
Header-based Canary
• When we specify the canary header, the route matches and we (and
only we) are routed to service1a
• When header is not present, the route doesn’t match and we go on to
the next route, sending traffic to service1
Probabilis5c Rollout
• With the run5me match, we choose this route 25% of the 5me,
sending 25% of our traffic to service1
Traffic Shioed
25% of traffic to service1a
The Easy Way
• Restar5ng servers on every config change is tedious in this demo.
• It’s even more tedious in produc5on.
• Envoy provides a beYer way—the xDS APIs.

Recommended for you

Kubernetes Security with Calico and Open Policy Agent
Kubernetes Security with Calico and Open Policy AgentKubernetes Security with Calico and Open Policy Agent
Kubernetes Security with Calico and Open Policy Agent

Ray Kao and Kevin Harris from Microsoft presenting ‘Kubernetes Security with Calico and Open Policy Agent’ at the spring 2019 Kubernetes and Cloud Native meetup in Toronto.

kubernetescalicoopen policy agent
Argus Production Monitoring at Salesforce
Argus Production Monitoring at SalesforceArgus Production Monitoring at Salesforce
Argus Production Monitoring at Salesforce

We’ll present details about Argus, a time-series monitoring and alerting platform developed at Salesforce to provide insight into the health of infrastructure as an alternative to systems such as Graphite and Seyren.

salesforceargushbase
Embracing Failure - Fault Injection and Service Resilience at Netflix
Embracing Failure - Fault Injection and Service Resilience at NetflixEmbracing Failure - Fault Injection and Service Resilience at Netflix
Embracing Failure - Fault Injection and Service Resilience at Netflix

A presentation given at AWS re:Invent on how Netflix induces failure to validate and harden production systems. Technologies discussed include the Simian Army (Chaos Monkey, Gorilla, Kong) and our next gen Failure Injection Test framework (FIT).

availabilityfailuresimian army
xDS APIs
• CDS - discover clusters, which are logical groupings of endpoints.
• A cluster defini5on can have a reference to an EDS endpoint
• EDS - discover endpoints for a cluster.
• LDS - discover listeners for an Envoy
• A listener’s filter chain can have a reference to an RDS endpoint
• RDS - discover routes for a filter chain
Dynamic Config
• The xDS APIs give you a central point-of-control to manage a fleet of
Envoys
• Bridge service discovery (e.g. from Kubernetes) to Envoy
• Bridge rou5ng config (e.g. from Houston) to Envoy
Advanced Rou5ng with EDS
• CDS (cluster discovery service) defines groups of endpoints.
• EDS (endpoint discovery service) discovers the actual endpoints for
clusters.
• EDS allows you to aYach metadata to an endpoint.
• Our mul5-cluster example can be collapsed to a metadata based
approach on a single cluster.
Even Easier with Houston
• An CDS/EDS server with integra5ons to EC2, ECS, Kubernetes,
Consul, DC/OS, or JSON files
• An LDS/RDS server with an intui5ve route configura5on UI
• Stats parsing, forwarding, and change tracking

Recommended for you

(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...

Complex distributed systems fail. They fail more frequently, and in different ways, as they scale and evolve over time. In this session, you learn how Netflix embraces failure to provide high service availability. Netflix discusses their motivations for inducing failure in production, the mechanics of how Netflix does this, and the lessons they learned along the way. Come hear about the Failure Injection Testing (FIT) framework and suite of tools that Netflix created and currently uses to induce controlled system failures in an effort to help discover vulnerabilities, resolve them, and improve the resiliency of their cloud environment.

awscloudtechnical decision maker
Service Stampede: Surviving a Thousand Services
Service Stampede: Surviving a Thousand ServicesService Stampede: Surviving a Thousand Services
Service Stampede: Surviving a Thousand Services

How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity? This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem. Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.

scalaakkaakka streams
Deep Dive on Elastic Load Balancing
Deep Dive on Elastic Load BalancingDeep Dive on Elastic Load Balancing
Deep Dive on Elastic Load Balancing

This document provides an in-depth overview of Elastic Load Balancing (ELB) on AWS, covering topics like security, scalability, availability, and best practices. It discusses how ELB handles SSL/TLS encryption and security policies. It explains how ELB scales dynamically based on load and optimizes latency, throughput, and wait times. It also covers how ELB provides high availability across Availability Zones, performs health checks, and integrates with services like Route 53, CloudWatch, and Auto Scaling.

colm maccárthaighawsaws summit 2016
Ques5ons/Contact
Mark McBride
mark@turbinelabs.io
Twitter - @mccv
http://www.turbinelabs.io

More Related Content

What's hot

Challenges of Kubernetes On-premise Deployment
Challenges of Kubernetes On-premise DeploymentChallenges of Kubernetes On-premise Deployment
Challenges of Kubernetes On-premise Deployment
Vietnam Open Infrastructure User Group
 
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...
Ambassador Labs
 
Meetup 23 - 02 - OVN - The future of networking in OpenStack
Meetup 23 - 02 - OVN - The future of networking in OpenStackMeetup 23 - 02 - OVN - The future of networking in OpenStack
Meetup 23 - 02 - OVN - The future of networking in OpenStack
Vietnam Open Infrastructure User Group
 
Helm - Package Manager for Kubernetes
Helm - Package Manager for KubernetesHelm - Package Manager for Kubernetes
Helm - Package Manager for Kubernetes
Knoldus Inc.
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
Bytemark
 
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Christian Posta
 
Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)
DongHyeon Kim
 
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-inNews And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
ShapeBlue
 
Deep dive into Kubernetes Networking
Deep dive into Kubernetes NetworkingDeep dive into Kubernetes Networking
Deep dive into Kubernetes Networking
Sreenivas Makam
 
Persistent Storage with Containers with Kubernetes & OpenShift
Persistent Storage with Containers with Kubernetes & OpenShiftPersistent Storage with Containers with Kubernetes & OpenShift
Persistent Storage with Containers with Kubernetes & OpenShift
Red Hat Events
 
Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!
Krishna-Kumar
 
OVN - Basics and deep dive
OVN - Basics and deep diveOVN - Basics and deep dive
OVN - Basics and deep dive
Trinath Somanchi
 
Introduction to Kafka Streams
Introduction to Kafka StreamsIntroduction to Kafka Streams
Introduction to Kafka Streams
Guozhang Wang
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Raffaele Di Fazio
 
GitOps - Operation By Pull Request
GitOps - Operation By Pull RequestGitOps - Operation By Pull Request
GitOps - Operation By Pull Request
Kasper Nissen
 
Linux Networking Explained
Linux Networking ExplainedLinux Networking Explained
Linux Networking Explained
Thomas Graf
 
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
OpenStack Korea Community
 
ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리
confluent
 
Kubernetes Security with Calico and Open Policy Agent
Kubernetes Security with Calico and Open Policy AgentKubernetes Security with Calico and Open Policy Agent
Kubernetes Security with Calico and Open Policy Agent
CloudOps2005
 
Argus Production Monitoring at Salesforce
Argus Production Monitoring at SalesforceArgus Production Monitoring at Salesforce
Argus Production Monitoring at Salesforce
HBaseCon
 

What's hot (20)

Challenges of Kubernetes On-premise Deployment
Challenges of Kubernetes On-premise DeploymentChallenges of Kubernetes On-premise Deployment
Challenges of Kubernetes On-premise Deployment
 
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...
 
Meetup 23 - 02 - OVN - The future of networking in OpenStack
Meetup 23 - 02 - OVN - The future of networking in OpenStackMeetup 23 - 02 - OVN - The future of networking in OpenStack
Meetup 23 - 02 - OVN - The future of networking in OpenStack
 
Helm - Package Manager for Kubernetes
Helm - Package Manager for KubernetesHelm - Package Manager for Kubernetes
Helm - Package Manager for Kubernetes
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
 
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...
 
Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)
 
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-inNews And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
News And Development Update Of The CloudStack Tungsten Fabric SDN Plug-in
 
Deep dive into Kubernetes Networking
Deep dive into Kubernetes NetworkingDeep dive into Kubernetes Networking
Deep dive into Kubernetes Networking
 
Persistent Storage with Containers with Kubernetes & OpenShift
Persistent Storage with Containers with Kubernetes & OpenShiftPersistent Storage with Containers with Kubernetes & OpenShift
Persistent Storage with Containers with Kubernetes & OpenShift
 
Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!
 
OVN - Basics and deep dive
OVN - Basics and deep diveOVN - Basics and deep dive
OVN - Basics and deep dive
 
Introduction to Kafka Streams
Introduction to Kafka StreamsIntroduction to Kafka Streams
Introduction to Kafka Streams
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
GitOps - Operation By Pull Request
GitOps - Operation By Pull RequestGitOps - Operation By Pull Request
GitOps - Operation By Pull Request
 
Linux Networking Explained
Linux Networking ExplainedLinux Networking Explained
Linux Networking Explained
 
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
[OpenInfra Days Korea 2018] (Track 4) - Grafana를 이용한 OpenStack 클라우드 성능 모니터링
 
ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리
 
Kubernetes Security with Calico and Open Policy Agent
Kubernetes Security with Calico and Open Policy AgentKubernetes Security with Calico and Open Policy Agent
Kubernetes Security with Calico and Open Policy Agent
 
Argus Production Monitoring at Salesforce
Argus Production Monitoring at SalesforceArgus Production Monitoring at Salesforce
Argus Production Monitoring at Salesforce
 

Similar to Traffic Control with Envoy Proxy

Embracing Failure - Fault Injection and Service Resilience at Netflix
Embracing Failure - Fault Injection and Service Resilience at NetflixEmbracing Failure - Fault Injection and Service Resilience at Netflix
Embracing Failure - Fault Injection and Service Resilience at Netflix
Josh Evans
 
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
Amazon Web Services
 
Service Stampede: Surviving a Thousand Services
Service Stampede: Surviving a Thousand ServicesService Stampede: Surviving a Thousand Services
Service Stampede: Surviving a Thousand Services
Anil Gursel
 
Deep Dive on Elastic Load Balancing
Deep Dive on Elastic Load BalancingDeep Dive on Elastic Load Balancing
Deep Dive on Elastic Load Balancing
Amazon Web Services
 
Client Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right WayClient Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right Way
DataStax Academy
 
(CMP401) Elastic Load Balancing Deep Dive and Best Practices
(CMP401) Elastic Load Balancing Deep Dive and Best Practices(CMP401) Elastic Load Balancing Deep Dive and Best Practices
(CMP401) Elastic Load Balancing Deep Dive and Best Practices
Amazon Web Services
 
Deep Dive on Elastic Load Balancing
Deep Dive on Elastic Load BalancingDeep Dive on Elastic Load Balancing
Deep Dive on Elastic Load Balancing
Amazon Web Services
 
Impala Performance Update
Impala Performance UpdateImpala Performance Update
Impala Performance Update
Cloudera, Inc.
 
Concurrency at Scale: Evolution to Micro-Services
Concurrency at Scale:  Evolution to Micro-ServicesConcurrency at Scale:  Evolution to Micro-Services
Concurrency at Scale: Evolution to Micro-Services
Randy Shoup
 
AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...
AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...
AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...
Amazon Web Services
 
Making communication across boundaries simple with Azure Service Bus
Making communication across boundaries simple with Azure Service BusMaking communication across boundaries simple with Azure Service Bus
Making communication across boundaries simple with Azure Service Bus
Particular Software
 
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
IBM MQ: Managing Workloads, Scaling and Availability with MQ ClustersIBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
David Ware
 
AWS Webinar 201: Designing scalable, available & resilient cloud applications
AWS Webinar 201: Designing scalable, available & resilient cloud applicationsAWS Webinar 201: Designing scalable, available & resilient cloud applications
AWS Webinar 201: Designing scalable, available & resilient cloud applications
Amazon Web Services
 
Dealing with the Three Horrible Problems in Verification
Dealing with the Three Horrible Problems in VerificationDealing with the Three Horrible Problems in Verification
Dealing with the Three Horrible Problems in Verification
DVClub
 
Deploying at will - SEI
 Deploying at will - SEI Deploying at will - SEI
VMworld 2014: Extreme Performance Series
VMworld 2014: Extreme Performance Series VMworld 2014: Extreme Performance Series
VMworld 2014: Extreme Performance Series
VMworld
 
Nokia kpi and_core_optimization
Nokia kpi and_core_optimizationNokia kpi and_core_optimization
Nokia kpi and_core_optimization
debasish goswami
 
“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud
Amazon Web Services
 
“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud
Amazon Web Services
 
Production Ready Microservices at Scale
Production Ready Microservices at ScaleProduction Ready Microservices at Scale
Production Ready Microservices at Scale
Rajeev Bharshetty
 

Similar to Traffic Control with Envoy Proxy (20)

Embracing Failure - Fault Injection and Service Resilience at Netflix
Embracing Failure - Fault Injection and Service Resilience at NetflixEmbracing Failure - Fault Injection and Service Resilience at Netflix
Embracing Failure - Fault Injection and Service Resilience at Netflix
 
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
(PFC305) Embracing Failure: Fault-Injection and Service Reliability | AWS re:...
 
Service Stampede: Surviving a Thousand Services
Service Stampede: Surviving a Thousand ServicesService Stampede: Surviving a Thousand Services
Service Stampede: Surviving a Thousand Services
 
Deep Dive on Elastic Load Balancing
Deep Dive on Elastic Load BalancingDeep Dive on Elastic Load Balancing
Deep Dive on Elastic Load Balancing
 
Client Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right WayClient Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right Way
 
(CMP401) Elastic Load Balancing Deep Dive and Best Practices
(CMP401) Elastic Load Balancing Deep Dive and Best Practices(CMP401) Elastic Load Balancing Deep Dive and Best Practices
(CMP401) Elastic Load Balancing Deep Dive and Best Practices
 
Deep Dive on Elastic Load Balancing
Deep Dive on Elastic Load BalancingDeep Dive on Elastic Load Balancing
Deep Dive on Elastic Load Balancing
 
Impala Performance Update
Impala Performance UpdateImpala Performance Update
Impala Performance Update
 
Concurrency at Scale: Evolution to Micro-Services
Concurrency at Scale:  Evolution to Micro-ServicesConcurrency at Scale:  Evolution to Micro-Services
Concurrency at Scale: Evolution to Micro-Services
 
AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...
AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...
AWS re:Invent 2016: Global Traffic Management with Amazon Route 53 Traffic Fl...
 
Making communication across boundaries simple with Azure Service Bus
Making communication across boundaries simple with Azure Service BusMaking communication across boundaries simple with Azure Service Bus
Making communication across boundaries simple with Azure Service Bus
 
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
IBM MQ: Managing Workloads, Scaling and Availability with MQ ClustersIBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
 
AWS Webinar 201: Designing scalable, available & resilient cloud applications
AWS Webinar 201: Designing scalable, available & resilient cloud applicationsAWS Webinar 201: Designing scalable, available & resilient cloud applications
AWS Webinar 201: Designing scalable, available & resilient cloud applications
 
Dealing with the Three Horrible Problems in Verification
Dealing with the Three Horrible Problems in VerificationDealing with the Three Horrible Problems in Verification
Dealing with the Three Horrible Problems in Verification
 
Deploying at will - SEI
 Deploying at will - SEI Deploying at will - SEI
Deploying at will - SEI
 
VMworld 2014: Extreme Performance Series
VMworld 2014: Extreme Performance Series VMworld 2014: Extreme Performance Series
VMworld 2014: Extreme Performance Series
 
Nokia kpi and_core_optimization
Nokia kpi and_core_optimizationNokia kpi and_core_optimization
Nokia kpi and_core_optimization
 
“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud
 
“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud“Spikey Workloads” Emergency Management in the Cloud
“Spikey Workloads” Emergency Management in the Cloud
 
Production Ready Microservices at Scale
Production Ready Microservices at ScaleProduction Ready Microservices at Scale
Production Ready Microservices at Scale
 

Recently uploaded

FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)
FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)
FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)
Roshan Dwivedi
 
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
Semiosis Software Private Limited
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdf
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdfAWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdf
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdf
karim wahed
 
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
avufu
 
Overview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptxOverview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptx
Mitchell Marsh
 
NYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdfNYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdf
AUGNYC
 
ENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentationENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentation
sofiafernandezon
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
DNUG e.V.
 
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdfIndependence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
Livetecs LLC
 
ANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdfANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdf
sachin chaurasia
 
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
ThousandEyes
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
Mitchell Marsh
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
ssuser2b426d1
 
Folding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a seriesFolding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a series
Philip Schwarz
 
WEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service ProvidersWEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service Providers
Severalnines
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Sparity1
 
Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …
908dutch
 
Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)
miso_uam
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
karim wahed
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
shivamt017
 

Recently uploaded (20)

FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)
FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)
FAST Channels: Explosive Growth Forecast 2024-2027 (Buckle Up!)
 
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdf
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdfAWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdf
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) AWS Security .pdf
 
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
 
Overview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptxOverview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptx
 
NYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdfNYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdf
 
ENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentationENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentation
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
 
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdfIndependence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
 
ANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdfANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdf
 
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
 
Folding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a seriesFolding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a series
 
WEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service ProvidersWEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service Providers
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
 
Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …
 
Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
 

Traffic Control with Envoy Proxy

  • 1. Better Traffic Control with Envoy Mark McBride 1/31/2018
  • 2. Why Care about Traffic Control Generaliza5ons: Kubernetes leads to a bunch of good things. • Crea5ng new services is easier. • Deploying new service versions is easier. • Deploying smaller services is easier.
  • 3. Why Care about Traffic Control But the good things aren’t free. • New code needs to be (safely!) integrated with your request flow. • Addi5onal abstrac5ons have < 100% reliability. • Longer call chains introduce more chances for failure.
  • 4. Goals of Traffic Control • Resilience • Distributed systems are never “up”1. Dealing with failures should be straighTorward. • Rou5ng • Introducing a new code to the call chain is a common opera5on. It should be straighTorward. 1. Charity Majors, hYps://opensource.com/ar5cle/17/7/state-systems-administra5on
  • 5. The Setup • Create scenarios using augmented Envoy examples • Use wrk to drive load against the system and measure results • Curl, because no demo is complete without some curl • A preview of envoy-tools to observe Envoy stats directly
  • 6. Control Requires Visibility • Making unobservable changes is not advised. • Envoy comes with great tools out of the box. • Stats on listeners, clusters, protocols, and more. • An admin server for direct observa5on and control. • envoy-tools (coming soon!) – a repository of tools that provide a more approachable interface.
  • 8. Adding Reality to Examples • Add configurable latency and success rate
  • 9. Adding Reality to Examples
  • 10. Retries Envoy supports retry policies aYached to routes • Select error codes to retry on. • Configure 5meouts for each retry. • Configure number of retries.
  • 12. Retries—a Closer Look Failures when calling service1 No failures returned to client
  • 13. Safe Retries • Usually you don’t want to retry all requests. • Side effects are important to consider. • Atomicity is important to consider. • Computa5onal expense is important to consider. • Add more routes, and configure retries accordingly.
  • 14. Load Shedding • Some5mes you get more traffic than you can handle. • Envoy supports request limits on a per-cluster basis. • Envoy also supports two priority groups, allowing you to save slots for important traffic.
  • 15. Without Circuit Breakers Failures are fine, but 99% latency is slowwwwww as requests just back up Also, POST requests are totally offline because we’re swamped with GETs
  • 16. Without Circuit Breakers Retries overflow, which is slow
  • 17. With Circuit Breakers Gobs of failures, but p99 latency is s5ll good. Also, POST requests are available. Also, we told clients to back off with the x-envoy-overloaded response header.
  • 18. With Circuit Breakers Pending requests overflow, which is fast!
  • 19. An Overview of Rou5ng • Endpoint metadata for richer rou5ng primi5ves • Probabilis5c distribu5on of traffic across mul5ple clusters • 1% of traffic to my-great-rewrite, 99% to legacy • 1% of traffic to v2 of my service, 99% to v1 • Header based rou5ng to cluster subsets • If “x-canary” is set route to endpoints with a version label of v2 • Priority rou5ng, which we saw in the circuit breaking example • Zone aware rou5ng
  • 20. Traffic Shioing the Hard Way • Mul5ple clusters • Mul5ple routes
  • 21. Header-based Canary • When we specify the canary header, the route matches and we (and only we) are routed to service1a • When header is not present, the route doesn’t match and we go on to the next route, sending traffic to service1
  • 22. Probabilis5c Rollout • With the run5me match, we choose this route 25% of the 5me, sending 25% of our traffic to service1
  • 23. Traffic Shioed 25% of traffic to service1a
  • 24. The Easy Way • Restar5ng servers on every config change is tedious in this demo. • It’s even more tedious in produc5on. • Envoy provides a beYer way—the xDS APIs.
  • 25. xDS APIs • CDS - discover clusters, which are logical groupings of endpoints. • A cluster defini5on can have a reference to an EDS endpoint • EDS - discover endpoints for a cluster. • LDS - discover listeners for an Envoy • A listener’s filter chain can have a reference to an RDS endpoint • RDS - discover routes for a filter chain
  • 26. Dynamic Config • The xDS APIs give you a central point-of-control to manage a fleet of Envoys • Bridge service discovery (e.g. from Kubernetes) to Envoy • Bridge rou5ng config (e.g. from Houston) to Envoy
  • 27. Advanced Rou5ng with EDS • CDS (cluster discovery service) defines groups of endpoints. • EDS (endpoint discovery service) discovers the actual endpoints for clusters. • EDS allows you to aYach metadata to an endpoint. • Our mul5-cluster example can be collapsed to a metadata based approach on a single cluster.
  • 28. Even Easier with Houston • An CDS/EDS server with integra5ons to EC2, ECS, Kubernetes, Consul, DC/OS, or JSON files • An LDS/RDS server with an intui5ve route configura5on UI • Stats parsing, forwarding, and change tracking