Service Mesh

Definition

A service mesh is a dedicated infrastructure layer that handles service-to-service communication in a microservices architecture. It manages traffic control, observability, and security between services without requiring changes to application code.

The most common implementation uses sidecar proxies — lightweight proxies deployed alongside each service instance.

How It Works

Service A Pod
├── App Container (business logic)
└── Sidecar Proxy (Envoy/Istio) — intercepts all inbound/outbound traffic

Service B Pod
├── App Container (business logic)
└── Sidecar Proxy (Envoy/Istio)

Traffic flows: App A → Sidecar A → Network → Sidecar B → App B
                  ↑ all traffic intercepted and controlled

Service Mesh Capabilities

Capability Description
Traffic Management Routing, canary, blue-green, A/B testing
Observability Distributed tracing, metrics, logs
Security mTLS, authentication, authorization
Resilience Retries, timeouts, circuit breakers, rate limiting
Policy Enforcement Consistent policies across all services
Mesh Provider Notes
Istio Google/IBM/Lyft Most popular, feature-rich
Linkerd Buoyant Lightweight, simple, CNCF graduated
Consul Connect HashiCorp Integrates with Consul service discovery
App Mesh AWS AWS-specific, Envoy-based
OpenServiceMesh Linux Foundation Multi-cluster, CNCF sandbox
Kuma Kong Multi-mesh, Kubernetes + VM support

Service Mesh vs API Gateway

Feature Service Mesh API Gateway
Scope Internal service-to-service External client-to-service
Layer L4-L7 within cluster L7 at edge
Traffic East-west (internal) North-south (external)
Use case Microservices communication External API management
Example Istio, Linkerd Kong, AWS API Gateway

Service Mesh Trade-offs

Pros:

  • Transparent to application code
  • Consistent policies across all services
  • Built-in observability
  • mTLS for service authentication

Cons:

  • Added complexity and operational overhead
  • Performance overhead from sidecar proxies
  • Learning curve for configuration
  • Not needed for small deployments (< 10 services)
  • Kubernetes — proxy used by Istio, App Mesh
  • mTLS — mutual TLS for service authentication
  • Microservices — service mesh for microservices communication
  • Distributed Tracing — observability in service mesh
  • API Gateway — complements service mesh (external vs internal)

References