Service Mesh¶

Advanced (roadmap.sh marks this Advanced — reach for it once the service count and traffic complexity justify the operational cost, not as a default addition to every cluster). · prereq 04 (Orchestration · Kubernetes & Helm), 17 (Observability & Monitoring) — a mesh's sidecars run inside a Kubernetes cluster, and its per-call telemetry feeds the same observability stack module 17 (Observability & Monitoring) already covers. · ~1 day (4 reference submodules — what a mesh is, mTLS/traffic/resilience, Istio & Linkerd & Envoy, and observability/cost). Reality check: this is the 80/20 on-ramp; real fluency (running Istio or Linkerd against production traffic, tuning its policies) is built on the job.

Module 04 (Kubernetes) gets pods talking to each other; it doesn't decide how that traffic is encrypted, split for a canary release, retried on failure, or observed per-call. A service mesh is the layer that adds all of that — uniformly, across every service — without touching application code.

TL;DR — in 30 seconds:

A mesh injects a sidecar proxy into every pod; all network traffic flows through it instead of directly between application containers, giving a data plane (the proxies) configured centrally by a control plane.
That buys three things for free, app-code-untouched: mTLS (automatic, mutual, service-to-service encryption + identity), traffic management (canary/A-B routing by rule), and resilience (retries, timeouts, circuit breaking).
Istio (Envoy sidecars + istiod control plane) and Linkerd (its own lightweight micro-proxy) are the two dominant meshes — feature breadth versus operational simplicity.
A mesh also gives uniform per-call observability, but it isn't free — extra latency, sidecar resource overhead, and a control plane to operate. It earns its cost at real service count and traffic complexity, not by default.

Learning objectives¶

By the end, the junior can:

Explain the sidecar/data-plane/control-plane model and why it delivers uniform networking behavior without application code changes.
Explain what mTLS adds beyond ordinary TLS, and describe one traffic-management use case (canary release) and one resilience use case (circuit breaking) a mesh provides.
Compare Istio and Linkerd at a concept level, and name what Envoy is versus what a full mesh adds on top of it.
Explain what per-call observability a mesh provides, and weigh it against the mesh's real costs (latency, resource overhead, operational complexity) to judge when adopting one is and isn't worth it.

Do it with Claude

Pair, Do and Prove run in Claude Code, with the onboarding-tutor skill. Click a button to copy the exact prompt, then paste it into Claude Code in this repo.

1. Learn · acquire the concept¶

No external course — read this brief, then go deeper in the four reference pages below.

Mental model (20 minutes):

flowchart LR
    subgraph "Pod A"
        AppA["App container"] <-->|"localhost"| ProxyA["Sidecar proxy"]
    end
    subgraph "Pod B"
        AppB["App container"] <-->|"localhost"| ProxyB["Sidecar proxy"]
    end
    ProxyA <-->|"mTLS, retries,<br/>traffic rules, metrics"| ProxyB
    CP["Control plane<br/>(e.g. Istio's istiod)"] -.->|"configures"| ProxyA
    CP -.->|"configures"| ProxyB

Sidecar proxies form the data plane — every pod gets a proxy that intercepts its traffic; a service's request to another service actually flows sidecar-to-sidecar.
A control plane configures the fleet — operators set routing rules, security policy, and telemetry config once, centrally; it pushes to every sidecar rather than each being hand-configured.
Three behaviors come for free once traffic flows through the sidecar: encryption + identity (mTLS), rule-based routing (traffic management), and failure handling (retries/timeouts/circuit breaking) — uniformly, for every service, with zero app-code changes.
The payoff isn't free — an extra network hop, a sidecar's resource footprint per pod, and a control plane to operate are the real cost that has to be worth paying.

Go deeper: What a service mesh is · mTLS, traffic management, and resilience · Istio, Linkerd, and Envoy · Observability and when a mesh is worth it

Check yourself — a team asks whether they need to add a retry library to every service now that the cluster has a mesh. What's the answer?

No — retries (along with timeouts and circuit breaking) are handled by the sidecar proxy at the network layer, uniformly for every service in the mesh, with no per-service library or code change needed.

Check yourself — what's the difference between Envoy and a service mesh like Istio?

Envoy is a general-purpose proxy — the data-plane piece that runs as a sidecar. A mesh also needs a control plane (Istio's istiod) to configure routing, security, and telemetry across every sidecar centrally; Envoy alone, hand-configured, isn't yet "a mesh."

Done when: you can explain, in order, why the sidecar model needs no app-code changes, what three behaviors it buys, and why the payoff has a real operational cost — before moving to Pair.

2. Pair · passive → active¶

Drive the onboarding-tutor:

"Give me a Kubernetes deployment with no mesh and make me name exactly what a sidecar proxy would take over — traffic in, traffic out, and what stays the application's job."
"Quiz me on mTLS versus ordinary TLS, and make me explain why 'mutual' matters for service-to-service trust."
"Give me a canary-release scenario and make me describe it as a mesh traffic-management rule, not an application-level feature flag."
"Walk me through a downstream dependency failing intermittently, and make me explain what the mesh does (retry, timeout, circuit break) versus what would happen with no mesh at all."
"Quiz me on Istio vs. Linkerd, and make me justify a choice for a specific team size and need rather than picking the 'more popular' one by default."
"Give me a small, low-traffic service set and make me argue whether adopting a mesh is worth its cost — and defend the answer either way."

Common gotchas

Assuming the mesh replaces Kubernetes networking. Fix: it runs on top of the cluster's own pod-to-pod networking, adding a policy/observability layer above it — not replacing the CNI.
Stacking application-level retries on top of mesh-level retries. Fix: that can multiply load during an incident (a retry storm) — pick one layer to own retries for a given call path.
Adopting a mesh "because it's best practice" with no concrete traffic-management, security, or observability gap to close. Fix: name the specific gap; if there isn't one, the operational cost (latency, sidecar overhead, another control plane) isn't buying anything yet.
Treating Envoy alone as "a mesh." Fix: Envoy is the data-plane proxy; a mesh needs a control plane configuring it fleet-wide too.

Done when: you can defend a mesh adoption decision (yes or no) for a specific scenario using its actual costs and benefits, without notes.

3. Do · produce an artifact¶

Exercise — a service mesh adoption memo:

Pick a set of services from an earlier module (e.g. something deployed in module 04 (Kubernetes)) as your subject.
Write service-mesh-memo.md stating whether you'd adopt a mesh for this set of services right now — yes or no — and name the specific traffic-management, security (mTLS), or observability gap that justifies the answer either way.
If yes: name which mesh (Istio or Linkerd) and one concrete reason for that choice over the other.
Add a section naming the three real costs (latency, sidecar resource overhead, control-plane operational complexity) and how you'd justify them against the benefit you named in step 2.

Done when: service-mesh-memo.md states a clear yes/no with a specific justifying gap (not "best practice"), and — if yes — names a specific mesh with a specific reason.

4. Prove · understanding gate¶

Mandatory. Pass bar in the Socratic gate.

Sidecar model: Why does the sidecar/data-plane/control-plane split let a mesh add mTLS, traffic management, and resilience with zero application code changes?
mTLS: What does "mutual" add beyond ordinary TLS, and why does that matter for service-to-service trust?
Choosing a mesh: Name one concrete factor that would push a team toward Istio and one that would push toward Linkerd.
Cost/benefit: Name the three real costs of adopting a mesh, and describe a scenario where the benefit doesn't justify them.
Verification: Which specific claim in your memo did the tutor suggest, and how did you verify it was correct — by checking documented Istio/Linkerd behavior, not by trusting the explanation?

Pass bar: you can defend every choice in your memo by the specific guarantee or cost it involves — not just "it's best practice" — and show how you verified a tutor-suggested detail against real documentation. "It works but I can't say why" = back to Pair.

5. Retain · fight forgetting¶

This module has no generated Anki deck yet. Recall by re-reading the four reference pages and re-running the Do exercise's memo against a different service set until you can name the sidecar model, the three free behaviors, and the mesh's real costs from memory, unprompted. Done when you can name all three, cold, for any service set someone hands you.

Key takeaways¶

Sidecar proxies (data plane) + a central control plane let a mesh add networking behavior to every service uniformly, with zero application code changes.
The mesh gives three things for free: mTLS (mutual encryption + identity), traffic management (canary/rule-based routing), and resilience (retries, timeouts, circuit breaking).
Istio (Envoy sidecars + istiod) trades operational complexity for feature breadth; Linkerd (its own micro-proxy) trades feature breadth for operational simplicity. Envoy alone is a proxy, not a mesh.
A mesh also gives uniform per-call observability, feeding the same stack as module 17 (Observability & Monitoring).
The payoff has a real cost — latency, sidecar resource overhead, control-plane complexity — that has to be justified by actual service count and traffic-management/security need, not adopted by default.

Go deeper — curated resources¶

Hand-picked external material to take this topic further — the best books, courses, talks, and writing. Cost is tagged Free, Free online (full text/course free on the web), or Paid.

Docs · Istio documentation — Istio project. The canonical reference for Istio's architecture, traffic management, and security model. Free.
Docs · Linkerd documentation — CNCF / Linkerd project. The canonical reference for Linkerd's architecture and its lightweight-by-design approach. Free.
Docs · Envoy proxy documentation — Envoy project (CNCF). The canonical reference for the proxy underlying Istio and several other meshes. Free.

Gate log (mentor fills on pass)¶

Date passed:
Passed by: / checked by:
Weak spots noticed (feeds system improvement):