Scheduling & Resources

Scope: the core-concepts page introduced the scheduler as the control-plane piece that assigns a Pod to a node. This page goes one layer deeper on how it decides, what requests and limits actually control, and the knobs (nodeSelector, affinity, taints/tolerations, topology spread) that steer or restrict that decision.

TL;DR — in 30 seconds:

• The scheduler only looks at requests when deciding if a Pod fits a node; limits are enforced later by the node itself at runtime — skip requests and the Pod can land anywhere with no reserved floor.
• QoS class (Guaranteed/Burstable/BestEffort, derived from requests vs limits) decides eviction order under node pressure — request = limit on every container is what earns Guaranteed.
• Placement has two directions: a Pod pulls itself toward a node (nodeSelector/affinity); a node pushes Pods away (taints), and a Pod needs a matching toleration to land there anyway.

1. Requests and limits

A Pod sits Pending for ten minutes while three other nodes in the cluster have gigabytes of memory to spare. Nothing is broken — the scheduler is honoring a number set on the container weeks ago and since forgotten. Every container can declare two numbers per resource (CPU, memory):

Field	Meaning	What happens if you skip it
request	The amount the scheduler reserves for this container when picking a node — a guaranteed floor	Scheduler treats it as effectively zero; the Pod can land anywhere, and offers no floor to protect it under contention
limit	The hard ceiling the container may use at runtime	Container can use as much of the node's spare capacity as exists, with no ceiling

The scheduler only ever looks at requests to decide whether a node has room for a Pod — it sums the requests of everything already running on a node and checks the candidate Pod's requests fit in what's left. Limits are enforced later, by the node itself at run time, not by the scheduler.

• Exceeding a memory limit gets the container killed (OOMKilled) — memory can't be reclaimed gracefully, so the kubelet terminates it.
• Exceeding a CPU limit does not kill anything — the container is throttled, its usage capped for the remainder of that scheduling period, since CPU is a time-shareable resource.

2. QoS classes

Kubernetes derives a Quality of Service class for every Pod from how its requests and limits compare, and uses it to decide which Pods get evicted first when a node runs low on resources:

Class	Condition	Eviction priority under node pressure
Guaranteed	Every container sets request = limit for both CPU and memory	Evicted last
Burstable	At least one container sets a request, but not every container meets the Guaranteed condition	Evicted after Guaranteed, before BestEffort
BestEffort	No container sets any request or limit	Evicted first

This is why a production workload that needs to survive node pressure sets request = limit: it's not just about performance, it changes the Pod's standing in the eviction order — the Pod Quality of Service classes docs walk through the exact derivation rule per resource.

3. Steering placement: nodeSelector and affinity

Three mechanisms narrow or bias which nodes a Pod can land on, from simplest to most expressive:

• nodeSelector — the blunt tool: a Pod only lands on a node whose labels match a fixed key/value map, exactly. No "prefer" option, no partial match.
• Node affinity — the same idea, but with a real expression language (In, NotIn, Exists, ranges) and two strengths: requiredDuringSchedulingIgnoredDuringExecution (a hard rule, like nodeSelector but richer) and preferredDuringSchedulingIgnoredDuringExecution (a soft rule — the scheduler tries to honor it but still places the Pod if no node matches).
• Pod affinity / anti-affinity — the rule targets other Pods' labels instead of node labels: "schedule me on the same node/zone as Pods matching this label" (affinity) or "never on the same node/zone as Pods matching this label" (anti-affinity). Anti-affinity between replicas of the same Deployment is the standard way to spread them across failure domains instead of letting them stack on one node.

4. Taints and tolerations

Where nodeSelector/affinity are things a Pod declares to pull itself toward a node, a taint is the opposite direction: something a node declares to push Pods away.

A taint on a node (key=value:effect) repels every Pod, unless that Pod carries a matching toleration that says "I'm willing to tolerate this taint." The effect controls how strongly: NoSchedule (don't place new Pods here), PreferNoSchedule (avoid if possible), and NoExecute (evict Pods already running here that don't tolerate it, in addition to blocking new ones).

The common uses are dedicating nodes (a taint that only a specific team's tolerated Pods can land on) and marking a node unhealthy (the node controller taints a node NoExecute when it stops reporting, evicting Pods so they reschedule elsewhere).

Note. Affinity and taints solve different problems: affinity is a Pod asking to go somewhere; taints are a node refusing Pods that don't explicitly tolerate it. A Pod can satisfy a node's affinity rule and still be rejected by an untolerated taint on that same node.

flowchart LR
  pod["Pod"] -->|"nodeSelector / affinity<br/>(Pod asks to go somewhere)"| node["Node"]
  node -->|"taint<br/>(Node refuses Pods)"| untol["Pod with no matching<br/>toleration → rejected"]
  node -->|"taint, tolerated"| tol["Pod with a matching<br/>toleration → allowed"]

5. Topology spread constraints

Anti-affinity keeps replicas apart, but expressing "spread evenly across all 3 zones" with anti-affinity alone is awkward. Topology spread constraints say this directly: given a topologyKey (e.g. a zone label) and a maxSkew, the scheduler keeps the difference between the most- and least-populated domains within that bound — evenly distributing Pods across zones or nodes without hand-writing pairwise anti-affinity rules.

6. How the scheduler decides: filter, then score

Putting it together, the scheduler processes each unscheduled Pod in two phases:

flowchart LR
  pod["Unscheduled Pod"] --> filter["Filtering<br/>(drop nodes that fail a hard rule)"]
  filter --> score["Scoring<br/>(rank the survivors)"]
  score --> bind["Bind to<br/>highest-scoring node"]

  subgraph filterRules["Filtering considers"]
    f1["enough requested<br/>CPU/memory free"]
    f2["required node affinity"]
    f3["tolerated taints"]
  end
  subgraph scoreRules["Scoring considers"]
    s1["preferred affinity"]
    s2["topology spread"]
    s3["resource balance<br/>across nodes"]
  end
  filterRules -.-> filter
  scoreRules -.-> score

Filtering removes every node that fails a hard requirement — not enough free requested capacity, a required node-affinity rule that doesn't match, or an untolerated taint. Scoring then ranks the remaining candidates by soft preferences — preferred affinity, topology spread, how evenly resource usage would balance across the cluster — and the Pod binds to the top-scoring node.

7. How this connects

• This extends the core-concepts page's scheduler mention — that page says the scheduler "picks a node"; this page covers the two-phase filter→score mechanism and every knob that feeds it.
• QoS-driven eviction is the scheduling-side half of the story the observability/debugging page's get→describe→logs→events loop investigates from the running-Pod side (a Pod stuck Pending is usually a filtering failure; an evicted Pod is usually a QoS/eviction event).
• Taints are also how a cluster marks a node unhealthy — the same signal that autoscaling (the autoscaling page) reacts to when deciding to replace a node.

Common gotchas

• Skipping resource requests "to keep it simple." Fix: an unset request isn't neutral — the scheduler treats it as zero (no floor under contention) and the Pod becomes BestEffort, first in line for eviction under node pressure.
• Expecting a CPU limit breach to kill the container like a memory limit breach does. Fix: exceeding a memory limit gets the container OOMKilled (memory can't be reclaimed gracefully); exceeding a CPU limit just throttles it for the rest of that period — nothing dies.
• Assuming nodeSelector/affinity and taints solve the same problem. Fix: they're opposite directions — affinity is a Pod asking to go somewhere, a taint is a node refusing Pods that don't tolerate it; a Pod can satisfy a node's affinity rule and still be rejected by an untolerated taint on that same node.
• Setting request = limit without realizing it also changes eviction priority. Fix: it's not just a performance choice — it's what earns the Guaranteed QoS class, evicted last under node pressure; a Pod with no requests at all is BestEffort and evicted first.

Check yourself — a Pod sits Pending even though three nodes have plenty of free memory. What's the likely cause?

The scheduler only counts a node's free capacity against Pod requests, not actual usage — if the Pod's request (or the sum of requests already on candidate nodes) doesn't leave room, it stays Pending regardless of how much memory is genuinely idle. Check the Pod's requests and what's already reserved (not used) on each node.

Check yourself — under node pressure, which gets evicted first: a Pod with request=limit on every container, or a Pod with no requests or limits set at all?

The one with no requests/limits — that's BestEffort, evicted first. request=limit on every container earns Guaranteed, evicted last; QoS class (not just "how much it's using") drives eviction order.

You can defend this when you can explain why the scheduler only looks at requests (not limits) when placing a Pod, say what happens differently when a container exceeds a CPU limit versus a memory limit, rank the three QoS classes by eviction order and say what determines each, and explain the difference between a Pod's affinity rule and a node's taint.