Disruptions

Sooner or later every Pod is going to be removed. The question is whether you did it, and whether your application noticed.

Kubernetes splits this into two categories, and the distinction is worth learning precisely because it decides what you can protect against.

Involuntary disruption — a node fails, runs out of memory, or is reclaimed by the cloud provider. Nothing you configure prevents it. It is what replicas and spreading are for.
Voluntary disruption — draining a node for an upgrade, scaling a node pool down, or deleting a Pod yourself. This you can control, and it is what a PodDisruptionBudget governs.

Reference

Terminology

PodDisruptionBudget (PDB) — A rule about how many Pods of a set may be unavailable at the same time during voluntary disruption. It does not keep Pods running; it makes their removal wait.
Eviction API — The API that asks for a Pod to be removed politely. It consults PDBs. kubectl drain uses it; kubectl delete pod does not.
minAvailable — How many Pods must remain available, as a number or a percentage.
maxUnavailable — The complement: how many may be missing at once. Set one or the other, never both.
Topology spread constraint — A rule about how replicas are distributed across zones or nodes, so that losing one node does not take the whole service with it.

Why the Eviction API Exists

Deleting a Pod is immediate and unconditional. Draining a node is neither — it is a request to move workloads elsewhere, and it should be refused if honouring it would take a service below its budget.

kubectl drain node-1
        │
        ├── for each Pod on the node: POST /eviction
        │         │
        │         └── PDB consulted: would this breach the budget?
        │                   ├── no  → the Pod is removed
        │                   └── yes → the eviction is refused, and retried
        └── the drain blocks until it succeeds, or you give up

That refusal is the feature. A drain that hangs is a drain telling you that upgrading this node right now would take your service below the availability you said you needed.

A PDB, and the Mistake Everyone Makes

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: web

With three replicas and minAvailable: 2, exactly one Pod may be taken at a time:

kubectl get pdb web
# NAME   MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
# web    2               N/A               1                     10s

ALLOWED DISRUPTIONS is the column to read. If it is 0, nothing voluntary can move — and if it stays 0 forever, your node drains will hang indefinitely.

The two ways to get stuck at zero:

minAvailable equal to the replica count, so no Pod may ever be removed.
A selector that matches no Pods at all — the budget is trivially satisfied for a set that does not exist, and ALLOWED DISRUPTIONS stays at zero while your real Pods are unprotected.

Both look like a healthy PDB in a listing. Check the number of Pods the selector actually matches, not just that the object exists.

Exercises

Create a Deployment and a PDB, and read the budget.
```
kubectl create deployment web --image=docker.io/library/nginx:1.27 --replicas=3
kubectl expose deployment web --port=80
```
Then apply a PDB with minAvailable: 2 and kubectl get pdb web. Note ALLOWED DISRUPTIONS: 1. Scale the Deployment to 2 and watch it become 0.
See the difference between deleting and evicting. Reduce to two replicas with minAvailable: 2 — nothing may be evicted — then:
```
kubectl delete pod <one-of-the-pods>          # succeeds immediately; the PDB is not consulted
```
A replacement appears, but your budget was not respected. kubectl delete is not a graceful operation, and knowing that is the point.
Test the drain behaviour, if your cluster exposes nodes.
```
kubectl get nodes
kubectl drain <node> --ignore-daemonsets --delete-emptydir-data
kubectl get pdb web -w          # ALLOWED DISRUPTIONS, and the drain waiting
kubectl uncordon <node>
```
Depending on how your cluster presents its nodes, this may not be available to you — in which case you will see the full drain demonstration on the shared cluster. Either way, the object to watch is ALLOWED DISRUPTIONS.
Break it deliberately. Set minAvailable: 3 with three replicas and try to drain. The drain hangs. This is the classic self-inflicted outage during a cluster upgrade, and it is much better to meet it here than at two in the morning.

What to Take Away

Involuntary disruption is survivable only through replicas and spread; no budget helps.
A PDB governs voluntary disruption, and only through the Eviction API — kubectl delete ignores it.
Read ALLOWED DISRUPTIONS, and watch out for 0 caused by an over-strict budget or a selector matching nothing.
A hanging drain is information, not a failure.

DaemonSets