Persistent volumes

Everything you have run so far has been disposable. Delete the Pod and its filesystem goes with it, which is fine for a web server and fatal for a database.

Kubernetes splits the problem in two, and the split is the idea: a PersistentVolumeClaim is what you ask for — “1Gi, read-write, one node” — and a PersistentVolume is what the cluster hands you, with a lifecycle of its own. Something has to turn the first into the second, and that is normally a StorageClass.

Things You’ll Need

kubectl: The Kubernetes command-line tool to interact with the cluster and manage Pods.
Text Editor: a basic editor like Vim or Nano, or the VS Code already open in your workspace.
A Kubernetes cluster: your own, reached with the kubeconfig in ~/.kube/config. You are the administrator of it, so you can create storage classes and inspect volumes directly.

Reference

Terminology

PersistentVolume (PV) — A cluster-level piece of storage, provisioned by an administrator or dynamically on demand. It exists independently of any Pod, and survives deleting the workload that was using it.
PersistentVolumeClaim (PVC) — A request for storage by a workload. Kubernetes binds a claim to a matching volume by capacity and access mode, and the workload only ever references the claim.
StorageClass — Describes a kind of storage and the provisioner that creates it. When a claim names a class, the cluster provisions a volume on the spot. One class is usually marked default, so a claim that names no class gets that one.
Dynamic provisioning — Creating the PV automatically in response to a claim, instead of an administrator pre-creating volumes by hand.
accessModes — How a volume may be mounted. ReadWriteOnce (RWO) is read-write by a single node, ReadOnlyMany (ROX) is read-only from many, ReadWriteMany (RWX) is read-write from many. Note that these are node level: several Pods on the same node can share an RWO volume.
persistentVolumeReclaimPolicy — What happens to the volume when its claim is deleted. Retain keeps the volume and its data for manual recovery; Delete removes both. For a dynamically provisioned volume this is inherited from the StorageClass.
volumeBindingMode — Immediate provisions as soon as the claim is created; WaitForFirstConsumer waits until a Pod actually uses it, so the volume can be created in the same zone as the Pod.
allowVolumeExpansion — A StorageClass setting that permits growing a claim after it is bound. Without it, a claim can only be made smaller by starting again.
VolumeSnapshot — A point-in-time copy of a claim’s data, created against a VolumeSnapshotClass in the same way a PV is created against a StorageClass.

The Model

Pod ──mounts──▶ PVC ──bound to──▶ PV ──backed by──▶ real storage
                 │                │
           "what I want"    "what I got"
                 └── StorageClass decides how the PV is created,
                     and what happens to it when the claim goes away

Three consequences worth holding onto, because they cause most storage incidents in production:

Deleting a Pod does not touch the data. Deleting the claim might, depending on the reclaim policy.
A claim only helps if you write to it. Data written anywhere else in the container is in the container filesystem and vanishes with the Pod.
Pending means “waiting”, not “broken”. Nothing has given the claim a volume yet, and the reason is in the claim’s events rather than its spec.

Claim It and Keep It

This is Lab 04. Set up the starting state first — a namespace lab-04 containing a Pod named app with no storage at all.

labctl start 04

Every command below uses -n lab-04 explicitly. If you would rather not repeat it, point your context at the namespace once:

kubectl config set-context --current --namespace=lab-04

Ask for storage. Leave storageClassName out so the cluster’s default class applies — naming a class that does not exist is the most common way to get a claim stuck:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Apply it and watch it bind. You are looking for Bound:
```
kubectl -n lab-04 apply -f pvc.yaml
kubectl -n lab-04 get pvc data -w
```
If it stays Pending, jump to Why a Claim Stays Pending before going on.

Mount it in the Pod. The Pod needs two halves: a volume that names the claim, and a volumeMount that names a path inside the container.

apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 101
    runAsGroup: 101
    fsGroup: 101
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: docker.io/nginxinc/nginx-unprivileged:1.27-alpine
      command: ["sleep", "infinity"]
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: data

The securityContext is not decoration, and it is worth understanding before you copy it. Without it this Pod is rejected outright by any namespace enforcing the restricted Pod Security Standard; the unprivileged nginx image is what makes running as a non-root user possible at all; and fsGroup is what lets that non-root user write to a mounted volume. Those three things appear together in every well-behaved workload you will meet, and they are covered properly on Day 4.

Pod specs are immutable where volumes are concerned, so replacing the Pod the lab created means deleting it first:

kubectl -n lab-04 delete pod app
kubectl -n lab-04 apply -f pod.yaml
kubectl -n lab-04 get pod app -o wide

Prove the data outlives the Pod. This is the part that matters — write a file into the volume, destroy the Pod, bring it back, read the file:
```
kubectl -n lab-04 exec app -- sh -c "echo persistence works > /data/keepme.txt"

kubectl -n lab-04 delete pod app
kubectl -n lab-04 apply -f pod.yaml
kubectl -n lab-04 wait --for=condition=Ready pod/app --timeout=90s

kubectl -n lab-04 exec app -- cat /data/keepme.txt
```
If the file is gone, the volume is not where you think it is. Check that the volumeMount is on the container rather than only on the Pod, and that the path matches the one in step 3.
Check your work, then reset when you are done experimenting:
```
labctl check 04
labctl reset 04
```
labctl check 04 reports each checkpoint separately. When one fails it prints labctl hint 04.2 — asking twice gives a more specific hint, and labctl solution 04 shows the reference manifests if you would rather read the answer.

Why a Claim Stays Pending

Almost always one of these. The events at the bottom of describe name it:

kubectl -n lab-04 describe pvc data

What `describe` says	What it means	What to do
`no persistent volumes available for this claim and no storage class is set`	There is no default StorageClass, so nothing provisioned one	Name a class in `storageClassName`, or ask the platform team to mark one default
`storageclass.storage.k8s.io "x" not found`	The named class does not exist — usually a typo	Correct the name, or remove the field to use the default
`waiting for first consumer to be created before binding`	The class uses `WaitForFirstConsumer`	Nothing is wrong. Create the Pod that uses the claim
`requested storage is too large` / capacity errors	No volume of that size can be provisioned	Ask for less
No events at all	The provisioner may be unhealthy	`kubectl get storageclass`, then look at the provisioner’s own Pods

Access Modes and Reclaim Policies

What your claim asked for, and what it got:

kubectl -n lab-04 get pvc data -o jsonpath='{.spec.accessModes}{"\n"}{.spec.resources.requests.storage}{"\n"}'
kubectl -n lab-04 get pvc data -o jsonpath='{.spec.volumeName}{"\n"}'   # the PV it bound to

Access mode	Short	Meaning at the node level
`ReadWriteOnce`	RWO	Read-write, mounted by one node at a time. The usual choice for a database
`ReadOnlyMany`	ROX	Read-only, many nodes
`ReadWriteMany`	RWX	Read-write, many nodes. Needs shared storage — on this platform, Azure Files rather than Azure Disk

Reclaim policy	When the claim is deleted
`Delete`	The volume and its data are removed. The default for dynamically provisioned volumes
`Retain`	The volume survives, `Released`, and keeps the data until someone reclaims it by hand
`Recycle`	Deprecated. Do not use it

RWO is a node level guarantee, not a Pod level one: two Pods scheduled onto the same node can both mount the same RWO volume read-write, while a Pod on a second node cannot. That is why a Deployment with a single RWO volume does not scale beyond the node it landed on.

Going Further on the Shared Cluster

Two things are deliberately out of scope for the lab, because they depend on the storage driver rather than on Kubernetes:

Growing a claim. Requires allowVolumeExpansion: true on the class and a driver that supports it. You edit spec.resources.requests.storage and the volume grows in place.
Snapshots. A VolumeSnapshotClass plus a VolumeSnapshot, then a new claim that restores from it with dataSource. The useful question is not how to take one, but whether restoring it actually gives you your data back.

Both are done in the afternoon session on the shared cluster, where the real Azure Disk and Azure Files classes live and you can author a StorageClass of your own. The reason is worth stating plainly: in your own cluster you are the administrator, but the storage driver belongs to the platform. You can create classes and inspect volumes; you cannot install a new driver. That is exactly the split you will meet in production, and knowing which side of it you are standing on saves a great deal of time.

What to Take Away

A claim is a request; a volume is the answer; a class is the policy that connects them.
Pending is information, not failure — read the events.
Data lives in the volume only if something writes to the mounted path.
Access modes and reclaim policies are the two settings that decide whether your workload can scale and whether your data survives a mistake.

Ephemeral volumes Other volume types