All writing
Updated 4 min readNaren

Persistent storage: volumes, PVs, PVCs and StorageClasses

A container loses its files when it stops. Volumes, PersistentVolumes, PersistentVolumeClaims and StorageClasses are how Kubernetes keeps data that has to outlive it.

KubernetesInfrastructure

The last post secured applications with network policies, controlling how Pods communicate. This one is about data.

By default, containers and Pods are ephemeral. If a Pod restarts, crashes or is deleted, any data stored in its container's filesystem is lost. That is fine for stateless applications, but databases, user uploads and application logs all need to survive the Pod that wrote them.

This post covers how Kubernetes gives Pods persistent data: Volumes, PersistentVolumes (PVs), PersistentVolumeClaims (PVCs) and StorageClasses.

Why storage needs its own abstraction

Pods and containers are ephemeral by design. It is what lets Kubernetes scale them and recover from failures, but it is a problem for stateful applications: when a database Pod restarts, you can't lose the database.

Kubernetes solves this by separating storage from compute, so you can attach many kinds of storage to your Pods.

Volumes

A volume is a directory that the containers in a Pod can access. You declare it in the Pod's configuration and mount it into the container's filesystem at a mountPath.

Unlike the container's own temporary filesystem, data in a volume survives a container restart. But a volume's lifecycle is tied to the Pod it belongs to, and for most volume types, deleting the Pod deletes the data too.

Defining and mounting a volume

A Pod that uses a volume needs two sections in its YAML:

  1. spec.volumes defines the volumes available to the Pod. Each one needs a name and some configuration specific to its type.
  2. spec.containers[*].volumeMounts goes in each container that needs a volume. It names the volume to mount and the mountPath inside the container.

Common volume types

emptyDir, ConfigMap and Secret volumes are ephemeral: they exist only as long as the Pod does. hostPath is the exception, because its files live on the node.

emptyDir, for scratch space

An emptyDir volume is created, empty, when a Pod is assigned to a node, and it exists as long as that Pod runs on that node. When the Pod is removed, the data is deleted. Use it for temporary scratch space, caching, logging, or any application that needs a temporary directory while it runs.

By default it uses the node's disk. Set medium: Memory for a RAM-backed filesystem (tmpfs), which is faster, but its data is lost when the node reboots.

apiVersion: v1
kind: Pod
metadata:
  name: my-emptydir-pod
spec:
  containers:
  - name: my-container
    image: busybox
    command: ["/bin/sh", "-c", "echo 'Hello from emptyDir' > /app/data/hello.txt && sleep 3600"]
    volumeMounts:
    - name: my-temp-storage
      mountPath: /app/data
  volumes:
  - name: my-temp-storage
    emptyDir: {}

hostPath, for files on the node

A hostPath volume mounts a file or directory from the node's own filesystem into your Pod. It is for containers that need node-specific files, such as system logs or the Docker socket.

Avoid it in production on multi-node clusters. It ties your Pod to one node, and if the Pod is rescheduled onto another node, the data does not follow it.

apiVersion: v1
kind: Pod
metadata:
  name: my-hostpath-pod
spec:
  containers:
  - name: my-container
    image: busybox
    command: ["/bin/sh", "-c", "echo 'Hello from hostPath' > /mnt/data/hello.txt && sleep 3600"]
    volumeMounts:
    - name: host-data
      mountPath: /mnt/data
  volumes:
  - name: host-data
    hostPath:
      path: "/tmp/hostpath-data" # This path must exist on the host node
      type: DirectoryOrCreate # Creates the directory if it doesn't exist

ConfigMap, for configuration

A ConfigMap volume injects non-sensitive configuration data into a Pod as files. Use it for application configuration files, environment variables or command-line arguments.

First create the ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-app-config
data:
  config.properties: |
    app.name=MyApplication
    app.version=1.0
  log_level: INFO

Then use it in a Pod:

apiVersion: v1
kind: Pod
metadata:
  name: my-configmap-pod
spec:
  containers:
  - name: my-container
    image: busybox
    command: ["/bin/sh", "-c", "cat /etc/config/config.properties && echo Log Level: $(cat /etc/config/log_level) && sleep 3600"]
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
  volumes:
  - name: config-volume
    configMap:
      name: my-app-config

Secret, for sensitive data

A Secret volume works like a ConfigMap volume, but it is meant for sensitive data such as passwords and API keys. Kubernetes handles Secrets more carefully than ConfigMaps, but they are not encrypted at rest unless you configure that. Use them to give applications credentials and other sensitive information.

First create the Secret. Its values are base64 encoded:

apiVersion: v1
kind: Secret
metadata:
  name: my-db-secret
type: Opaque
data:
  username: YWRtaW4= # "admin" base64 encoded
  password: c2VjcmV0cGFzc3dvcmQ= # "secretpassword" base64 encoded

Then use it in a Pod:

apiVersion: v1
kind: Pod
metadata:
  name: my-secret-pod
spec:
  containers:
  - name: my-container
    image: busybox
    command: ["/bin/sh", "-c", "echo 'DB Username:' $(cat /etc/secrets/username) && echo 'DB Password:' $(cat /etc/secrets/password) && sleep 3600"]
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
      readOnly: true # Best practice for secrets
  volumes:
  - name: secret-volume
    secret:
      secretName: my-db-secret

PVs, PVCs and StorageClasses

For data that has to outlive Pods, and even nodes, Kubernetes adds a layer of abstraction made of three objects: PersistentVolumes, PersistentVolumeClaims and StorageClasses. This is the standard way to run stateful applications in Kubernetes.

User/Developer Cluster Administrator Kubernetes Control Plane triggers dynamically provisions binds to mounts requests defines Pod PersistentVolumeClaim StorageClass Storage Provisioner PersistentVolume

PersistentVolume: the storage itself

A PV is a cluster-wide resource that represents a piece of real storage. An administrator provisions it, or it is provisioned dynamically. It hides the details of the storage behind it, such as a network file share or a cloud disk like AWS EBS or Azure Disk.

A PV has a capacity, which is its size. It has accessModes, which say how it can be mounted, such as ReadWriteOnce, ReadOnlyMany or ReadWriteMany. And it has a persistentVolumeReclaimPolicy, which decides what happens to the data when the PV is released.

Here is a static PV, for demonstration:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 1Gi # 1 Gigabyte of storage
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce # Can be mounted as read-write by a single Node
  persistentVolumeReclaimPolicy: Retain # Keep the data after PVC is deleted
  storageClassName: manual # A custom StorageClass name
  hostPath:
    path: "/mnt/data" # Path on the Node (for local testing only!)

Production clusters rarely back a PV with hostPath. They use storage from the cloud provider or a network storage system.

PersistentVolumeClaim: a request for storage

A PVC is a request for storage, made by a user or an application. It lets you use storage without knowing how it was provisioned. A PVC asks for a capacity and accessModes, and optionally a StorageClass.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce # Request ReadWriteOnce access
  resources:
    requests:
      storage: 500Mi # Request 500 Megabytes of storage
  storageClassName: manual # Request a PV from the 'manual' StorageClass

Kubernetes tries to bind this PVC to an available PV that meets its requirements: 500Mi, ReadWriteOnce and the manual StorageClass. The PV can be one created ahead of time or one provisioned dynamically.

StorageClass: storage on demand

A StorageClass defines how PersistentVolumes are provisioned dynamically. Administrators use them to describe classes of storage, such as fast SSD, slow HDD or network-attached storage. When a PVC asks for a class, Kubernetes creates the PV automatically, so users never have to create PVs themselves.

This one provisions AWS EBS volumes through a CSI driver:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-storage
provisioner: ebs.csi.aws.com # Example for AWS EBS using the modern CSI driver
parameters:
  type: gp3
reclaimPolicy: Delete # Delete the PV and underlying volume when PVC is deleted
volumeBindingMode: WaitForFirstConsumer

The provisioner and parameters depend on your cloud provider or on-premises storage. On AWS, ebs.csi.aws.com is the current standard, replacing the older in-tree kubernetes.io/aws-ebs provisioner.

Use the PVC in a Pod

Once the PVC is bound to a PV, a Pod can use it:

apiVersion: v1
kind: Pod
metadata:
  name: my-app-with-pvc
spec:
  containers:
  - name: my-container
    image: busybox:1.36
    command: ["/bin/sh", "-c", "echo 'Data from PVC' > /data/output.txt && sleep 3600"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data # Mount the PVC at /data inside the container
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: my-pvc # Reference the PVC by name

Data written to /data/output.txt now survives the Pod being deleted and recreated, as long as the PV and the PVC exist.

The next post turns to cluster administration, starting with installing a production-ready cluster: Building a cluster with kubeadm.