All writing
Updated 4 min readNaren

Hardening a cluster: CIS benchmarks and API access

Check a cluster against the CIS Kubernetes Benchmark with kube-bench, then control who can do what through the API server with authentication, RBAC and network policies.

KubernetesSecurity

The last post covered multi-container Pods and the sidecar, ambassador, adapter and init container patterns. With the applications in place, the next job is securing the cluster they run on.

This post covers two parts of cluster security. The first is hardening the cluster's configuration against the CIS Benchmarks for Kubernetes. The second is restricting access to the API with authentication, RBAC and network policies. Both protect the cluster from unauthorized access and attacks, and both are central to the CKS exam.

CIS Benchmarks for Kubernetes

The Center for Internet Security (CIS) Kubernetes Benchmark is a comprehensive set of security recommendations for hardening Kubernetes. It is a detailed checklist, written by security experts, for protecting your cluster against common vulnerabilities.

Following it fixes the common misconfigurations that attackers exploit, and it shrinks your attack surface by locking down access and features you don't need. Regulations such as PCI DSS and HIPAA often require it. And it is built by consensus, so it is a reasonable baseline of security practice.

The benchmark covers:

  • the control-plane components: the API server, etcd, the scheduler and the controller manager
  • the worker nodes, including the kubelet and other node configuration
  • policies: RBAC, Pod Security Standards, network policies and secrets management
  • the API server's settings, since it is the cluster's main management interface

Automating the checks with kube-bench

Checking hundreds of recommendations by hand is not practical. kube-bench, an open-source tool from Aqua Security, automates it: it checks whether your Kubernetes deployment meets the CIS Kubernetes Benchmark.

You can run it in several ways.

Inside the cluster, run it as a Kubernetes Job. kube-bench needs access to the host's PID namespace and to configuration directories on the host, and the project's job.yaml sets that up. A Job also suits repeated or scheduled scans:

kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml
kubectl get pods
kubectl logs job/kube-bench

The results are in the Pod's logs. To check a control-plane node, the Pod has to be scheduled on that node, which means adding a node selector and tolerations to the Job.

You can also run the kube-bench binary directly on your control-plane or worker nodes.

kube-bench detects the Kubernetes version you are running and picks the matching CIS Benchmark version. The --benchmark flag overrides that choice.

Reading the results

kube-bench reports PASS, FAIL, WARN or INFO for each test. For a FAIL, it usually adds a description and remediation advice that tells you how to fix the misconfiguration.

On managed Kubernetes services such as EKS, GKE and AKS, the cloud provider handles many of the control-plane recommendations. Use a benchmark made for that service where one exists, such as the CIS GKE Benchmark.

Restricting API access

The Kubernetes API server is the hub for every operation in the cluster, so restricting access to it matters more than anything else here. That takes several layers:

sends request 1. Authentication validates identity 2. Authorization checks permissions 3. Admission Control validates/mutates request stores in User/ServiceAccount API Server Authn Plugin Authz Plugin (RBAC) Admission Controller etcd

1. Authentication: who are you?

Before anyone or anything can use the API, Kubernetes needs to know who is asking. That is authentication, and Kubernetes supports several methods:

  • Client certificates (X.509), common for administrators, where a certificate signed by the cluster's certificate authority identifies the user.
  • Bearer tokens. Service account tokens are what applications in Pods use to talk to the API. OpenID Connect (OIDC) tokens connect Kubernetes to external identity providers.
  • An authenticating proxy, which handles authentication outside the cluster.

Kubernetes distinguishes normal users, who are managed outside the cluster, from service accounts, which Kubernetes manages for applications running inside it.

2. RBAC: what are you allowed to do?

Once a user or service account is authenticated, role-based access control (RBAC) decides what it may do. RBAC is the main authorization mechanism in Kubernetes.

Cluster-Wide Namespace-Specific is bound by is bound by grants permissions from grants permissions from User/ServiceAccount ClusterRole ClusterRoleBinding Role RoleBinding

RBAC uses four Kubernetes objects:

  • A Role defines a set of permissions within one namespace.
  • A ClusterRole defines permissions that are cluster-wide, for resources that don't belong to a namespace, or for namespaced resources across all namespaces.
  • A RoleBinding grants the permissions in a Role to a user or service account within one namespace.
  • A ClusterRoleBinding grants the permissions in a ClusterRole to a user or service account across the whole cluster.

Permissions only add up. RBAC has no deny rules.

Example: a service account that can only read Pods

This example creates a service account that can only list and get Pods, in one namespace.

Create the namespace. Save this as namespace.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: restricted-app
kubectl apply -f namespace.yaml

Create the service account, as serviceaccount.yaml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader-sa
  namespace: restricted-app
kubectl apply -f serviceaccount.yaml

Create a Role that grants get and list on Pods, as role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader-role
  namespace: restricted-app # Role is specific to this namespace
rules:
- apiGroups: [""] # Core API group
  resources: ["pods"]
  verbs: ["get", "list"]
kubectl apply -f role.yaml

Bind pod-reader-sa to pod-reader-role with a RoleBinding, as rolebinding.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-reader-binding
  namespace: restricted-app
subjects:
- kind: ServiceAccount
  name: pod-reader-sa
  namespace: restricted-app
roleRef:
  kind: Role
  name: pod-reader-role
  apiGroup: rbac.authorization.k8s.io
kubectl apply -f rolebinding.yaml

Any Pod that runs as pod-reader-sa in the restricted-app namespace can now only get and list Pods in that namespace.

3. Network policies: who can talk to whom?

RBAC controls who can use the Kubernetes API. Network policies control which Pods can talk to each other, and to endpoints outside the cluster. They act as a firewall for your Pods.

A common practice is a default deny policy, which blocks all traffic between Pods unless something explicitly allows it. Policies use selectors, such as podSelector, namespaceSelector and ipBlock, to define the sources and destinations of allowed traffic, down to specific ports and protocols.

The post on network policies covers them in detail.

4. Securing the API server

Always reach the API server over HTTPS, with TLS certificates. Disable anonymous access, so unauthenticated users can't make API requests.

Grant users and service accounts the least privilege they need, and avoid wildcards (*) in RBAC rules. Review your RBAC configuration and network policies regularly.

Where you can, don't expose the Kubernetes API server publicly at all.

The next post continues with security, at the level of the Pod: Pod Security Standards, AppArmor, seccomp and sandboxed runtimes. Pod security.