Kubernetes security

Kubernetes gives you a powerful API for running containers, and by default it is generous: pods can talk to every other pod, every pod gets an API token, and a workload can request root, host networking or privileged mode unless something says no. Securing a cluster means saying no in the right places. This lesson focuses on what you, as the person deploying workloads, control: who can do what through the API (RBAC and ServiceAccounts), what pods may request (Pod Security Standards and securityContext), who pods can talk to (NetworkPolicy), and what Kubernetes Secrets do and don't protect.

#The attack surface

An attacker in a cluster usually arrives through a workload: a vulnerable app in a pod. From there, what they can reach depends on three things you configure:

• The API server. If the pod's ServiceAccount token has broad permissions, the attacker can read secrets, create pods or modify deployments through the Kubernetes API.
• The node. If the pod runs privileged, as root, or mounts host paths, escaping to the node is often trivial, and the node's kubelet credentials open further doors.
• The network. With no network policies, a compromised pod can connect to every database, internal admin service and metadata endpoint in the cluster.

Each section below closes one of these.

#RBAC: who can do what

Every API request is authenticated (who are you?) and then authorised. Role-based access control (RBAC) is the standard authoriser. It has four object types:

RBAC is additive only: there are no deny rules. Anything not granted is forbidden, so least privilege means granting narrowly.

Here, the orders team can inspect workloads and read logs in their namespace, but not read secrets or exec into pods:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: orders-developer
  namespace: orders
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log", "services", "configmaps", "events"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments", "replicasets"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: orders-developers
  namespace: orders
subjects:
  - kind: Group
    name: orders-devs            # group name from your identity provider
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: orders-developer
  apiGroup: rbac.authorization.k8s.io

Bind to groups from your identity provider (via OIDC), not to individual users, so joiners and leavers are handled in one place.

Check what a subject can actually do with kubectl auth can-i, impersonating it:

kubectl auth can-i list pods -n orders --as=jane --as-group=orders-devs
kubectl auth can-i get secrets -n orders --as=jane --as-group=orders-devs
kubectl auth can-i create pods --subresource=exec -n orders --as=jane --as-group=orders-devs

yes
no
no

Some permissions are far more powerful than they look. Grant them only deliberately:

• get/list on secrets: list returns every secret's contents, not just names.
• create on pods (or deployments, jobs): a new pod can mount any secret and run as any ServiceAccount in the namespace, inheriting all their permissions.
• create on pods/exec: a shell in someone else's running container.
• escalate, bind, impersonate: ways to grant yourself more than you have.
• Wildcards ("*"): grant every current and future resource or verb.

#ServiceAccounts: identities for workloads

Pods authenticate to the API as a ServiceAccount. Every namespace has one named default, and unless told otherwise every pod uses it and gets its token mounted at /var/run/secrets/kubernetes.io/serviceaccount/token. Most applications never call the Kubernetes API, so that token is purely a gift to an attacker.

Give each workload its own ServiceAccount, grant it only what it needs (often nothing), and don't mount the token unless the app uses it:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: orders-api
  namespace: orders
automountServiceAccountToken: false

Modern clusters issue bound, projected tokens: short-lived (about an hour, rotated automatically by the kubelet), tied to the specific pod, and invalid once the pod is deleted. Avoid creating legacy long-lived token Secrets for ServiceAccounts.

For CI deploying to the cluster, create a dedicated ServiceAccount with a Role limited to the namespace and resources it deploys (for example patch on deployments), and prefer short-lived credentials from your cloud's identity integration over a static kubeconfig stored as a CI secret. The same goes for pods that call cloud APIs: workload identity (EKS Pod Identity, GKE Workload Identity, Azure Workload Identity) maps a ServiceAccount to a cloud role, so no cloud keys are stored in the cluster.

#Pod Security Standards

The Pod Security Standards define three profiles for what a pod may request:

The built-in Pod Security Admission controller enforces them per namespace using labels. Each label sets a mode: enforce rejects violating pods, warn returns a warning to the user, audit records the violation in the audit log.

Before enforcing, check what would break. A server-side dry run reports every existing pod that violates the profile:

kubectl label --dry-run=server --overwrite ns orders \
  pod-security.kubernetes.io/enforce=restricted

Warning: existing pods in namespace "orders" violate the new PodSecurity enforce level "restricted:latest"
Warning: orders-api-7d9c6b8f4-x2kqp (and 2 other pods): allowPrivilegeEscalation != false, unprivileged capabilities, runAsNonRoot != true, seccompProfile
namespace/orders labeled (server dry run)

Then declare the labels in the namespace manifest, so they're version-controlled with everything else:

apiVersion: v1
kind: Namespace
metadata:
  name: orders
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/audit: restricted

From now on, a non-compliant pod is rejected with a precise explanation:

Error from server (Forbidden): error when creating "debug.yaml": pods "debug" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "debug" must set securityContext.allowPrivilegeEscalation=false), unprivileged capabilities (container "debug" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "debug" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "debug" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Pod Security Admission checks pods, but the error surfaces differently for controllers: a non-compliant Deployment is accepted, and its ReplicaSet then fails to create pods. Check kubectl get events -n orders or the ReplicaSet's status when a rollout stalls. For rules beyond the three profiles (allowed registries, required labels, signed images), add a policy engine such as Kyverno or OPA Gatekeeper, or the built-in ValidatingAdmissionPolicy.

#securityContext: a restricted-compliant workload

The securityContext fields are how a pod meets the restricted profile, and they map one to one onto the runtime hardening from the container security lesson:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
  namespace: orders
spec:
  replicas: 3
  selector:
    matchLabels: { app: orders-api }
  template:
    metadata:
      labels: { app: orders-api }
    spec:
      serviceAccountName: orders-api
      automountServiceAccountToken: false
      securityContext:                   # pod level
        runAsNonRoot: true
        runAsUser: 65532
        runAsGroup: 65532
        fsGroup: 65532
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: api
          image: ghcr.io/acme/orders-api@sha256:4c1e...
          ports:
            - containerPort: 8080
          securityContext:               # container level
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          resources:
            requests: { cpu: 100m, memory: 256Mi }
            limits:   { memory: 512Mi }
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir:
            medium: Memory
            sizeLimit: 64Mi

readOnlyRootFilesystem isn't required by the restricted profile, but add it anyway, with an emptyDir for any path the app genuinely writes. The image is referenced by digest, so what runs is exactly what was scanned and signed.

#NetworkPolicy: default deny

Without policies, the cluster network is flat. A NetworkPolicy selects pods and lists the traffic allowed to (ingress) or from (egress) them. Once any policy selects a pod for a direction, everything not explicitly allowed in that direction is dropped.

Start with a namespace-wide default deny, then allow specific flows:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: orders
spec:
  podSelector: {}          # every pod in the namespace
  policyTypes: ["Ingress", "Egress"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: orders
spec:
  podSelector: {}
  policyTypes: ["Egress"]
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - { protocol: UDP, port: 53 }
        - { protocol: TCP, port: 53 }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: orders-api
  namespace: orders
spec:
  podSelector:
    matchLabels: { app: orders-api }
  policyTypes: ["Ingress", "Egress"]
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx
      ports:
        - { protocol: TCP, port: 8080 }
  egress:
    - to:
        - podSelector:
            matchLabels: { app: orders-db }
      ports:
        - { protocol: TCP, port: 5432 }

The simulator applies these policies to a small cluster: the ingress controller, orders-api, its database, cluster DNS, and an analytics namespace with one compromised pod. It adds one policy the YAML above leaves out, orders-db, because default-deny-all isolates the database for ingress too. Step through the presets, then pick any cell to see both halves of the decision: egress from the source and ingress to the destination.

Watch the YAML structure in the DNS rule: namespaceSelector and podSelector in the same list item mean "pods labelled kube-dns in kube-system". Written as two separate items (each starting with -), they would mean "anything in kube-system or any kube-dns-labelled pod in orders" (a podSelector on its own only looks in the policy's namespace), which is much wider. Preset 5 shows the same mistake letting an attacker reach the database. Default deny for egress also blocks pods from reaching the cloud metadata endpoint (169.254.169.254), a classic route to stealing node credentials.

#Secrets: what they do and don't protect

A Kubernetes Secret is a ConfigMap with better manners, not a vault. Its values are base64-encoded, not encrypted:

kubectl get secret orders-db -n orders -o jsonpath='{.data.password}' | base64 -d

s3cr3t-Passw0rd

The caveats to design around:

• Storage. Secrets are stored in etcd. Unless the cluster configures encryption at rest (an EncryptionConfiguration, ideally backed by a cloud KMS), anyone with etcd access or an etcd backup reads them in plain text. Managed clusters usually offer KMS encryption as a setting; turn it on.
• Access is broader than RBAC on Secrets suggests. Anyone who can create a pod in a namespace can mount any Secret in it. Treat "can deploy to namespace X" as "can read every secret in namespace X."
• Exposure in the pod. Environment variables leak into crash dumps, debug endpoints and child processes; mounted files are slightly safer and update when the Secret changes.
• Git. Never commit Secret manifests. Use the External Secrets Operator or the Secrets Store CSI driver to sync values from your secret manager at runtime, or Sealed Secrets/SOPS if secrets must live in Git encrypted. The "Secrets in Kubernetes" section of the secrets management lesson compares the three, with the workload identity setup that lets them reach the manager without static cloud keys.

#Practice

You're handed a namespace payments running one Deployment, payments-api (port 8080, listens only for traffic from the ingress-nginx namespace, talks to payments-db on 5432). Lock it down:

1. Enforce the restricted Pod Security Standard on the namespace.
2. Give payments-api its own ServiceAccount with no API token.
3. Create a Role and RoleBinding so the CI ServiceAccount deployer can update the Deployment's image but cannot read secrets or exec into pods, and prove it with kubectl auth can-i.
4. Apply default-deny network policies plus the minimal allows.

apiVersion: v1
kind: Namespace
metadata:
  name: payments
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted
---
apiVersion: v1
kind: ServiceAccount
metadata: { name: payments-api, namespace: payments }
automountServiceAccountToken: false
---
apiVersion: v1
kind: ServiceAccount
metadata: { name: deployer, namespace: payments }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: { name: deployer, namespace: payments }
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    resourceNames: ["payments-api"]
    verbs: ["get", "list", "watch", "patch"]
  - apiGroups: ["apps"]
    resources: ["replicasets"]
    verbs: ["get", "list", "watch"]   # for kubectl rollout status
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: { name: deployer, namespace: payments }
subjects:
  - kind: ServiceAccount
    name: deployer
    namespace: payments
roleRef:
  kind: Role
  name: deployer
  apiGroup: rbac.authorization.k8s.io

SA=system:serviceaccount:payments:deployer
kubectl auth can-i patch deployment/payments-api -n payments --as=$SA   # yes
kubectl auth can-i patch deployment/other-api -n payments --as=$SA     # no
kubectl auth can-i get secrets -n payments --as=$SA                    # no
kubectl auth can-i create pods --subresource=exec -n payments --as=$SA # no

#Key takeaways

• RBAC is additive: grant narrow verbs on named resources to groups and ServiceAccounts, and treat secrets access, pod creation, exec and wildcards as high-privilege.
• Give every workload its own ServiceAccount and disable token automounting unless the app calls the API.
• Enforce the restricted Pod Security Standard with namespace labels, after a server-side dry run shows what would break.
• Set runAsNonRoot, drop ALL capabilities, disable privilege escalation, use seccomp RuntimeDefault and a read-only root filesystem.
• Start every namespace with default-deny NetworkPolicies for ingress and egress, then allow DNS and the specific flows you need, and verify your CNI enforces them.
• Secrets are base64, not encryption: enable KMS-backed encryption at rest, and remember that deploy rights imply secret access.