CKA Concepts: Understand, Apply, Verify
Build the knowledge to explain what Kubernetes is doing, make the requested change, and prove that it works. Read a domain here, use the matching section in Playbook, then attempt an exercise before opening its solution.
This guide follows the five published CKA domains. The weights below and Kubernetes v1.35 exam version were checked on 19 September 2026 against the Linux Foundation CKA overview ↗. Check that page again before your exam; versions change. In Kubernetes documentation, select the version matching your environment.
| Domain | Weight | What you should be able to do |
|---|---|---|
| Cluster Architecture, Installation & Configuration | 25% | Build, maintain, extend, and control access to a cluster. |
| Workloads & Scheduling | 15% | Run reliable applications and control their placement and scale. |
| Services & Networking | 20% | Connect workloads and diagnose traffic paths. |
| Storage | 10% | Provision, attach, and preserve application data. |
| Troubleshooting | 30% | Find the failing layer, repair it, and verify recovery. |
These are original practice materials based on public objectives. The goal is exam-relevant capability: completing a precise task under time pressure and checking its observable outcome.
Build the command-to-evidence habit
Coach Caz: Before you lift the keyboard, name the evidence you expect. Random commands are cardio.
Use each Practice loop below as a short rehearsal. Read its starting state, predict the result of the first command, then compare that prediction with the evidence. The outputs are illustrative: names, timestamps, addresses, and counts vary. Commands use the example's names; substitute the actual task's namespace, resource, host, and file names when transferring the pattern to a drill.
- Recognize: say which symptom puts you on this path.
- Inspect: run the smallest check that separates two plausible causes.
- Act: change the field or configuration supported by that evidence.
- Prove: repeat the failed operation and check a constraint that must still hold.
- Repeat: hide the commands, change one input, and rebuild the sequence from memory.
Use a disposable practice environment for changes. A case that assumes an installed controller, tool, or prepared resource says so; it does not install those prerequisites for you. Linux host commands belong inside a practice Linux VM or the specified lab node. These examples teach the investigation pattern; use the CLI's assigned task and grader for recorded reps.
1. Cluster Architecture, Installation & Configuration · 25%
Explore this section
Know which component owns the next stepAccess: identity, permission, scopeInstallation, lifecycle, and availabilityInstallation tools and extension pointsCRI, CNI, and CSI: choose the failing interfaceStatic Pods: the kubelet owns the lifecycleHelm: chart, values, release, revisionHelm: inspect, install, upgrade, recoverKustomize: base, overlay, rendered resourcesKustomize: render an overlay and verify the changeCRDs and operators: an API type needs a controllerPractice loop: a forbidden request is an access questionKnow which component owns the next step
A request reaches the API server, which validates it and stores cluster state in etcd. Controllers reconcile desired and observed state. The scheduler assigns unscheduled Pods to suitable nodes. Each node's kubelet asks its container runtime to run those Pods. Service traffic is implemented by kube-proxy or a replacement supplied by the networking implementation.
Remember: the scheduler chooses a node; the kubelet runs the Pod. A Pod with no node assignment and a Pod that crashes after assignment need different investigations. Kubernetes components ↗
Access: identity, permission, scope
A kubeconfig selects a cluster, a user identity, and optionally a namespace through a context. Authentication establishes who you are; RBAC decides which API actions that identity may perform.
A Role grants permissions within a namespace. A ClusterRole can describe namespaced or cluster-scoped permissions. A RoleBinding grants permissions in its own namespace, even when it refers to a ClusterRole. A ClusterRoleBinding grants the referenced ClusterRole across the cluster. ServiceAccounts are identities for workloads.
Remember: a role describes permissions; a binding gives them to a subject. Verify both an allowed action and an action that should remain denied. RBAC ↗ · Kubeconfig ↗
Installation, lifecycle, and availability
Before kubeadm init, prepare the host: supported OS and packages, unique node identity, network reachability and required ports, a compatible CRI runtime, matching cgroup configuration, and the documented swap configuration. Choose non-overlapping Pod and Service networks. Initialize the first control-plane node, configure administrative access, install a compatible Pod network, and join remaining nodes with the generated join instructions. Validate node readiness and cross-node connectivity. Install kubeadm ↗ · Create a cluster ↗
For upgrades, follow the target version's supported sequence and version-skew rules. Upgrade the first control-plane node, additional control-plane nodes, then workers. Drain where required, update kubelet and kubectl, restart kubelet, verify, and uncordon. cordon blocks new scheduling; drain also evicts eligible workloads. PodDisruptionBudgets can prevent a drain. Kubeadm upgrades ↗ · Drain a node ↗
High availability needs redundant control-plane components behind a stable API endpoint and a healthy etcd quorum. Know stacked etcd versus external etcd. Three healthy etcd members can tolerate one member failure; extra API servers do not compensate for lost etcd quorum. HA topologies ↗
Certificates have different owners. Inspect which certificate, identity, or trust chain is failing before renewing anything. Kubeadm certificate renewal is not a general fix for a kubelet authentication problem. Practise backup and recovery on a disposable cluster using its actual etcd topology and tool versions. Certificate management ↗ · Operating etcd ↗
Installation tools and extension points
| Tool or interface | Purpose | Useful distinction |
|---|---|---|
| Helm | Installs a chart as a versioned release using values. | Chart version and container image version are separate. |
| Kustomize | Builds manifests from a base plus overlays and patches. | Inspect rendered output before applying it. |
| CRI | Connects the kubelet to a container runtime. | Runtime failures can prevent containers from starting. |
| CNI | Provides Pod networking through a network implementation. | Installing Kubernetes alone does not install a complete Pod network. |
| CSI | Connects Kubernetes to storage drivers. | A StorageClass needs a working provisioner. |
| CRD and operator | Adds an API type; a controller reconciles instances of that type. | Creating a custom resource does not install its controller. |
References: Helm ↗ · Kustomize ↗ · CRI ↗ · Network plugins ↗ · CSI ↗ · CRDs ↗ · Operators ↗
Practise: grant a ServiceAccount read-only Pod access in one namespace; render a Kustomize overlay; install and roll back a Helm release. Prove it: check effective permissions, resulting resources, and release history. Then rehearse installation, upgrades, and HA on a dedicated kubeadm cluster.
CRI, CNI, and CSI: choose the failing interface
CRI runs containers; CNI connects Pods; CSI supplies storage. The API can accept a Pod before any of those dependencies succeeds. A sandbox creation error suggests runtime or networking setup; an attach/mount error suggests the storage path. Read the actual event before restarting unrelated components.
Use crictl info on the node with the runtime endpoint configured for that kubelet, the network agent on affected nodes, and the CSI controller/node components for a failing volume. A healthy API server does not prove any of these interfaces works. Prove it: launch a workload, reach it across nodes, and read/write its volume; those are three independent checks. CRI ↗ · Network plugins ↗ · CSI ↗
Static Pods: the kubelet owns the lifecycle
A static Pod comes from a node's configured staticPodPath. Its API mirror normally has the Pod's metadata.name plus the node suffix. The file can be called pod.yaml; the filename is not the Pod identity. The kubelet ignores dotfiles but reads other files without filtering by extension, so move backups outside the watched directory.
Deleting the mirror through the API does not remove the source manifest. To change or remove the workload, change that source on the correct node. When the API is down, inspect the kubelet journal and runtime with crictl; a missing mirror alone does not tell you whether a container is running. Prove it: correct node, expected image, running workload, and kubelet/file ownership evidence. Static Pods ↗
Helm: chart, values, release, revision
Helm packages related Kubernetes resources so you can install and maintain a component as one release. A networking controller, metrics provider, or application can all be delivered this way. Four terms answer different questions:
| Term | Question it answers | Example |
|---|---|---|
| Chart | Which package of templates and defaults? | team/agent |
| Values | Which settings should those templates use? | Replica count, image tag, Service type |
| Release | Which installed instance, in which namespace? | node-tools in operations |
| Revision | Which recorded change to that release? | Revision 3 after an upgrade |
A chart version selects the package; an image tag selects a container image. The chart's appVersion is descriptive metadata, not a substitute for checking the rendered image. A release revision is neither of those versions. Using Helm ↗
Values are inputs to templates, not arbitrary Kubernetes fields. A chart might expose replicaCount; another might use a different key. Inspect helm show values and the chart's documentation first. Chart defaults are overridden by your values files, then command-line overrides. With several -f files, later files win for overlapping keys. Values files ↗ · Upgrade options ↗
Form check: “Install chart version 2.4.0 with image 1.27” asks for two settings. --version 1.27 does not set the container image. Caz: “Read the plates before you load the bar.”
Helm: inspect, install, upgrade, recover
Replace the uppercase placeholders below with the task's repository, chart, chart version, release, and namespace. VALUES_FILE must be an existing file you create or the task supplies. These commands illustrate a workflow, not an installed sample chart.
helm repo add REPO REPOSITORY_URL
helm repo update
helm search repo REPO/CHART --versions
helm show values REPO/CHART --version CHART_VERSION
helm template RELEASE REPO/CHART --version CHART_VERSION \
-n NAMESPACE -f VALUES_FILE
Repository setup makes chart versions discoverable; it does not install a release. Rendering shows what the chosen values produce without installing it. Check names, namespace, images, RBAC, and any cluster-scoped resources before proceeding. A successful render alone does not prove the cluster can run the component. Using Helm ↗
helm upgrade --install RELEASE REPO/CHART --version CHART_VERSION \
-n NAMESPACE --create-namespace -f VALUES_FILE --wait --timeout 5m
helm status RELEASE -n NAMESPACE
helm get values RELEASE -n NAMESPACE
helm get manifest RELEASE -n NAMESPACE
helm history RELEASE -n NAMESPACE
For an upgrade, deliberately carry forward the required settings: provide the complete intended values file, or use --reuse-values when retaining the previous release values is appropriate. Do not assume a new one-key override preserves everything. --wait adds readiness checks; still verify the requested Service response or component behavior. Use documentation matching helm version in your environment. Upgrade behavior ↗
A rollback takes a revision from helm history:
helm rollback RELEASE REVISION -n NAMESPACE --wait --timeout 5m
helm history RELEASE -n NAMESPACE
Rollback creates another history entry restoring the chosen release configuration. It is not a database restore. Confirm the workload recovered; do not stop at the release status. Rollback ↗
Recognition cues: an absent release may be in another namespace (helm list -A); a correct Pod count can still hide incorrect stored values; editing a Deployment directly does not update the Helm values that should reproduce it.
Prove it: the intended release and namespace, chart version, stored settings, rendered objects, and healthy workload all agree. Practise repository install (kubefit drill cka-16-helm-repo-install) and upgrade/rollback (kubefit drill cka-15-helm-upgrade-rollback).
Kustomize: base, overlay, rendered resources
Kustomize transforms ordinary Kubernetes YAML without chart templates. A base holds reusable resources. An overlay refers to that base and describes an environment's differences. Its kustomization.yaml is input to the build, not an API object to create.
Use resources to include files or bases, images to change image references, replicas to change counts, and targeted patches for other fields. Generators can create ConfigMaps and Secrets; generated names normally include a content hash, with recognized references updated in the output. Read the rendered names rather than guessing them. Kustomize concepts and fields ↗
Remember: Helm tracks an installed release and its history; Kustomize renders a desired set of objects. Kustomize does not give you helm rollback. To undo an overlay change, restore its earlier configuration and apply the rendered result. A Deployment's rollout history only covers its own Pod template.
Form check: an overlay requesting three replicas should not require editing the shared base. That would silently change every environment using it.
Kustomize: render an overlay and verify the change
Here is a complete small example you can save in a new practice directory. Create base/ and overlays/practice/ beneath it. All filenames below belong to this example; no files from another repository are assumed.
Save as base/deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: practice-web
spec:
replicas: 1
selector:
matchLabels:
app: practice-web
template:
metadata:
labels:
app: practice-web
spec:
containers:
- name: web
image: nginx:1.27
Save as base/kustomization.yaml:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
Save as overlays/practice/kustomization.yaml:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
namespace: kustomize-practice
replicas:
- name: practice-web
count: 3
images:
- name: nginx
newTag: 1.27-alpine
patches:
- target:
kind: Deployment
name: practice-web
patch: |-
- op: add
path: /spec/template/metadata/labels/tier
value: frontend
From the practice directory, rendering should show namespace kustomize-practice, three replicas, image nginx:1.27-alpine, and Pod label tier: frontend. The base stays unchanged. A patch target must identify the intended object; changing a label must not break its selector. Render and customize resources ↗
kubectl kustomize overlays/practice
On a disposable practice cluster, create the namespace and apply the directory with -k. The namespace field sets resource namespaces; it does not create a Namespace object for you.
kubectl create namespace kustomize-practice
kubectl diff -k overlays/practice
kubectl apply -k overlays/practice
kubectl -n kustomize-practice rollout status deployment/practice-web --timeout=120s
kubectl -n kustomize-practice get deployment practice-web -o yaml
kubectl -n kustomize-practice get pods -L tier
kubectl diff exits 1 when it finds differences; that is expected before a change. apply -f reads resource files, while apply -k builds a kustomization. Applying the base would miss the overlay's changes. Apply reference ↗ · Diff reference ↗
Prove it: inspect both rendered output and live state: three ready replicas, the requested image and Pod label, correct namespace, unchanged base. The existing Kustomize drill is in CKAD (kubefit drill ckad-06-kustomize); it exercises the same tool and selects CKAD as your current training track. There is no dedicated CKA Kustomize drill yet.
CRDs and operators: an API type needs a controller
A CustomResourceDefinition registers a resource type. A custom resource is an instance of that type. An operator runs reconciliation logic for those instances. Installing a CRD makes an API available; it does not make the desired application appear by itself. CRDs ↗ · Operators ↗
Think of installation as dependencies: register the API type, ensure the controller and its permissions are ready, then create the requested custom resource. If the API reports an unknown kind, check the installed CRD and served API version. If the object exists but nothing happens, inspect controller readiness, logs, RBAC, and the custom resource's status.
Charts may carry CRDs in a crds/ directory. Helm installs those definitions before chart templates, but does not automatically upgrade or delete them through the normal CRD handling path. Follow the component's lifecycle instructions rather than assuming rollback or uninstall reverses a CRD change. Helm CRD lifecycle ↗
Prove it: the CRD is established, the controller is healthy, and the custom resource produces its expected state. Practise Helm and CRDs (kubefit drill cka-11-helm-crd).
Practice loop: a forbidden request is an access question
Starting state: in a disposable cluster, namespace concepts-cka contains ServiceAccount reader. Your administrator identity can impersonate it. The requirement is to read Pods in that namespace, without reading Secrets or deleting Pods.
Recognize → inspect: Forbidden means the API received the request but refused it. Test the actual identity and scope before changing a workload:
kubectl config current-context
kubectl -n concepts-cka get serviceaccount reader
kubectl auth can-i list pods -n concepts-cka --as=system:serviceaccount:concepts-cka:reader
kubectl -n concepts-cka get role,rolebinding -o yaml
Look for: no plus a missing binding, wrong subject namespace, or Role without list on core-group pods. Unauthorized instead points first to authentication; a timeout points first to connectivity. Creating another Role cannot fix either.
Act: for this case, create a namespaced read-only grant:
kubectl -n concepts-cka create role pod-observer --verb=get,list,watch --resource=pods
kubectl -n concepts-cka create rolebinding pod-observer --role=pod-observer --serviceaccount=concepts-cka:reader
kubectl auth can-i list pods -n concepts-cka --as=system:serviceaccount:concepts-cka:reader
kubectl auth can-i delete pods -n concepts-cka --as=system:serviceaccount:concepts-cka:reader
kubectl auth can-i get secrets -n concepts-cka --as=system:serviceaccount:concepts-cka:reader
Prove: expect yes, no, no. If a forbidden operation returns yes, inspect other RoleBindings and ClusterRoleBindings: grants add together. Do not assume the new narrow Role removed an older broad grant.
Repeat from memory: change the required resource to ConfigMaps. Name the API group, verbs, subject, and scope before typing. RBAC ↗
Coach Caz: Check the membership card before rebuilding the gym. Identity and permission come before workload repairs.
2. Workloads & Scheduling · 15%
Choose a controller and understand recovery
A Deployment manages interchangeable application replicas through ReplicaSets. A StatefulSet provides stable Pod identities and storage associations. A DaemonSet runs a Pod on each eligible node. A Job completes work; a CronJob schedules Jobs.
Controllers replace failed Pods to restore desired state. They cannot fix a bad image, missing configuration, or an application defect. Deployment template changes trigger a rollout; changing a Service does not. Rollback restores an earlier Pod template, not your database or every related resource. Deployments ↗ · Workload controllers ↗
A DaemonSet's DESIRED count is the number of nodes its Pod template is allowed on, not the number of nodes in the cluster. A control-plane node carries the node-role.kubernetes.io/control-plane:NoSchedule taint, so a DaemonSet without a matching toleration silently covers one node fewer than you expect. Recognition cue: DESIRED 2 on a three-node cluster. Proof of success: desired, current, and ready all equal the node count, kubectl get pods -o wide shows a Pod on the control plane, and the taint is still there — removing it is not a fix. The update strategy matters too: RollingUpdate replaces Pods after a template change, OnDelete waits for you to delete each one. DaemonSet on Every Node (kubefit drill cka-18-daemonset-node-coverage) practises exactly this: tolerate the taint, state RollingUpdate with maxUnavailable: 1, and prove coverage rather than assume it. DaemonSet ↗
Health, configuration, and scale
Readiness controls whether a Pod is a ready Service endpoint. Liveness can restart an unhealthy container. A startup probe allows initialization to finish before liveness and readiness probes begin. A running Pod can still be unready. Probes ↗
ConfigMaps hold ordinary configuration; Secrets hold sensitive values. Base64 encoding is not encryption. Both can supply environment variables or mounted files. Environment variables do not refresh in a running container when the source changes; projected files update eventually, except subPath mounts. The application must also reload changed files. ConfigMaps ↗ · Secrets ↗
The HPA changes replica count using observed metrics. CPU utilization targets depend on CPU requests, and resource metrics require a metrics provider such as Metrics Server. <unknown> metrics are a dependency to investigate, not evidence that scaling works. Horizontal Pod autoscaling ↗
Admission is different from scheduling
Admission may reject or modify a request before a Pod is stored. ResourceQuota, LimitRange, and Pod Security Admission can affect acceptance. A successfully created Pod may then remain Pending because no node satisfies its scheduling constraints. Admission controllers ↗ · ResourceQuota ↗ · LimitRange ↗
Requests reserve capacity for scheduling; limits constrain runtime use. CPU can be throttled; exceeding a memory limit can cause an OOM kill. nodeSelector and required node affinity restrict placement; preferred affinity expresses a preference. A toleration permits scheduling onto a matching tainted node but does not attract a Pod there or guarantee placement.
Remember: requests fit; affinity selects; tolerations permit. Read events before changing constraints. Resources ↗ · Node affinity ↗ · Taints and tolerations ↗
Practise: roll out an image, recover a failed rollout, repair a Pending Pod, cover every node with a DaemonSet, and configure an HPA. Prove it: inspect the Pod template, readiness, placement, replica count, and metrics—not just whether an apply command succeeded.
Practice loop: find why a Pod cannot be scheduled
Starting state: Deployment web in concepts-cka has a Pending Pod and a required nodeSelector of disk: fast. You are to repair an incorrect selector, not change node labels or resource requirements.
kubectl -n concepts-cka get pods -o wide
kubectl -n concepts-cka describe pod WEB_POD
kubectl -n concepts-cka get deployment web -o jsonpath='{.spec.template.spec.nodeSelector}{"\n"}'
kubectl get nodes -L disk
Replace WEB_POD with the Pending name from the first command. Look for: a FailedScheduling event such as didn't match Pod's node affinity/selector. If eligible nodes have disk=ssd and the requirement says SSD, the mismatch is in the controller template. Insufficient cpu would send you to requests and capacity instead; adding a toleration would not fix that.
Act → prove: after confirming the intended label value, update the template and inspect the replacement:
kubectl -n concepts-cka patch deployment web --type=merge -p '{"spec":{"template":{"spec":{"nodeSelector":{"disk":"ssd"}}}}}'
kubectl -n concepts-cka rollout status deployment/web --timeout=90s
kubectl -n concepts-cka get pods -o wide
A successful rollout and a Pod on an eligible node establish more than an accepted patch. If it remains Pending, read the new event; multiple constraints may fail in sequence.
Repeat: have a partner change either a selector or a CPU request. Identify which one from evidence before inspecting the answer. Scheduling constraints ↗
Coach Caz: A reserved rack is not an available rack. Read the scheduling reason before adding more weight.
3. Services & Networking · 20%
Follow the traffic path
Containers in a Pod share a network namespace and can communicate over localhost. Pod-to-Pod traffic depends on the cluster network. A Service supplies a stable destination for a changing set of backends, typically selected by labels and represented in EndpointSlices.
Remember: selector → ready endpoint → listening port. A Service's port is the client-facing port; targetPort directs traffic to the backend. A Pod's containerPort declaration does not make its application listen on that port. Services ↗ · EndpointSlices ↗ · Cluster networking ↗
| Service type | Use it for |
|---|---|
| ClusterIP | A stable address inside the cluster. |
| NodePort | A port exposed on nodes, subject to node networking and reachability. |
| LoadBalancer | An external load balancer supplied by a supported implementation. |
A pending external address can mean no load-balancer implementation is installed. Creating the resource alone does not provide one.
Control allowed connections
NetworkPolicies select Pods and allow particular ingress or egress traffic. Enforcement requires a supporting network implementation. Policies are additive: traffic allowed by any applicable policy is allowed. If both ends are isolated, source egress and destination ingress must both allow the connection.
Within one peer, namespaceSelector plus podSelector means both must match. Separate list entries mean either may match. An empty podSelector selects all Pods in that namespace. DNS may need an explicit egress allowance when egress is restricted. NetworkPolicy ↗
Route external traffic and resolve names
An Ingress describes HTTP(S) host/path routing and requires an Ingress controller. Gateway API separates infrastructure from routing: GatewayClass identifies an implementation, Gateway configures listeners, and HTTPRoute attaches routing rules to them. Check route acceptance, reference resolution, and Gateway readiness before testing traffic. Cross-namespace references may need permission through a ReferenceGrant. Ingress ↗ · Gateway API ↗ · HTTP routing guide ↗
For a plain Ingress the working parts are the class (ingressClassName must name the class the installed controller claims), the host, and one path rule per backend with an explicit pathType. Two Prefix rules such as /api and / do not conflict: the longest matching path wins, and matching is by path element, so /apix goes to /. The backend port is the Service's port, not the container's. Recognition cue: the object is accepted, yet every request returns the controller's 404 — usually a missing Host header in the test, a class no controller claims, or a Service in another namespace. Proof of success: a request from inside the cluster to the controller's Service with the right Host header returns each backend's own response. Ingress Path Routing (kubefit drill cka-21-ingress-path-routing) routes two paths on one host through an internal NGINX controller. Ingress controllers ↗
CoreDNS typically provides cluster DNS. Short Service names resolve relative to a Pod's namespace; service.namespace identifies a Service in another namespace. Diagnose DNS separately from application reachability: successful name resolution does not prove that a Service has healthy backends. DNS for Services and Pods ↗ · Debug DNS ↗
Practise: repair a Service, allow one client through policy, configure an HTTPRoute, route two paths through one Ingress, and trace a DNS failure. Prove it: test from the intended client and also test a client that should remain blocked.
Practice loop: repair a Service without touching healthy Pods
Starting state: this self-contained example needs a disposable cluster that can pull nginx. Create a working Deployment and deliberately mismatched Service:
kubectl create namespace concepts-service
kubectl -n concepts-service create deployment web --image=nginx:1.27
kubectl -n concepts-service rollout status deployment/web --timeout=90s
kubectl -n concepts-service create service clusterip web --tcp=80:80
kubectl -n concepts-service patch service web --type=merge -p '{"spec":{"selector":{"app":"wrong"}}}'
Recognize → inspect: the application Pods are ready, but the Service has no usable destination. Compare selectors to labels, then inspect EndpointSlices:
kubectl -n concepts-service get pods --show-labels
kubectl -n concepts-service get service web -o yaml
kubectl -n concepts-service get endpointslices -l kubernetes.io/service-name=web -o yaml
Look for: Pods labeled app=web, Service selecting app=wrong, and no ready endpoint addresses. A ClusterIP merely shows that an address was allocated. If ready endpoints already exist, compare port, targetPort, and the actual listener instead of changing the selector.
Act → prove: repair only the mismatched selector. Test through Service DNS from another Pod; a port-forward would bypass part of the path being tested.
kubectl -n concepts-service patch service web --type=merge -p '{"spec":{"selector":{"app":"web"}}}'
kubectl -n concepts-service get endpointslices -l kubernetes.io/service-name=web
kubectl -n concepts-service run client --image=busybox:1.36 --restart=Never --command -- wget -qO- -T 5 http://web
kubectl -n concepts-service logs client
Wait for client to run if logs initially say it is still starting. Expected evidence: an endpoint address appears and the client prints nginx HTML. Could not resolve calls for DNS checks; refusal or timeout after resolution calls for port/path checks.
Repeat: remove the client Pod, change the Service target port to 8080, and diagnose the new failure without changing healthy Pod labels. Restore port 80, retest, then clean up this example with kubectl delete namespace concepts-service. Service debugging ↗
Coach Caz: The class is ready. The sign points to the broom cupboard. Fix the selector, not the class.
4. Storage · 10%
Separate the request from the storage
A PersistentVolumeClaim (PVC) requests storage in a namespace. A PersistentVolume (PV) represents storage available to the cluster. A StorageClass describes provisioning behavior. Dynamic provisioning creates a volume through a driver; static provisioning starts with an existing PV. Binding must satisfy capacity, class, access mode, and other selection constraints. Persistent volumes ↗ · Dynamic provisioning ↗
ReadWriteOnce means read-write mounting from one node, potentially by multiple Pods on that node. ReadWriteOncePod restricts access to one Pod when supported by the CSI driver. ReadWriteMany supports multiple nodes. Choose a mode the storage backend supports.
A reclaim policy applies after a claim is released: Retain keeps the volume for manual recovery; Delete removes the volume and, for supporting drivers, its backing storage. Do not clear a retained PV's claim reference unless a deliberate data-recovery/reuse procedure calls for it.
Binding and mounting are separate steps
WaitForFirstConsumer delays binding/provisioning until scheduling a consuming Pod can account for storage topology. Pending can be expected before a consumer exists; persistent Pending still needs event inspection. A bound claim can fail later during attach or mount. StorageClasses ↗
emptyDir lives with a Pod, surviving container restarts but not Pod removal. A hostPath exposes a path on one node and is not portable shared storage. Local PVs require node affinity. For persistent application data, verify the actual backing storage and driver. Volumes ↗ · Local volumes ↗
A claim naming a nonexistent StorageClass cannot dynamically provision a volume. That may be a typo to repair, or a deliberate static-binding task: read the requirement first. For static binding, the PV and PVC need matching class strings, compatible access and volume modes, sufficient capacity, and matching selection constraints. A matching StorageClass object is not required for that static match. A node-local volume also needs appropriate node affinity. Prove it: the existing claim binds to the intended volume, retains its identity, and a Pod on the correct node reads and writes data. Static PV for a Pending Claim (kubefit drill cka-19-static-pv-binding).
Remember: class provisions; claim binds; Pod mounts. Practise: mount a claim into a Deployment; supply a static PV for a claim no class will serve. Prove it: write a marker, replace the managed Pod, and read the marker again from the same claim.
Practice loop: a Pending claim is not yet a mount failure
Starting state: PVC data and Pod writer exist in concepts-cka. The Pod mounts that claim. The task requires the cluster's provided storage class; do not invent a hostPath volume to silence an event.
kubectl -n concepts-cka get pvc data
kubectl -n concepts-cka describe pvc data
kubectl get storageclass
kubectl -n concepts-cka describe pod writer
Read the evidence in order:
| Observation | Meaning | Next check |
|---|---|---|
storageclass ... not found | Requested provisioner configuration is absent | Compare the claim's exact class with the task and available classes |
waiting for first consumer | Delayed binding may be intentional | Inspect whether the consumer can be scheduled |
Claim Bound, Pod FailedMount | Allocation succeeded; attachment/mount did not | Read the Pod event and inspect the CSI node/controller path |
Act: follow the branch you actually observed. A wrong unbound claim may require replacement because claim fields are restricted; first establish whether it is bound or contains data and follow the task's recovery constraints. Do not delete a data-bearing claim as a generic repair.
Prove: in this example the container has a shell and mounts the volume at /data:
kubectl -n concepts-cka wait --for=condition=Ready pod/writer --timeout=90s
kubectl -n concepts-cka exec writer -- sh -c 'printf "volume-check\n" > /data/check; cat /data/check'
kubectl -n concepts-cka get pvc data -o wide
Expect volume-check and the intended Bound claim. To prove persistence, use a controller-managed consumer, replace its Pod, and read the same marker through the same claim; never delete the claim for that test.
Repeat: explain why Pending, Bound, and “read/write succeeded” answer three different questions. Persistent volumes ↗
Coach Caz: Booking storage, attaching storage, and using storage are three reps. Count all three.
5. Troubleshooting · 30%
Investigate the first failing layer
Use a repeatable loop: observe → narrow → repair → verify. Start with the requested outcome and compare it with current state. Change the smallest thing that explains the evidence.
| Symptom | First useful evidence | Common next layer |
|---|---|---|
| API request fails | Context, exact error, API endpoint reachability | Credentials, trust, RBAC, or API server |
| Node NotReady | Node conditions and events | Kubelet, runtime, disk, or network plugin |
| Pod Pending | Node assignment and scheduling events | Requests, affinity, taints, or storage |
| Container waiting or restarting | Describe output, current and previous logs | Image, command, configuration, probes, or memory |
| Service unreachable | Selector, EndpointSlices, client test | Readiness, ports, policy, DNS, or the Service dataplane (kube-proxy) |
| Metrics missing | Metrics API and provider health | Collection pipeline and resource requests |
Pod events explain orchestration failures; container logs explain application behavior. kubectl logs --previous retrieves the previous container instance when available. kubectl top shows recent resource metrics, not historical monitoring. Debug applications ↗ · Resource monitoring ↗
When the selector, endpoints, ports, and DNS are all correct and a Pod IP answers but the ClusterIP does not, the failing layer is the Service dataplane: kube-proxy (or the network implementation replacing it) programs each ClusterIP into every node's packet rules. Recognition cues: kube-proxy Pods in kube-system crash-looping with the same log line on every node, and — the subtle one — older Services still working, because the rules kube-proxy wrote earlier outlive its process; only Services created or changed after the failure are missing. In this exercise kube-proxy runs as a DaemonSet; inspect its template, configuration, logs, and node networking before choosing a repair. Restore the faulty setting and retest the Service. Proof of success: ds/kube-proxy reports ready on every node and a fresh Pod gets HTTP 200 through the ClusterIP. Repair a Broken kube-proxy (kubefit drill cka-20-kube-proxy-broken). Virtual IPs and Service proxies ↗
If the API is unavailable, inspect the host directly. Kubeadm control-plane components commonly run as static Pods; kubelet watches their manifest directory. Use runtime tools such as crictl and host logs to inspect components without relying on a healthy API. Keep manifest backups outside the watched directory. Debug clusters ↗ · Static Pods ↗ · crictl ↗
Practise: diagnose before opening a solution, write down your evidence, and repair only the identified cause. Prove it: repeat the original failed operation, confirm health remains stable, and check that you did not remove a required constraint.
Separate control-plane health from node health
An unavailable API calls for host-level evidence: API endpoint reachability, serving certificates, kubelet/runtime health, static-pod logs, and etcd availability. A reachable API with a NotReady worker calls for that node's conditions, pressure signals, journal, and runtime. Do not reboot every node to investigate one failure.
kubectl get --raw '/readyz?verbose' reports API readiness when access works; it does not replace checking etcd quorum or application availability. An etcd snapshot is database state, not application volume data. Rehearse restore with the installed etcd tools on a disposable recovery target, then verify API reads/writes and controller convergence. Prove it: the originally failing operation works and unrelated workloads retain their state. Cluster debugging ↗ · etcd recovery ↗
Practice loop: choose API evidence or node evidence
Starting state: a practice worker is NotReady. The API still responds. Start there rather than restarting the whole cluster:
kubectl get nodes
kubectl describe node WORKER
kubectl get --raw '/readyz?verbose'
Replace WORKER with the affected node. Look for: the Ready condition's reason/message, pressure conditions, and heartbeat timing. A healthy API readiness response does not certify that worker's runtime.
Move to the affected Linux node through the lab's documented access method. On a systemd/kubeadm node:
sudo systemctl status kubelet --no-pager
sudo journalctl -u kubelet -b -n 60 --no-pager
sudo crictl info
crictl must use that node's configured runtime endpoint. Example clue: a kubelet log reports it cannot decode its configuration file. Inspect the referenced file and recent change; repair the invalid field, then restart kubelet. If the log instead reports a runtime socket failure, inspect the runtime service and endpoint first. Repeated restarts cannot correct either file.
Prove: return to the API, wait for the node's Ready condition, then run or observe a workload on it. If the API itself stops answering, use node-side runtime logs for control-plane static Pods; kubectl describe cannot diagnose through an unavailable API.
Repeat: state your first three checks for “one worker NotReady” and for “API connection refused.” The two lists should not be identical. Cluster debugging ↗
Coach Caz: One treadmill stopped. That is not your cue to reboot the building.
Turn understanding into exam practice
Use Playbook for worked tasks and verification commands, then start an independent drill in the CLI. After each attempt, answer: What failed? Which evidence identified it? Why did the change work? What proves the task is complete?
Practise looking up one exact field or procedure in the official docs instead of copying an entire unrelated example. Review the exam's allowed resources ↗ before your sitting; a useful study link is not automatically an allowed exam resource.
All 25 CKA drills run in your terminal through the kubefit CLI. Start with A healthy app. A broken Service. (kubefit drill cka-03-broken-service), or let kubefit next pick. Each drill seeds its task files into a local kind cluster; use the stated node-access commands for host-level work. The dedicated upgrade drill covers one supplied workflow. Full cluster installation, multi-control-plane upgrades, and HA failure recovery still need a suitable kubeadm topology beyond these exercises.
Further applied practice
Protect a workload during maintenance
A PodDisruptionBudget limits voluntary evictions through the Eviction API. It cannot prevent a node failure, and it does not control a Deployment's rolling-update strategy. Choose either minAvailable or maxUnavailable, make the selector match the intended Pods, and inspect currentHealthy, desiredHealthy, and disruptionsAllowed before a drain.
With three healthy replicas and minAvailable: 2, one eviction can proceed; another may wait for replacement capacity. Repair unhealthy replicas or insufficient capacity rather than deleting the budget. Prove it: the node drains while the required healthy replicas remain available, then the node is safely returned to service.
Set resource defaults and namespace limits
A LimitRange supplies defaults and constrains individual objects. A ResourceQuota caps namespace totals or object counts. A Deployment may be accepted while its ReplicaSet cannot create a Pod because admission rejects the missing request or exceeded quota. In that case there is no Pending Pod for the scheduler to fix.
Read kubectl describe resourcequota, kubectl describe limitrange, and ReplicaSet events in the task's namespace. Compare requested resources with remaining quota. Prove it: a compliant Pod is admitted with the intended defaults, an oversized request is rejected, and unrelated namespaces are unaffected.
Perform a kubeadm minor-version upgrade
A kubeadm upgrade updates control-plane components and configuration; the kubelet binary still needs its own upgrade. Verify versions, readiness and workload identity after maintenance.
Restore an etcd snapshot into a working member
Snapshot validation proves that a backup can be read. Recovery additionally requires restoring member data and starting a healthy etcd process from it.
Expand a CSI claim without losing data
Expansion needs permission from the StorageClass and implementation by the CSI driver. Follow the request through PVC status and preserve data and volume identity.