Kubernetes, simulated

The diagram. An architecture diagram drawn from the simulator’s state. At the top, the control plane: the controllers of kube-controller-manager (a green dot when desired equals actual, amber while reconciling; the box lights up when it acts), then etcd, kube-apiserver and kube-scheduler. On the left, kubectl (you), the client sending traffic, the ingress controller and the Services with their endpoints (green = ready). On the right, metrics-server, CoreDNS and the image registry. Below, a card per worker node: its kubelet (with the age of its last Lease heartbeat), containerd and kube-proxy (with its rule count), then its Pods. Each Pod shows its kubectl status, ready containers and restarts, a CPU and a memory bar per container (the tick is the request, the bar’s end the limit; red when throttled or near the limit) and its probe lights (S, L, R). The card’s footer compares requests with allocatable and shows actual use. Everything that talks to the API server does so along the bus under the control plane. Messages: requests to the API server (blue), its answers, etcd writes (violet), watch events (orange), kubelet and runtime calls and image pulls (amber), probes, client requests (green), metrics (pink), DNS, Lease heartbeats (grey) and errors (red). Tap any box to inspect it and act on it — describe, logs, delete, fail a probe, crash, leak memory, cordon, drain, power off.

Counters. Pods ready, API requests, the etcd revision, watch events, Pods scheduled and unschedulable, container restarts, OOM kills, evictions and the share of client requests that succeeded in the last 10 s.

Desired vs actual. Every Deployment, ReplicaSet, StatefulSet, DaemonSet, Job, HPA and PodDisruptionBudget with what its spec asks for and what its status reports.

Scheduler. For each scheduling decision: which filter plugin rejected which node and why, and each feasible node’s score per plugin, weighted, with the winner highlighted.

HPA. Each calculation: utilization, target, the ratio, ceil(current × utilization ÷ target), and what the tolerance, the rate limits and the stabilisation window made of it.

Terminal. A pretend kubectl against the simulated API server (type help): get (pods, deploy, rs, svc, nodes, events, endpoints, endpointslices, hpa, sts, pvc, pv, ds, jobs, cj, pdb, ing, all; -o wide|yaml|json|name, -l), describe, apply -f a built-in manifest or your own YAML (-f -), create deployment, run, expose, scale, autoscale, set image, rollout status|history|undo|restart, delete, cordon, uncordon, drain, taint, top, logs (with --previous) and nslookup. Output is formatted the way kubectl formats it. Writes print when the API server answers, one command at a time. Below it, chaos buttons: kill, restore or add a node; fail a Pod’s liveness or readiness, crash it, fill its memory; spike, calm or stop the load.

Events (kubectl get events) and What happened, a log of every decision the simulator made.

Settings: the number of worker nodes and the random seed (Pod name suffixes, scheduler ties, load balancing), and the list of timers with their Kubernetes defaults.

The API server and etcd. Every change is a request: authentication and admission (defaults such as the rolling-update limits, and the DefaultTolerationSeconds tolerations), validation, a write to etcd with a new revision, the answer, and watch events to the components that watch that kind. Cluster architecture

Controllers compare desired with actual state and write the difference (controllers). The Deployment controller hashes the Pod template into a ReplicaSet per version and runs the rolling update exactly as its rolloutRolling does: maxSurge rounds up, maxUnavailable rounds down (if both are 0, maxUnavailable becomes 1), the new ReplicaSet is scaled up to at most replicas
maxSurge, unhealthy old Pods are removed first, then old ones while at least replicas − maxUnavailable are available; progressDeadlineSeconds (600) sets ProgressDeadlineExceeded; rollout undo copies the previous revision’s template back and that ReplicaSet gets the next revision number. The ReplicaSet controller creates or deletes the difference, ranks Pods for deletion as Kubernetes does, and uses expectations so it never acts twice on the same change. Also: EndpointSlice, node lifecycle and taint-based eviction, HPA, Job (completions, parallelism, backoffLimit, back-off 10 s doubling to 6 min), CronJob (cron schedule, history limits), StatefulSet (OrderedReady, stable names, a PersistentVolumeClaim per replica, reverse-order scale-down, claims retained), DaemonSet (one Pod per eligible node, pinned by node affinity, with the DaemonSet tolerations), disruption (PodDisruptionBudget status) and the garbage collector.

The scheduler runs the default filters — NodeUnschedulable, NodeName, TaintToleration, NodeAffinity (and nodeSelector), NodeResourcesFit on requests against allocatable, and required inter-pod anti-affinity — and the default score plugins with their weights: NodeResourcesFit (LeastAllocated) 1, BalancedAllocation 1, ImageLocality 1, NodeAffinity 2, PodTopologySpread 2 (the system default spreading) and TaintToleration 3, with ties broken at random. An unschedulable Pod gets the scheduler’s own 0/3 nodes are available: … message and is retried on cluster events or after 5 minutes (kube-scheduler, scheduling plugins).

Kubelets create a Pod sandbox (its IP), pull images (cached per node; IfNotPresent unless the tag is latest; ErrImagePull then ImagePullBackOff), start containers, run startup, liveness and readiness probes with their thresholds, restart containers by restartPolicy with the back-off of 10 s doubling to 300 s (reset after 10 minutes running), enforce limits — CPU is throttled, memory over the limit is OOM-killed (exit 137) — and evict Pods when memory.available drops below 100Mi, BestEffort and over-request Pods first (Pod lifecycle, node-pressure eviction). QoS classes are computed as Kubernetes does.

Nodes renew a Lease every 10 s. After 50 s of silence (--node-monitor-grace-period) a node is Ready=Unknown, tainted node.kubernetes.io/unreachable, its Pods are marked not Ready, and after their tolerationSeconds (300) they are deleted and replaced (taint-based evictions).

Services: a ClusterIP (or headless) Service, EndpointSlices that follow readiness, kube-proxy on every node rewriting its rules from them and spreading requests at random over ready endpoints, NodePorts, an Ingress, and CoreDNS names, including pod.service names for a headless Service.

The HPA reads CPU from metrics-server (scraped every 15 s; a Pod needs two samples) every 15 s and computes desired = ceil(current × utilization ÷ target) as replica_calculator.go does — skipped within the 0.1 tolerance, unready Pods and Pods without metrics set aside — then applies the default behaviour: scale up at most +100% or +4 Pods per 15 s, scale down to the highest recommendation of the last 300 s (algorithm).

Drain cordons the node, skips DaemonSet Pods with --ignore-daemonsets, and evicts the rest through the Eviction API, which refuses (429) while a PodDisruptionBudget allows no disruption; kubectl retries every 5 s. Budgets as percentages round up (disruptions).

One namespace (default), no RBAC, no admission webhooks, no resource quotas or LimitRanges, no priority or preemption, no init or sidecar containers, no volumes beyond the claims a StatefulSet makes (bound at once by a standard StorageClass).

The API server and etcd are single and never fail; there is no leader election, no resourceVersion conflicts and no watch bookmarks. Status updates are drawn small.

Container images are a fixed catalogue (nginx, httpd, redis, postgres, busybox, perl, fluent-bit and a few registry.example.com apps that crash, start slowly, leak memory or burn CPU); others do not exist and fail to pull. An app’s CPU use is a simple function of requests per second.

The back-off follows the documented 10 s, 20 s, 40 s … 300 s; the real kubelet often restarts a container’s very first crash sooner. CPU throttling is shown as usage held at the limit, not CFS periods. Memory pressure is only memory, and a node’s own processes use a fixed share.

kube-proxy picks endpoints with equal probability, like its iptables mode; traffic enters through one node. Rolling updates of StatefulSets and DaemonSets, proportional scaling of a Deployment in the middle of a rollout, minReadySeconds, Pod topology spread constraints you write yourself and the HPA’s CPU initialisation period are not modelled.

kubectl is a subset with simplified flags and messages; its output follows real kubectl’s layout. Pod template hashes are computed like Kubernetes' (FNV-32a, then its safe alphabet) but over a JSON rendering, so they differ from a real cluster’s.