Kubernetes Deep Dive: Pod Startup and Failure Modes

"Walk me through what happens from kubectl apply to the first request hitting your container." It sounds like a warm-up question, but how you answer it can decide whether you land a senior or staff role. In this deep dive we trace a pod from creation to production and unpack the failure modes interviewers expect you to anticipate.You'll learn:- The full startup chain: API server, etcd, controllers, scheduler, kubelet and container runtime- Why init containers can't run in parallel, and why migrations belong in one for security- Startup, liveness and readiness probes, and the liveness trap that turns a 10-second database blip into an outage- Native sidecars in Kubernetes 1.29 (KEP-753) and the startup and shutdown races they fix- HashiCorp Vault: mutating webhook injection, TokenReview and system:auth-delegator, and dynamic database credentials- Zero trust with mTLS, SPIFFE identities and JWTs, plus the alg: none attack and audience checks- Service DNS, ndots:5, ClusterIP "phantom" IPs, iptables vs IPVS, and headless services for gRPC- Istio traps: DestinationRule ordering, empty AuthorizationPolicies and a stray YAML hyphen- Helm secrets, migration hooks, and a five-phase rollout for breaking API changes- A layered runbook for debugging intermittent 503sChapters00:00 Introduction00:22 What interviewers are really listening for01:31 From kubectl apply to "success"03:06 The scheduler and the kubelet04:18 Init containers05:26 Why migrations belong in an init container06:34 Startup, liveness and readiness probes08:09 The liveness probe thundering herd trap10:04 What a liveness probe should check10:53 Startup probes for slow-booting apps11:40 The sidecar startup and shutdown race12:52 Native sidecars in Kubernetes 1.2913:39 Vault injection with a mutating webhook14:48 Vault's init and sidecar agents15:34 Vault authentication and TokenReview16:43 Dynamic database credentials17:55 A Postgres quoting trap18:44 Zero trust and mutual TLS19:52 SPIFFE identities in Istio policies20:17 mTLS vs JWT20:39 Istio policy traps21:46 The alg: none attack and audience checks23:20 Service DNS and the phantom ClusterIP24:31 iptables vs IPVS and headless services25:41 Istio routing and deployment order26:29 The YAML hyphen trap26:53 Helm versions and secrets27:41 Helm migration hooks28:04 Five phases for a breaking API change29:14 Debugging 503s step by step30:00 Matching symptoms to causes30:47 Final thought: abstractions leak#Kubernetes #DevOps #SRE #CloudNative #Istio #SystemDesign #InterviewPrepInterview Prep Podcast