Please enable JavaScript.
Coggle requires JavaScript to display documents.
(DevOps & Kubernetes Home Lab: Complete Reference, Your Two-Machine…
-
-
Mac (Control Plane): Your laptop that sends commands to the cluster using kubectl. It never runs containers — it only orchestrates.
Dell G5 SE (Worker Node): Ubuntu 24.04 machine at himanshu-G5-5505.local that actually runs all your containers and workloads.
Why Two Machines: Mirrors real production — engineers never SSH into cloud servers to run containers. They issue commands remotely from their workstation.
-
What is SSH: Secure Shell — an encrypted protocol that lets you control a remote Linux machine as if you were sitting in front of it. Uses port 22 by default.
openssh-server: The daemon (background service) on the Dell that listens for incoming SSH connections. Ubuntu 24.04 does NOT ship with it — you must install it manually with apt install openssh-server.
ssh-keygen -t ed25519: Generates a public/private key pair on your Mac. The private key stays on your Mac (~/.ssh/id_ed25519). The public key is copied to the server. Ed25519 is faster and more secure than RSA.
-N "" flag (no passphrase): Creates a key with no passphrase so connections are fully automatic. For production, use a passphrase and cache it via ssh-agent. For home labs, empty passphrase is fine.
ssh-copy-id: Copies your Mac's public key into ~/.ssh/authorized_keys on the Dell. After this, SSH uses your key instead of a password. You ran this with -f to force re-copy.
macOS ssh-copy-id bug: On macOS, ssh-copy-id throws "Assertion failure: scratch_dir was not set up" — this is a known OS bug, completely harmless. The key still gets copied. Use -f flag to suppress it.
Passwordless SSH: Once your public key is on the server, SSH authenticates via cryptographic challenge-response — no password typed. If it still asks for a password, the key wasn't copied correctly.
~/.ssh/config: A config file on your Mac that stores SSH connection shortcuts. Instead of typing the full IP and username every time, you define a Host alias with hostname, user, and key path.
Host dell-worker: The alias name you type instead of the full connection string. Run ssh dell-worker and it reads the config block automatically.
HostName himanshu-G5-5505.local: The actual address SSH connects to. Using .local hostname instead of raw IP means it works on any network — WiFi, hotspot, office.
User himanshu: The Linux username on the Dell. This is NOT the hostname (himanshu-G5-5505) — it's the login user. Confirm with whoami on the remote machine.
ServerAliveInterval 60: Sends a keepalive packet every 60 seconds so idle SSH connections don't drop due to router/firewall timeouts. Essential for long kubectl watch sessions.
-
The IP Problem: Raw IP addresses change every time you switch networks — home WiFi gives 192.168.1.7, mobile hotspot gives 10.213.2.71. Every switch breaks your SSH config and kubeconfig.
avahi-daemon: A Linux service that implements mDNS (Multicast DNS). It broadcasts your machine's hostname over the local network so other devices can resolve it without a DNS server.
himanshu-G5-5505.local: Your Dell's permanent hostname on any local network. The .local suffix means it's resolved via mDNS, not regular DNS. Works automatically on Mac (Bonjour) and Linux (avahi).
mDNS vs DNS: Regular DNS requires a DNS server (like 8.8.8.8) configured somewhere. mDNS is zero-config — devices multicast hostname queries on the LAN. No router setup needed.
Limitation: .local only works on the same network. If you want to SSH from a different city, use Tailscale (a VPN that gives your Dell a permanent 100.x.x.x IP regardless of network).
Tailscale (Next Step): A zero-config VPN that gives every enrolled device a permanent IP. SSH from anywhere in the world to your Dell using the same address forever. Free for personal use.
-
What is systemd: The init system on Ubuntu 24.04 that manages all background services. It starts services at boot, restarts them if they crash, and tracks their state.
systemctl start ssh: Starts the SSH service immediately right now. Does NOT survive reboot — if you restart the machine, SSH won't be running unless you also enable it.
systemctl enable ssh: Configures the SSH service to start automatically every time the machine boots. Does NOT start it right now — combine start + enable for both effects.
systemctl status ssh: Shows whether a service is running, its last few log lines, and whether it's enabled. Active: active (running) is what you want to see.
systemctl restart: Stops then starts a service. Use this after changing config files. For containerd, you restart it after deleting the broken config.toml to force config regeneration.
Why containerd needed a restart: The default /etc/containerd/config.toml disables the CRI (Container Runtime Interface) plugin, which Kubernetes requires. Deleting it forces containerd to regenerate a working config on restart.
ufw (firewall): Ubuntu's firewall. If active, it blocks all ports by default including port 22 (SSH). Run ufw allow ssh to punch a hole for SSH connections before enabling it.
-
What is Kubernetes: An open-source container orchestration system. You tell it what containers to run and how many, and it handles scheduling, networking, scaling, and recovery — automatically.
Control Plane Components (on Dell in your setup): The brain of the cluster. Receives commands from kubectl, decides where to place pods, and constantly reconciles desired vs actual state.
kube-apiserver: The front door to Kubernetes. Every kubectl command hits this REST API. Also receives watch requests from kubelets and controllers. Runs on port 6443.
kube-scheduler: Watches for new pods with no node assigned and picks the best node for them based on resource requests, taints, and affinity rules.
kube-controller-manager: Runs control loops (controllers) that watch cluster state and make changes. The Deployment controller ensures replica count is always correct. The Node controller notices when nodes go down.
etcd: A distributed key-value store that holds ALL cluster state — every pod, deployment, secret, and config. If etcd dies, the cluster loses its memory. In production, etcd is replicated across 3+ nodes.
-
kubelet: An agent running on every node. It reads pod specs from the API server, tells containerd to start/stop containers, and reports pod health back to the control plane.
kube-proxy: Maintains iptables/ipvs rules on each node to implement Service networking. When you curl a ClusterIP, kube-proxy routes the traffic to a healthy pod behind the service.
containerd: The container runtime. It pulls Docker images, creates namespaces, and starts/stops containers. Kubernetes talks to it via the CRI (Container Runtime Interface) socket at /var/run/containerd/containerd.sock.
kubeadm: A tool that automates the complex setup of a Kubernetes cluster — generating certificates, writing config files, starting control plane components, and installing CoreDNS and kube-proxy.
kubeadm init: Bootstraps a new control plane on the machine you run it on. You pass --apiserver-advertise-address so the API server binds to your LAN IP, not just localhost.
--pod-network-cidr=10.244.0.0/16: Reserves the 10.244.x.x IP range for pods. Flannel (the CNI plugin) requires this specific range. Every pod in your cluster gets an IP from this pool.
kubeadm reset -f: Completely wipes a previous Kubernetes installation — removes certificates, configs, and networking rules. Run this before re-initializing if init fails with "port already in use" errors.
Control plane taint removal: By default, the control plane node has a taint preventing pod scheduling on it. Since your Dell is the only node, you remove this taint so your workloads can run there.
kubeconfig (~/.kube/config): A YAML file that tells kubectl how to connect to a cluster — the API server address, authentication certificate, and cluster name. You copy this from the Dell to your Mac to enable remote control.
sed -i '' to replace IP: After copying kubeconfig to Mac, the server address says 127.0.0.1 (localhost on Dell). You replace it with the Dell's .local hostname so Mac's kubectl reaches across the network.
CNI (Container Network Interface): A plugin that sets up pod networking. Without it, pods can't talk to each other. Flannel is the simplest CNI — it creates a flat overlay network across all nodes.
Flannel: Creates a vxlan overlay network. Each node gets a subnet (e.g. 10.244.0.0/24) and Flannel ensures traffic between pods on different nodes is tunneled correctly.
-
Namespace: A virtual cluster inside your physical cluster. Resources in different namespaces are isolated — you can have a pod named "webapp" in both dev and prod namespaces with no conflict.
devops-lab namespace: The namespace you created for this project. All your deployments, services, and configs live here. Keeps lab work separate from kube-system (Kubernetes internals).
kube-system namespace: Where Kubernetes runs its own components — CoreDNS, kube-proxy, Flannel, metrics-server. Never delete things here unless you know exactly what you're doing.
Resource isolation: You can set ResourceQuotas per namespace to limit how much CPU/memory a team or environment can consume. Standard practice in multi-tenant clusters.
Pod: The smallest deployable unit in Kubernetes. A pod wraps one or more containers that share a network namespace (same IP) and can share storage volumes.
Ephemeral nature: Pods are designed to die. If a pod crashes, Kubernetes creates a new one — it doesn't try to resurrect the dead pod. Never store state inside a pod's filesystem.
Pod IP: Every pod gets a unique IP inside the cluster network. This IP is NOT stable — it changes when the pod is replaced. Always access pods via a Service, not their direct IP.
Multi-container pods: Advanced pattern where a main app container and a sidecar container (logging, proxy) share the same pod. The Istio service mesh uses this pattern extensively.
Deployment: A higher-level object that manages pods. You declare desired state (e.g. 2 replicas of nginx) and the Deployment controller continuously ensures that state is maintained.
spec.replicas: How many identical pod copies to run. Set to 2 in your tutorial. The controller creates new pods if count drops below this, and terminates excess pods if count exceeds it.
selector.matchLabels: Tells the Deployment which pods it owns. Pods with matching labels (app: webapp) are managed by this Deployment. Labels are the glue holding Kubernetes together.
Rolling update strategy: The default strategy. When you change the image, Kubernetes brings up new pods one at a time before terminating old ones. Ensures zero downtime during deploys.
resources.requests: The minimum CPU/memory a pod needs. The scheduler uses this to find a node with enough free capacity. 100m CPU = 0.1 CPU cores. 64Mi = 64 megabytes RAM.
resources.limits: The maximum a pod can consume. If it exceeds the memory limit, Kubernetes OOMKills it. Prevents one runaway pod from starving other pods on the same node.
ReplicaSet: The object a Deployment creates to manage pod replicas. The Deployment owns the ReplicaSet; the ReplicaSet owns the pods. You rarely interact with ReplicaSets directly.
Service: A stable network endpoint in front of a group of pods. Since pod IPs change, Services give you a fixed ClusterIP and DNS name that always routes to healthy pods.
selector: Services use label selectors to find their target pods. Any pod with label app: webapp is automatically added to the Service's endpoint list when it becomes healthy.
ClusterIP: The default Service type. Gives a stable internal IP only reachable inside the cluster. Used for microservice-to-microservice communication.
NodePort: Exposes the service on a port (30000-32767) on every node's IP. This is how you accessed your app at himanshu-G5-5505.local:30080 from your browser.
LoadBalancer: Cloud-specific type. Creates an external load balancer (like an AWS ELB) with a public IP. Not available in bare-metal home labs without MetalLB.
-
Cluster Network Model: Every pod gets a unique IP. All pods can talk to all other pods directly (no NAT). This flat network is created by the CNI plugin (Flannel).
kube-proxy: Runs on every node and maintains iptables rules. When a packet hits a ClusterIP, kube-proxy's rules DNAT it to one of the healthy pod IPs behind the service.
CoreDNS: A DNS server running inside the cluster as pods in kube-system. Every service and pod gets a DNS record. This is how services find each other without hardcoded IPs.
DNS format: service-name.namespace.svc.cluster.local — e.g. webapp-service.devops-lab.svc.cluster.local. Within the same namespace, you can use just the service name.
Why DNS not IPs: Pod IPs change on restart. Service ClusterIPs are stable but only within one cluster. DNS names work across cluster recreations. Hardcoded IPs are an anti-pattern.
NodePort mechanics: The service listens on port 80 internally. kube-proxy opens port 30080 on the node. Requests to himanshu-G5-5505.local:30080 hit kube-proxy, which forwards to port 80 on a pod.
kubectl port-forward: Creates a tunnel from your Mac's localhost port directly to a specific pod's port. Bypasses the Service entirely. Used to debug one individual pod without affecting production traffic.
Ingress (Next Step): A layer 7 (HTTP) router. Instead of NodePort per service, one Ingress controller handles all traffic and routes by hostname or URL path. Standard in production clusters.
-
Why external config: Docker images should be immutable — the same image runs in dev, staging, and prod. Environment-specific config (database URLs, API keys) is injected at runtime, not baked in.
12-Factor App methodology: A set of principles for building cloud-native apps. Factor III says "store config in the environment" — never hardcode URLs or credentials. Kubernetes ConfigMaps and Secrets implement this.
ConfigMap: Stores non-sensitive configuration as key-value pairs. Mounted into pods as environment variables or as files. Values are stored in plain text in etcd — do not use for secrets.
--from-literal: Creates a ConfigMap entry inline from the command line. For larger configs, use --from-file to load an entire config file (nginx.conf, app.properties) into a ConfigMap.
valueFrom.configMapKeyRef: Injects a specific ConfigMap key as an environment variable inside the container. The app reads os.environ["APP_ENV"] and gets "production" without knowing about Kubernetes.
Secret: Stores sensitive data (passwords, API keys, TLS certs). Values are base64-encoded in etcd (NOT encrypted by default). In production, enable etcd encryption at rest or use HashiCorp Vault.
base64 encoding: Not encryption — just encoding. Anyone with kubectl get secret access can decode values. RBAC (Role-Based Access Control) restricts who can read secrets in production.
secretKeyRef: Injects a secret value as an environment variable. The container sees DB_PASSWORD=supersecretpassword without knowing the value is stored in Kubernetes.
Never commit to Git: Kubernetes Secret YAML files contain base64 values that are trivially decoded. Use tools like Sealed Secrets, External Secrets Operator, or Vault to manage secrets in GitOps workflows.
HashiCorp Vault (Production Pattern): A dedicated secrets management system. Apps authenticate to Vault and retrieve secrets dynamically. Secrets are encrypted, versioned, and audited. The industry standard for serious secret management.
-
Reconciliation Loop: The core Kubernetes philosophy. Controllers constantly compare desired state (what you declared in YAML) with actual state (what's running). Any divergence triggers automatic corrective action.
kubectl apply -f: The idempotent way to create or update resources. Kubernetes computes the diff between your YAML and current state and applies only the changes. Running it twice is safe.
Rolling Update: Kubernetes replaces pods one at a time (by default). It starts a new pod, waits for it to pass readiness checks, then terminates one old pod. Continues until all pods run the new version.
maxSurge: How many extra pods can exist above the desired count during an update. Default is 25%. With 4 replicas, up to 5 pods may exist temporarily during a rolling update.
maxUnavailable: How many pods can be unavailable during an update. Default is 25%. With 4 replicas, at most 1 pod can be down at any moment. Set to 0 for zero-downtime guaranteed.
kubectl set image: Updates the container image in a Deployment, triggering a rolling update. In production, your CI/CD pipeline (GitHub Actions, Jenkins) runs this automatically after building a new image.
kubectl rollout status: Watches and reports the progress of a rolling update in real time. Exits 0 when complete, exits 1 if the rollout gets stuck (e.g. new pods fail readiness checks).
kubectl rollout undo: Reverts to the previous Deployment revision. Kubernetes stores rollout history (default: 10 revisions). Fastest way to recover from a bad deploy — runs in seconds.
kubectl rollout history: Lists all previous Deployment revisions with change-cause annotations. Add --revision=3 to see what image/config a specific revision used.
kubectl scale: Manually changes the replica count. In production this is done automatically by the Horizontal Pod Autoscaler (HPA) based on CPU/memory/custom metrics.
Self-Healing: When a pod crashes or is deleted, the Deployment controller notices the replica count dropped below desired. It immediately creates a replacement pod on any available node.
How it works: The Deployment controller runs a watch loop on the API server. When pod count drops, it creates a new pod spec and the scheduler picks a node. The whole recovery takes 5-30 seconds.
ImagePullBackOff: Error when Kubernetes can't pull the container image — wrong tag, private registry without credentials, or no internet. Fix with kubectl rollout undo or correct the image name.
CrashLoopBackOff: Pod starts, crashes immediately, Kubernetes restarts it, crashes again — exponential backoff. Indicates an app bug, missing env var, or wrong command. Debug with kubectl logs.
-
kubectl describe pod: The single most useful debugging command. Shows the full event log for a pod — image pull status, scheduling decisions, health check failures, OOMKill events, and resource usage.
kubectl logs: Streams stdout/stderr from a container. Add -f to follow in real time. Add --previous to see logs from a crashed container (before it restarted). Add --since=1h for recent logs only.
-l app=webapp: Log aggregation across all pods matching a label selector. Shows logs from all replicas simultaneously with --prefix to distinguish which pod each line came from.
kubectl exec -it: Opens an interactive shell inside a running container — equivalent to SSH for containers. Use /bin/sh (not /bin/bash) for minimal images like alpine that don't include bash.
Read-only principle: In production, exec for debugging only — never to fix a running container. Containers are immutable. If you need to change something, change the Dockerfile, rebuild, and redeploy.
kubectl get events: Shows all Kubernetes events in a namespace — pod scheduling, image pulls, OOMKills, scaling actions. Sorted by --sort-by='.lastTimestamp'. The first place to look during an incident.
kubectl top pods: Shows live CPU and memory usage per pod. Requires metrics-server to be installed. Essential for spotting memory leaks, CPU spikes, and over/under-provisioned resource limits.
metrics-server: A lightweight aggregator that scrapes resource usage from each node's kubelet. Provides the data for kubectl top and is also required for Horizontal Pod Autoscaler (HPA) to function.
--kubelet-insecure-tls: Flag needed in home labs where kubelets use self-signed certs. Never use this flag in production — configure proper TLS certificate verification instead.
kubectl port-forward (debugging): Tunnels traffic from localhost on your Mac to a specific pod. Lets you curl or browse a pod directly, bypassing load balancing. Great for testing one broken pod in isolation.
Readiness vs Liveness probes: Readiness — is the pod ready to receive traffic? Liveness — is the pod still alive? Kubernetes removes unready pods from Service endpoints and restarts pods that fail liveness checks.
-
Your Mac kubectl = Cloud engineer's laptop: Every cloud engineer runs kubectl locally and connects to remote clusters (EKS, GKE, AKS) via kubeconfig. Exactly what you built.
Dell G5 SE = EC2/GCE instance: Your physical Dell plays the same role as a cloud VM — it runs containers and reports to the control plane. In production there are tens or hundreds of such nodes.
himanshu-G5-5505.local = Internal DNS hostname: Production servers use hostnames like ip-10-0-1-42.ec2.internal or node1.us-east1.gke.internal. Same principle — stable names over volatile IPs.
NodePort = Poor man's LoadBalancer: NodePort works for home labs. In production, a cloud LoadBalancer or Ingress controller handles external traffic with SSL termination and hostname routing.
ConfigMap/Secret = AWS Parameter Store / Vault: Same pattern — inject config at runtime, not at build time. In AWS, apps fetch config from SSM Parameter Store or Secrets Manager instead of Kubernetes objects.
kubectl rollout undo = Emergency rollback in CI/CD: Production pipelines have automated rollback hooks — if health checks fail after deploy, the pipeline runs rollout undo automatically. You did this manually here.
Reconciliation loop = Infrastructure as Code enforcement: Tools like Argo CD watch a Git repo and ensure the cluster always matches the YAML in Git — if someone manually changes something, Argo CD reverts it. Same reconciliation philosophy.
-
Ingress Controller: Replaces multiple NodePort services with a single HTTP router. Install nginx-ingress, define Ingress rules by hostname (webapp.local, api.local), and reach all apps on port 80/443.
Helm: The package manager for Kubernetes. Charts are templated collections of YAML that install entire applications (Prometheus, PostgreSQL, cert-manager) with a single command and sane defaults.
Persistent Volumes: Storage that survives pod restarts. A PersistentVolumeClaim requests storage; a PVC mounts it into a pod. Required for databases — without it, all data is lost when the pod restarts.
Argo CD (GitOps): Watches a Git repository and automatically applies changes to the cluster when you push. The cluster always mirrors Git. No manual kubectl apply — Git is the single source of truth.
Prometheus + Grafana: The standard observability stack. Prometheus scrapes metrics from all pods. Grafana visualizes them as dashboards. Alertmanager sends alerts to Slack/PagerDuty when things go wrong.
Horizontal Pod Autoscaler: Automatically scales your Deployment up/down based on CPU usage or custom metrics. When traffic spikes, HPA adds pods. When it drops, HPA removes them. Requires metrics-server.
Tailscale: Adds a permanent IP to your Dell that works from anywhere in the world. SSH into your home lab from a cafe. Extends your setup to work beyond local network limitations.
cert-manager: Automatically provisions and renews TLS certificates from Let's Encrypt for your Ingress hostnames. Your apps get HTTPS for free, with zero manual certificate management.
Network Policies: Kubernetes-native firewall rules between pods. By default all pods can talk to all pods. Network Policies let you say "only the frontend pod can talk to the backend pod" — zero-trust networking.
-
-
Mac (Control Plane): Your laptop that sends commands to the cluster using kubectl. It never runs containers — it only orchestrates.
Dell G5 SE (Worker Node): Ubuntu 24.04 machine at himanshu-G5-5505.local that actually runs all your containers and workloads.
Why Two Machines: Mirrors real production — engineers never SSH into cloud servers to run containers. They issue commands remotely from their workstation.
-
What is SSH: Secure Shell — an encrypted protocol that lets you control a remote Linux machine as if you were sitting in front of it. Uses port 22 by default.
openssh-server: The daemon (background service) on the Dell that listens for incoming SSH connections. Ubuntu 24.04 does NOT ship with it — you must install it manually with apt install openssh-server.
ssh-keygen -t ed25519: Generates a public/private key pair on your Mac. The private key stays on your Mac (~/.ssh/id_ed25519). The public key is copied to the server. Ed25519 is faster and more secure than RSA.
-N "" flag (no passphrase): Creates a key with no passphrase so connections are fully automatic. For production, use a passphrase and cache it via ssh-agent. For home labs, empty passphrase is fine.
ssh-copy-id: Copies your Mac's public key into ~/.ssh/authorized_keys on the Dell. After this, SSH uses your key instead of a password. You ran this with -f to force re-copy.
macOS ssh-copy-id bug: On macOS, ssh-copy-id throws "Assertion failure: scratch_dir was not set up" — this is a known OS bug, completely harmless. The key still gets copied. Use -f flag to suppress it.
Passwordless SSH: Once your public key is on the server, SSH authenticates via cryptographic challenge-response — no password typed. If it still asks for a password, the key wasn't copied correctly.
~/.ssh/config: A config file on your Mac that stores SSH connection shortcuts. Instead of typing the full IP and username every time, you define a Host alias with hostname, user, and key path.
Host dell-worker: The alias name you type instead of the full connection string. Run ssh dell-worker and it reads the config block automatically.
HostName himanshu-G5-5505.local: The actual address SSH connects to. Using .local hostname instead of raw IP means it works on any network — WiFi, hotspot, office.
User himanshu: The Linux username on the Dell. This is NOT the hostname (himanshu-G5-5505) — it's the login user. Confirm with whoami on the remote machine.
ServerAliveInterval 60: Sends a keepalive packet every 60 seconds so idle SSH connections don't drop due to router/firewall timeouts. Essential for long kubectl watch sessions.
-
The IP Problem: Raw IP addresses change every time you switch networks — home WiFi gives 192.168.1.7, mobile hotspot gives 10.213.2.71. Every switch breaks your SSH config and kubeconfig.
avahi-daemon: A Linux service that implements mDNS (Multicast DNS). It broadcasts your machine's hostname over the local network so other devices can resolve it without a DNS server.
himanshu-G5-5505.local: Your Dell's permanent hostname on any local network. The .local suffix means it's resolved via mDNS, not regular DNS. Works automatically on Mac (Bonjour) and Linux (avahi).
mDNS vs DNS: Regular DNS requires a DNS server (like 8.8.8.8) configured somewhere. mDNS is zero-config — devices multicast hostname queries on the LAN. No router setup needed.
Limitation: .local only works on the same network. If you want to SSH from a different city, use Tailscale (a VPN that gives your Dell a permanent 100.x.x.x IP regardless of network).
Tailscale (Next Step): A zero-config VPN that gives every enrolled device a permanent IP. SSH from anywhere in the world to your Dell using the same address forever. Free for personal use.
-
What is systemd: The init system on Ubuntu 24.04 that manages all background services. It starts services at boot, restarts them if they crash, and tracks their state.
systemctl start ssh: Starts the SSH service immediately right now. Does NOT survive reboot — if you restart the machine, SSH won't be running unless you also enable it.
systemctl enable ssh: Configures the SSH service to start automatically every time the machine boots. Does NOT start it right now — combine start + enable for both effects.
systemctl status ssh: Shows whether a service is running, its last few log lines, and whether it's enabled. Active: active (running) is what you want to see.
systemctl restart: Stops then starts a service. Use this after changing config files. For containerd, you restart it after deleting the broken config.toml to force config regeneration.
Why containerd needed a restart: The default /etc/containerd/config.toml disables the CRI (Container Runtime Interface) plugin, which Kubernetes requires. Deleting it forces containerd to regenerate a working config on restart.
ufw (firewall): Ubuntu's firewall. If active, it blocks all ports by default including port 22 (SSH). Run ufw allow ssh to punch a hole for SSH connections before enabling it.
-
What is Kubernetes: An open-source container orchestration system. You tell it what containers to run and how many, and it handles scheduling, networking, scaling, and recovery — automatically.
Control Plane Components (on Dell in your setup): The brain of the cluster. Receives commands from kubectl, decides where to place pods, and constantly reconciles desired vs actual state.
kube-apiserver: The front door to Kubernetes. Every kubectl command hits this REST API. Also receives watch requests from kubelets and controllers. Runs on port 6443.
kube-scheduler: Watches for new pods with no node assigned and picks the best node for them based on resource requests, taints, and affinity rules.
kube-controller-manager: Runs control loops (controllers) that watch cluster state and make changes. The Deployment controller ensures replica count is always correct. The Node controller notices when nodes go down.
etcd: A distributed key-value store that holds ALL cluster state — every pod, deployment, secret, and config. If etcd dies, the cluster loses its memory. In production, etcd is replicated across 3+ nodes.
-
kubelet: An agent running on every node. It reads pod specs from the API server, tells containerd to start/stop containers, and reports pod health back to the control plane.
kube-proxy: Maintains iptables/ipvs rules on each node to implement Service networking. When you curl a ClusterIP, kube-proxy routes the traffic to a healthy pod behind the service.
containerd: The container runtime. It pulls Docker images, creates namespaces, and starts/stops containers. Kubernetes talks to it via the CRI (Container Runtime Interface) socket at /var/run/containerd/containerd.sock.
kubeadm: A tool that automates the complex setup of a Kubernetes cluster — generating certificates, writing config files, starting control plane components, and installing CoreDNS and kube-proxy.
kubeadm init: Bootstraps a new control plane on the machine you run it on. You pass --apiserver-advertise-address so the API server binds to your LAN IP, not just localhost.
--pod-network-cidr=10.244.0.0/16: Reserves the 10.244.x.x IP range for pods. Flannel (the CNI plugin) requires this specific range. Every pod in your cluster gets an IP from this pool.
kubeadm reset -f: Completely wipes a previous Kubernetes installation — removes certificates, configs, and networking rules. Run this before re-initializing if init fails with "port already in use" errors.
Control plane taint removal: By default, the control plane node has a taint preventing pod scheduling on it. Since your Dell is the only node, you remove this taint so your workloads can run there.
kubeconfig (~/.kube/config): A YAML file that tells kubectl how to connect to a cluster — the API server address, authentication certificate, and cluster name. You copy this from the Dell to your Mac to enable remote control.
sed -i '' to replace IP: After copying kubeconfig to Mac, the server address says 127.0.0.1 (localhost on Dell). You replace it with the Dell's .local hostname so Mac's kubectl reaches across the network.
CNI (Container Network Interface): A plugin that sets up pod networking. Without it, pods can't talk to each other. Flannel is the simplest CNI — it creates a flat overlay network across all nodes.
Flannel: Creates a vxlan overlay network. Each node gets a subnet (e.g. 10.244.0.0/24) and Flannel ensures traffic between pods on different nodes is tunneled correctly.
-
Namespace: A virtual cluster inside your physical cluster. Resources in different namespaces are isolated — you can have a pod named "webapp" in both dev and prod namespaces with no conflict.
devops-lab namespace: The namespace you created for this project. All your deployments, services, and configs live here. Keeps lab work separate from kube-system (Kubernetes internals).
kube-system namespace: Where Kubernetes runs its own components — CoreDNS, kube-proxy, Flannel, metrics-server. Never delete things here unless you know exactly what you're doing.
Resource isolation: You can set ResourceQuotas per namespace to limit how much CPU/memory a team or environment can consume. Standard practice in multi-tenant clusters.
Pod: The smallest deployable unit in Kubernetes. A pod wraps one or more containers that share a network namespace (same IP) and can share storage volumes.
Ephemeral nature: Pods are designed to die. If a pod crashes, Kubernetes creates a new one — it doesn't try to resurrect the dead pod. Never store state inside a pod's filesystem.
Pod IP: Every pod gets a unique IP inside the cluster network. This IP is NOT stable — it changes when the pod is replaced. Always access pods via a Service, not their direct IP.
Multi-container pods: Advanced pattern where a main app container and a sidecar container (logging, proxy) share the same pod. The Istio service mesh uses this pattern extensively.
Deployment: A higher-level object that manages pods. You declare desired state (e.g. 2 replicas of nginx) and the Deployment controller continuously ensures that state is maintained.
spec.replicas: How many identical pod copies to run. Set to 2 in your tutorial. The controller creates new pods if count drops below this, and terminates excess pods if count exceeds it.
selector.matchLabels: Tells the Deployment which pods it owns. Pods with matching labels (app: webapp) are managed by this Deployment. Labels are the glue holding Kubernetes together.
Rolling update strategy: The default strategy. When you change the image, Kubernetes brings up new pods one at a time before terminating old ones. Ensures zero downtime during deploys.
resources.requests: The minimum CPU/memory a pod needs. The scheduler uses this to find a node with enough free capacity. 100m CPU = 0.1 CPU cores. 64Mi = 64 megabytes RAM.
resources.limits: The maximum a pod can consume. If it exceeds the memory limit, Kubernetes OOMKills it. Prevents one runaway pod from starving other pods on the same node.
ReplicaSet: The object a Deployment creates to manage pod replicas. The Deployment owns the ReplicaSet; the ReplicaSet owns the pods. You rarely interact with ReplicaSets directly.
Service: A stable network endpoint in front of a group of pods. Since pod IPs change, Services give you a fixed ClusterIP and DNS name that always routes to healthy pods.
selector: Services use label selectors to find their target pods. Any pod with label app: webapp is automatically added to the Service's endpoint list when it becomes healthy.
ClusterIP: The default Service type. Gives a stable internal IP only reachable inside the cluster. Used for microservice-to-microservice communication.
NodePort: Exposes the service on a port (30000-32767) on every node's IP. This is how you accessed your app at himanshu-G5-5505.local:30080 from your browser.
LoadBalancer: Cloud-specific type. Creates an external load balancer (like an AWS ELB) with a public IP. Not available in bare-metal home labs without MetalLB.
-
Cluster Network Model: Every pod gets a unique IP. All pods can talk to all other pods directly (no NAT). This flat network is created by the CNI plugin (Flannel).
kube-proxy: Runs on every node and maintains iptables rules. When a packet hits a ClusterIP, kube-proxy's rules DNAT it to one of the healthy pod IPs behind the service.
CoreDNS: A DNS server running inside the cluster as pods in kube-system. Every service and pod gets a DNS record. This is how services find each other without hardcoded IPs.
DNS format: service-name.namespace.svc.cluster.local — e.g. webapp-service.devops-lab.svc.cluster.local. Within the same namespace, you can use just the service name.
Why DNS not IPs: Pod IPs change on restart. Service ClusterIPs are stable but only within one cluster. DNS names work across cluster recreations. Hardcoded IPs are an anti-pattern.
NodePort mechanics: The service listens on port 80 internally. kube-proxy opens port 30080 on the node. Requests to himanshu-G5-5505.local:30080 hit kube-proxy, which forwards to port 80 on a pod.
kubectl port-forward: Creates a tunnel from your Mac's localhost port directly to a specific pod's port. Bypasses the Service entirely. Used to debug one individual pod without affecting production traffic.
Ingress (Next Step): A layer 7 (HTTP) router. Instead of NodePort per service, one Ingress controller handles all traffic and routes by hostname or URL path. Standard in production clusters.
-
Why external config: Docker images should be immutable — the same image runs in dev, staging, and prod. Environment-specific config (database URLs, API keys) is injected at runtime, not baked in.
12-Factor App methodology: A set of principles for building cloud-native apps. Factor III says "store config in the environment" — never hardcode URLs or credentials. Kubernetes ConfigMaps and Secrets implement this.
ConfigMap: Stores non-sensitive configuration as key-value pairs. Mounted into pods as environment variables or as files. Values are stored in plain text in etcd — do not use for secrets.
--from-literal: Creates a ConfigMap entry inline from the command line. For larger configs, use --from-file to load an entire config file (nginx.conf, app.properties) into a ConfigMap.
valueFrom.configMapKeyRef: Injects a specific ConfigMap key as an environment variable inside the container. The app reads os.environ["APP_ENV"] and gets "production" without knowing about Kubernetes.
Secret: Stores sensitive data (passwords, API keys, TLS certs). Values are base64-encoded in etcd (NOT encrypted by default). In production, enable etcd encryption at rest or use HashiCorp Vault.
base64 encoding: Not encryption — just encoding. Anyone with kubectl get secret access can decode values. RBAC (Role-Based Access Control) restricts who can read secrets in production.
secretKeyRef: Injects a secret value as an environment variable. The container sees DB_PASSWORD=supersecretpassword without knowing the value is stored in Kubernetes.
Never commit to Git: Kubernetes Secret YAML files contain base64 values that are trivially decoded. Use tools like Sealed Secrets, External Secrets Operator, or Vault to manage secrets in GitOps workflows.
HashiCorp Vault (Production Pattern): A dedicated secrets management system. Apps authenticate to Vault and retrieve secrets dynamically. Secrets are encrypted, versioned, and audited. The industry standard for serious secret management.
-
Reconciliation Loop: The core Kubernetes philosophy. Controllers constantly compare desired state (what you declared in YAML) with actual state (what's running). Any divergence triggers automatic corrective action.
kubectl apply -f: The idempotent way to create or update resources. Kubernetes computes the diff between your YAML and current state and applies only the changes. Running it twice is safe.
Rolling Update: Kubernetes replaces pods one at a time (by default). It starts a new pod, waits for it to pass readiness checks, then terminates one old pod. Continues until all pods run the new version.
maxSurge: How many extra pods can exist above the desired count during an update. Default is 25%. With 4 replicas, up to 5 pods may exist temporarily during a rolling update.
maxUnavailable: How many pods can be unavailable during an update. Default is 25%. With 4 replicas, at most 1 pod can be down at any moment. Set to 0 for zero-downtime guaranteed.
kubectl set image: Updates the container image in a Deployment, triggering a rolling update. In production, your CI/CD pipeline (GitHub Actions, Jenkins) runs this automatically after building a new image.
kubectl rollout status: Watches and reports the progress of a rolling update in real time. Exits 0 when complete, exits 1 if the rollout gets stuck (e.g. new pods fail readiness checks).
kubectl rollout undo: Reverts to the previous Deployment revision. Kubernetes stores rollout history (default: 10 revisions). Fastest way to recover from a bad deploy — runs in seconds.
kubectl rollout history: Lists all previous Deployment revisions with change-cause annotations. Add --revision=3 to see what image/config a specific revision used.
kubectl scale: Manually changes the replica count. In production this is done automatically by the Horizontal Pod Autoscaler (HPA) based on CPU/memory/custom metrics.
Self-Healing: When a pod crashes or is deleted, the Deployment controller notices the replica count dropped below desired. It immediately creates a replacement pod on any available node.
How it works: The Deployment controller runs a watch loop on the API server. When pod count drops, it creates a new pod spec and the scheduler picks a node. The whole recovery takes 5-30 seconds.
ImagePullBackOff: Error when Kubernetes can't pull the container image — wrong tag, private registry without credentials, or no internet. Fix with kubectl rollout undo or correct the image name.
CrashLoopBackOff: Pod starts, crashes immediately, Kubernetes restarts it, crashes again — exponential backoff. Indicates an app bug, missing env var, or wrong command. Debug with kubectl logs.
-
kubectl describe pod: The single most useful debugging command. Shows the full event log for a pod — image pull status, scheduling decisions, health check failures, OOMKill events, and resource usage.
kubectl logs: Streams stdout/stderr from a container. Add -f to follow in real time. Add --previous to see logs from a crashed container (before it restarted). Add --since=1h for recent logs only.
-l app=webapp: Log aggregation across all pods matching a label selector. Shows logs from all replicas simultaneously with --prefix to distinguish which pod each line came from.
kubectl exec -it: Opens an interactive shell inside a running container — equivalent to SSH for containers. Use /bin/sh (not /bin/bash) for minimal images like alpine that don't include bash.
Read-only principle: In production, exec for debugging only — never to fix a running container. Containers are immutable. If you need to change something, change the Dockerfile, rebuild, and redeploy.
kubectl get events: Shows all Kubernetes events in a namespace — pod scheduling, image pulls, OOMKills, scaling actions. Sorted by --sort-by='.lastTimestamp'. The first place to look during an incident.
kubectl top pods: Shows live CPU and memory usage per pod. Requires metrics-server to be installed. Essential for spotting memory leaks, CPU spikes, and over/under-provisioned resource limits.
metrics-server: A lightweight aggregator that scrapes resource usage from each node's kubelet. Provides the data for kubectl top and is also required for Horizontal Pod Autoscaler (HPA) to function.
--kubelet-insecure-tls: Flag needed in home labs where kubelets use self-signed certs. Never use this flag in production — configure proper TLS certificate verification instead.
kubectl port-forward (debugging): Tunnels traffic from localhost on your Mac to a specific pod. Lets you curl or browse a pod directly, bypassing load balancing. Great for testing one broken pod in isolation.
Readiness vs Liveness probes: Readiness — is the pod ready to receive traffic? Liveness — is the pod still alive? Kubernetes removes unready pods from Service endpoints and restarts pods that fail liveness checks.
-
Your Mac kubectl = Cloud engineer's laptop: Every cloud engineer runs kubectl locally and connects to remote clusters (EKS, GKE, AKS) via kubeconfig. Exactly what you built.
Dell G5 SE = EC2/GCE instance: Your physical Dell plays the same role as a cloud VM — it runs containers and reports to the control plane. In production there are tens or hundreds of such nodes.
himanshu-G5-5505.local = Internal DNS hostname: Production servers use hostnames like ip-10-0-1-42.ec2.internal or node1.us-east1.gke.internal. Same principle — stable names over volatile IPs.
NodePort = Poor man's LoadBalancer: NodePort works for home labs. In production, a cloud LoadBalancer or Ingress controller handles external traffic with SSL termination and hostname routing.
ConfigMap/Secret = AWS Parameter Store / Vault: Same pattern — inject config at runtime, not at build time. In AWS, apps fetch config from SSM Parameter Store or Secrets Manager instead of Kubernetes objects.
kubectl rollout undo = Emergency rollback in CI/CD: Production pipelines have automated rollback hooks — if health checks fail after deploy, the pipeline runs rollout undo automatically. You did this manually here.
Reconciliation loop = Infrastructure as Code enforcement: Tools like Argo CD watch a Git repo and ensure the cluster always matches the YAML in Git — if someone manually changes something, Argo CD reverts it. Same reconciliation philosophy.
-
Ingress Controller: Replaces multiple NodePort services with a single HTTP router. Install nginx-ingress, define Ingress rules by hostname (webapp.local, api.local), and reach all apps on port 80/443.
Helm: The package manager for Kubernetes. Charts are templated collections of YAML that install entire applications (Prometheus, PostgreSQL, cert-manager) with a single command and sane defaults.
Persistent Volumes: Storage that survives pod restarts. A PersistentVolumeClaim requests storage; a PVC mounts it into a pod. Required for databases — without it, all data is lost when the pod restarts.
Argo CD (GitOps): Watches a Git repository and automatically applies changes to the cluster when you push. The cluster always mirrors Git. No manual kubectl apply — Git is the single source of truth.
Prometheus + Grafana: The standard observability stack. Prometheus scrapes metrics from all pods. Grafana visualizes them as dashboards. Alertmanager sends alerts to Slack/PagerDuty when things go wrong.
Horizontal Pod Autoscaler: Automatically scales your Deployment up/down based on CPU usage or custom metrics. When traffic spikes, HPA adds pods. When it drops, HPA removes them. Requires metrics-server.
Tailscale: Adds a permanent IP to your Dell that works from anywhere in the world. SSH into your home lab from a cafe. Extends your setup to work beyond local network limitations.
cert-manager: Automatically provisions and renews TLS certificates from Let's Encrypt for your Ingress hostnames. Your apps get HTTPS for free, with zero manual certificate management.
Network Policies: Kubernetes-native firewall rules between pods. By default all pods can talk to all pods. Network Policies let you say "only the frontend pod can talk to the backend pod" — zero-trust networking.