chapter-zero

KO | EN

gitlite — search

#swift #agent #large-language-models #cli #sql #agents #bash #typescript #html #java

(★ 9)

Kubernetes The Hard Way asks for four Debian 12 machines. This is where they come from — libvirt/KVM cloud images on your own host.

File Explorer

ZIP Download

- fetch-base
- fetch-debian-base
- k8s-cluster-rm
- k8s-cluster-up
- kthw-stand-rm
- kthw-stand-up
- vm-console
- vm-ip
- vm-rm
- vm-up
- vm-up-k8s-node
- user-data.yaml
- demo.gif
- distros.sh
- net.sh
- vm.sh
- base.sh
- k8s-node.sh
- kthw.sh
.gitignore
KTHW-PLAN.md
LICENSE
README.md
TODO.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

> Kubernetes The Hard Way starts at lab 01. This is chapter zero.

![One command, four machines to the lab 01 spec: static IPs, machines.txt, and a
host that calls itself server](docs/demo.gif)

Hightower's tutorial opens by asking for four Debian 12 machines and then says
nothing more about where they come from. This is that part: a fast cloud-image
VM workflow for local `virsh`/libvirt, and the k8s lab stands built on top of it.

1. Keep one immutable cloud image per distro in `images/`.
2. Start each VM from a small qcow2 overlay.
3. Inject the host user's SSH public key through cloud-init.

This avoids reinstalling Debian and avoids cloning identical SSH host keys into
every VM.

## Requirements

Everything runs against the host's system libvirt (`qemu:///system`), so this
is a Linux host with KVM. You need these commands on the host:

| Command | Debian/Ubuntu package | Arch package | Used for |
|---|---|---|---|
| `libvirtd` + KVM | `libvirt-daemon-system` `qemu-system-x86` | `libvirt` `qemu-desktop` | the hypervisor |
| `virsh` | `libvirt-clients` | `libvirt` | domains, networks, DHCP leases |
| `virt-install` | `virtinst` | `virt-install` | creating domains |
| `qemu-img` | `qemu-utils` | `qemu-img` | qcow2 overlay disks |
| `xorriso` | `xorriso` | `libisoburn` | the NoCloud seed ISO |
| `curl` or `wget` | `curl` / `wget` | `curl` / `wget` | `fetch-base` downloads |
| `ssh` | `openssh-client` | `openssh` | logging into guests |
| `osinfo-query` (optional) | `libosinfo-bin` | `libosinfo` | resolve the osinfo id; degrades gracefully if absent |
| `dhcp_release` (optional) | `dnsmasq-base` | `dnsmasq` | drop DHCP leases on teardown, so an address can be reused straight away |

Then make sure the daemon is running and the `default` network is up:

```bash
sudo systemctl enable --now libvirtd
sudo virsh net-start default && sudo virsh net-autostart default
```

Add your user to the `libvirt` (and `kvm`) group, or run as root, so the
scripts can reach `qemu:///system`. Guests install `qemu-guest-agent`
themselves through cloud-init — `vm-ip` asks it for the guest's address.

## First run

```bash
cd ~/devel/chapter-zero
bin/fetch-base debian
bin/vm-up cka1
bin/vm-ip cka1
ssh debian@$(bin/vm-ip cka1)
```

## Commands

```bash
bin/fetch-base [DISTRO]
bin/vm-up NAME [DISK_SIZE]
bin/vm-ip NAME
bin/vm-console NAME
bin/vm-rm NAME
```

## Layout

```
lib/distros.sh    cloud image registry: image, user, admin group, seed, video
lib/net.sh        dhcp-host reservations — static IPs on a libvirt network
lib/vm.sh         vm_create(): validation, seed ISO, overlay, virt-install
profiles/         one file per cloud-config flavour
bin/              thin wrappers that pick a profile and call vm_create
```

Every VM takes the same path through `vm_create`; a profile only supplies the
cloud-config. That is what keeps the seed-device and video quirks below in one
place instead of drifting between scripts.

`LIBVIRT_DEFAULT_URI` defaults to `qemu:///system` inside the scripts. Without
it a non-interactive `ssh host 'bin/vm-ip x'` reads the shell's environment,
not the profile that usually exports it, and answers from `qemu:///session` —
which reports an empty host with no VMs on it.

## Distros

`VM_DISTRO` picks the base image, the login account and the osinfo id together.
Defaults to `debian`, so existing usage is unchanged.

| `VM_DISTRO` | Image | User | Admin group | Seed | Video | Min RAM |
|---|---|---|---|---|---|---|
| `debian` (default) | Debian 12 genericcloud | `debian` | `sudo` | virtio disk | none | 1024 |
| `ubuntu` | Ubuntu 24.04 LTS cloudimg | `ubuntu` | `sudo` | virtio disk | none | 1024 |
| `rocky` | Rocky Linux 10 GenericCloud | `rocky` | `wheel` | cdrom | vga | 2048 |

```bash
bin/fetch-base rocky
VM_DISTRO=rocky bin/vm-up rocky1
ssh rocky@$(bin/vm-ip rocky1)
```

The registry lives in `lib/distros.sh`. Point a distro at another release with
`DEBIAN_CLOUD_URL` / `UBUNTU_CLOUD_URL` / `ROCKY_CLOUD_URL` — note that the
local image filename comes from the registry, so change both together when
switching releases.

The osinfo id is resolved against the host `osinfo-db` and falls back to the
closest known id, because an unknown id aborts `virt-install` and the db lags
new releases.

The last three columns are not cosmetic — each is a distro that refuses to boot
otherwise:

- **Seed device.** Debian and Ubuntu genericcloud kernels have no cdrom driver,
  so their seed goes on a virtio disk. Rocky documents a cdrom and its kernel
  has the driver.
- **Video.** Rocky's GRUB hangs before it emits a single character when the
  guest has no video device, which is exactly what `--graphics none` produces.
  Symptom: a VM burning 100% of a core, empty serial console, no DHCP lease.
  Debian and Ubuntu boot fine headless.
- **Admin group.** RHEL rebuilds use `wheel`; cloud-init fails to create the
  user at all if the group does not exist, leaving a VM you cannot log into.

## CPU model

`VM_CPU` defaults to `host-passthrough`. The previous `--cpu host` let libvirt
pick a "safe" custom model, which on this host resolved to `Denverton` —
x86-64-v2, no AVX2. Debian and Ubuntu do not care; Rocky 10 and RHEL 10 require
x86-64-v3 and will not run on it. Passthrough also matches what `virt-manager`
creates by default.

Defaults:

- distro: `debian` (seed ISO is attached as a virtio disk: the genericcloud
  cloud kernel has no cdrom driver, so a cdrom seed is invisible to cloud-init
  and the VM never gets an IP)
- image and user: from the distro registry, see [Distros](#distros)
- SSH key: `~/.ssh/id_ed25519.pub`
- network: libvirt `default`
- RAM: 2048 MiB
- vCPU: 2
- disk overlay size: 20G

Override with environment variables:

```bash
VM_MEMORY=4096 VM_VCPUS=4 VM_NET=default VM_SSH_KEY=~/.ssh/id_rsa.pub bin/vm-up node1 30G
VM_DISTRO=ubuntu bin/vm-up node2
```

`vm_create` aborts when host `MemAvailable` is below the VM's RAM plus 512 MiB;
`VM_FORCE=1` overrides.

## Profiles

`VM_PROFILE` picks the cloud-config. A profile is a file in `profiles/`
defining `profile_user_data()`, which prints cloud-config on stdout and may read
`$vm_hostname`, `$vm_user`, `$vm_admin_group`, `$vm_ssh_key_line` and `$vm_ip`.
Setting `PROFILE_PKG_FAMILY` makes it refuse distros it cannot provision, up
front rather than halfway through cloud-init.

| `VM_PROFILE` | What the VM boots with |
|---|---|
| `base` (default) | admin user + SSH key + guest agent |
| `k8s-node` | base plus containerd, k8s sysctls, kubelet/kubeadm/kubectl (apt only) |
| `kthw` | bare Debian for Kubernetes The Hard Way — see below (apt only) |

## Hostnames

`VM_HOSTNAME` sets the guest hostname independently of the libvirt domain name,
which defaults to it. Domain names are global to the host, so a tutorial that
wants to talk to a machine called `server` cannot own that name outright:

```bash
VM_HOSTNAME=server bin/vm-up kthw-server    # domain kthw-server, guest server
```

The hostname is validated as RFC 1123 (`[a-z0-9-]`, no underscores).

## k8s stand

`vm-up` stays generic; k8s-specific provisioning lives in separate scripts.
The node profile installs containerd and kubeadm from apt, so it accepts
`debian` and `ubuntu` and refuses other distros up front.

```bash
bin/k8s-cluster-up NAME [WORKERS]    # cp + N workers (default 2), static IPs, ansible inventory
bin/vm-up-k8s-node NAME IP [DISK]    # single k8s-ready node with a static IP
bin/k8s-cluster-rm NAME
```

- Static IPs: dnsmasq `dhcp-host` reservations in the libvirt network (MAC is
  derived from the last IP octet). `vm-rm` removes the reservation with the VM.
- Node profile (cloud-init): containerd with `SystemdCgroup`, overlay +
  br_netfilter, k8s sysctls, swap off, kubelet/kubeadm/kubectl from
  pkgs.k8s.io, version-pinned and held. `kubeadm init/join` is left to
  ansible or manual — inventory is written to `clusters/NAME/inventory.ini`.
- Defaults: cp1 = `.101`, workers = `.111+`. Env knobs: `K8S_VERSION` (1.35),
  `K8S_SUBNET_PREFIX` (192.168.200), `K8S_CP_OCTET`, `K8S_W_OCTET_BASE`.
- RAM: scripts abort when host `MemAvailable` is too low (`VM_FORCE=1`
  overrides). On a 16 GiB host cp + 2 workers @ 2048 MiB ≈ 6 GiB — stop other
  big VMs first.

## Kubernetes The Hard Way stand

```bash
bin/kthw-stand-up [NAME]    # jumpbox + server + 2 nodes, KTHW lab 01 spec
bin/kthw-stand-rm [NAME]
```

Kelsey Hightower's tutorial opens by asking for four Debian 12 machines and
then says nothing more about where they come from. This is that part, and only
that part.

| Machine | Guest hostname | IP | RAM | Disk |
|---|---|---|---|---|
| `NAME-jumpbox` | `jumpbox` | `.120` | 512 | 10G |
| `NAME-server` | `server` | `.121` | 2048 | 20G |
| `NAME-node-0` | `node-0` | `.122` | 2048 | 20G |
| `NAME-node-1` | `node-1` | `.123` | 2048 | 20G |

The table is the tutorial's spec, and it is what the stand defaults to. On a
tight host it goes lower: **768 MiB and a 10G disk carry `server` and the nodes
through all thirteen labs** — measured by walking the tutorial end to end on
exactly that. 512 MiB does not. That puts the whole stand at 2.75 GiB against
6.5 at the defaults:

```bash
KTHW_NODE_MEMORY=768 KTHW_NODE_DISK=10G bin/kthw-stand-up
```

`lib/distros.sh` sets `DISTRO_MIN_MEMORY=1024` for Debian, so each machine
prints a `Warning: ... wants at least 1024 MiB`. It is a warning, not a refusal
— expected here, and the stand comes up anyway.

`NAME` defaults to `kthw` and prefixes the libvirt domain names only — inside,
the machines are called what the tutorial calls them. The addresses avoid the
kubeadm stand's `.101`/`.111+`, and pod subnets `10.200.N.0/24` do not collide
with `192.168.200.0/24`. Knobs: `KTHW_NODES`, `KTHW_DOMAIN`,
`KTHW_SUBNET_PREFIX`, `KTHW_POD_SUBNET_PREFIX`, `KTHW_JUMPBOX_OCTET`,
`KTHW_SERVER_OCTET`, `KTHW_NODE_OCTET_BASE`, `KTHW_*_MEMORY`, `KTHW_*_DISK`,
`KTHW_VCPUS`.

The script prints `machines.txt` in lab 03's format and writes it to
`clusters/NAME/machines.txt`. It does not copy it to the jumpbox: that is a lab
03 step, and so are root SSH, the FQDNs and the shared `/etc/hosts`. The stand
hands over machines built to lab 01 and stops.

Two things the `kthw` profile does do, because neither is an exercise:

- **`manage_etc_hosts: false`.** cloud-init rewrites `/etc/hosts` from a
  template on every boot. Lab 03 has you put an entry per machine in that file;
  under the `base` profile they survive until the first reboot and then vanish,
  and the cluster comes apart by name with nothing obvious to blame. The
  jumpbox gets the same treatment — lab 03 writes to its `/etc/hosts` too.
- **`net.ipv4.ip_forward=1`.** Lab 11 routes pod subnets between nodes.
  `br_netfilter` and the bridge-nf sysctls are kubeadm's requirements, not this
  tutorial's, so the profile leaves them out.

`KTHW_SEED_ROOT=1` seeds root SSH for anyone skipping the first step of lab 03.
It writes `PermitRootLogin prohibit-password` as an `sshd_config.d` drop-in
rather than the tutorial's `yes` — login is by key either way.

chapter-zero

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README