ADR 0019 Portable Kubernetes Storage Capacity

Context

The runtime repeatedly exhausts worker root disks while a large physical device is hidden behind a Proxmox file-backed iSCSI LUN. Treating that LUN as shared Longhorn storage would couple Kubernetes recovery to one hypervisor and duplicate storage control planes.

Decision

Add capacity as dedicated Kubernetes storage workers. Attach one local block device exclusively to each worker, format and mount it with ext4 or XFS, then enroll it through Longhorn’s native node.longhorn.io/create-default-disk=config label and node.longhorn.io/default-disks-config annotation. Assembly enables that upstream path but never commits node names, hypervisor identifiers, device names, or host paths.

Applications consume only Assembly-owned StorageClasses. Longhorn owns block-volume placement and replica repair. Replicas must land on independent physical disks on different Kubernetes nodes; one disk split into multiple replicas is not availability. Stacks with native replication, including CloudNativePG and SeaweedFS, use a one-replica or strict-local volume profile when double replication adds no required protection. Backups leave the cluster through an S3-compatible target such as R2 and require restore proof.

The retired iSCSI plane is a migration source, not a Kubernetes backend. Move and verify every remaining guest before removing its LUN or reformatting the physical device. Longhorn native snapshot cleanup and filesystem trim run with bounded concurrency. Deletion of valid snapshots, retained volumes, or orphans remains an explicit owner-approved operation.

Consequences

Another Kubernetes cluster reproduces the design by providing eligible workers, mounted local disks, the two standard enrollment metadata fields, and equivalent StorageClasses. The environment adapter changes; application manifests do not. Capacity grows by adding disks or workers rather than enlarging a central LUN.

Implementation traps and verification commands are recorded in the Longhorn 1.11.1 integration guide.

Revision History

2026-07-30: Restored the completed CNPG object backup into an isolated 96 GiB/20 GiB cluster, reached Cluster in healthy state, and verified Dex offline_session=1 plus refresh_token=1. Removed the superseded recurring local snapshot schedule, eight database VolumeSnapshots, and 92.9 GB of detached retained Longhorn volumes after attachment checks.
2026-07-22: Enrolled the first dedicated 2 TB worker and began native replica evacuation from a saturated legacy disk.
2026-07-22: Accepted dedicated node-local capacity with native Longhorn enrollment and iSCSI retirement.