Files

kubernetes/apps/miroir-system/miroir/config/storageclass.yaml

1 ---

2 # yaml-language-server: $schema=https://k8s-schemas.home-operations.com/storage.k8s.io/storageclass_v1.json

3 apiVersion: storage.k8s.io/v1

4 kind: StorageClass

5 metadata:

6 name: miroir-fast-local

7 annotations:

8 storageclass.kubernetes.io/is-default-class: "true"

9 provisioner: miroir.home-operations.com

10 volumeBindingMode: WaitForFirstConsumer

11 allowVolumeExpansion: true

12 parameters:

13 miroir.home-operations.com/pool: fast

14 miroir.home-operations.com/replicas: "1"

15 csi.storage.k8s.io/fstype: ext4

16 ---

17 # yaml-language-server: $schema=https://k8s-schemas.home-operations.com/storage.k8s.io/storageclass_v1.json

18 apiVersion: storage.k8s.io/v1

19 kind: StorageClass

20 metadata:

21 name: miroir-fast

22 provisioner: miroir.home-operations.com

23 volumeBindingMode: WaitForFirstConsumer

24 allowVolumeExpansion: true

25 parameters:

26 miroir.home-operations.com/pool: fast

27 miroir.home-operations.com/replicas: "2"

28 miroir.home-operations.com/quorum: freeze

29 csi.storage.k8s.io/fstype: ext4

30 ---

31 # yaml-language-server: $schema=https://k8s-schemas.home-operations.com/storage.k8s.io/storageclass_v1.json

32 apiVersion: storage.k8s.io/v1

33 kind: StorageClass

34 metadata:

35 name: miroir-slow-local

36 provisioner: miroir.home-operations.com

37 volumeBindingMode: WaitForFirstConsumer

38 allowVolumeExpansion: true

39 parameters:

40 miroir.home-operations.com/pool: slow

41 miroir.home-operations.com/replicas: "1"

42 csi.storage.k8s.io/fstype: ext4

43 ---

44 # yaml-language-server: $schema=https://k8s-schemas.home-operations.com/storage.k8s.io/storageclass_v1.json

45 apiVersion: storage.k8s.io/v1

46 kind: StorageClass

47 metadata:

48 name: miroir-slow

49 provisioner: miroir.home-operations.com

50 volumeBindingMode: WaitForFirstConsumer

51 allowVolumeExpansion: true

52 parameters:

53 miroir.home-operations.com/pool: slow

54 miroir.home-operations.com/replicas: "2"

55 miroir.home-operations.com/quorum: freeze

56 csi.storage.k8s.io/fstype: ext4

Two replicas, and do not raise this to three — three was tried on…

why ·ok L26

Two replicas, and do not raise this to three — three was tried on 2026-08-12 and deadlocked snapshot backups within the hour.

Every kopiur snapshot stages a throwaway clone PVC on this class. At three diskful replicas that clone must full-sync the entire volume to two peers at once, which builds a sync dependency chain instead of a single pair. DRBD then pauses one leg of it:

k8s-0: repl( SyncSource -> PausedSyncS ) resync-susp( no -> connection dependency,disk inconsistent ) k8s-0: Resync suspended

Show the rest

k8s-0 was syncing from k8s-1; k8s-1 went Inconsistent to sync from k8s-2, which suspended the leg k8s-0 depended on. The clone `sabnzbd-20260812125200-src` sat Inconsistent on two of three nodes for over an hour with no progress, its mover stuck in Init, and sabnzbd had no working backup for that whole window. miroir was right to refuse to stage an Inconsistent replica — the refusal is what surfaced it.

The reason three was reached for is documented below and was mistaken.

With `replicas: "2"` one of the three nodes holds no local copy, and `autoDiskfulAfter: 10m` in the HelmRelease then converts it and runs a full resync. That conversion had fired on essentially the whole cluster — 33 of 37 fast-pool volumes were already at 3 replicas, each still carrying `fullSync: true` — so the declared replica count was a fiction and resyncs were near-permanent background churn. Provisioning all three diskful up front makes the spec match reality and removes the churn.

It costs nothing: the volumes were already at 3 replicas via that same auto-conversion, and the fast thin pool was 14.5% allocated (36.7 GB of 253.9 GB) when measured.

IMPORTANT — this setting does NOT prevent the 2026-08-11 data loss, and an earlier version of this annotation wrongly claimed it did. The root cause was established later that day from VictoriaLogs and Prometheus: a filesystem freeze taken by the snapshot path leaked, and a routine KEDA scale-to-zero relocated the pod onto another node while the device was still frozen. `NodeStageVolume` there ran `blkid`, which cannot probe a frozen device and exits 2 — a code Kubernetes' `GetDiskFormat` treats as "unformatted" — so `mount_linux.FormatAndMount` ran `mkfs.ext4 -F` on a populated volume and DRBD replicated the wipe to every peer. Replica count is irrelevant to that sequence.

The real fix is upstream in miroir: never format a device that failed to probe, and consult `MiroirVolume.status.formatted` before formatting. See the evidence pack referenced from the storage memory notes.

2026-08-1201KZTQZE9EAJHQE485GBSZ8B89

Default moved to the local class during the DRBD exodus

why ·ok L6

Moved from miroir-fast on 2026-09-03 as part of the user-mandated migration of every volume off DRBD-replicated classes ("until drbd fix let's move all") after the 2026-09-01 wedge. Any PVC that names no class — operator claim templates, ad-hoc claims, future apps — now lands on single-replica local LVM instead of a 2-replica DRBD device. Move it back to miroir-fast only together with the decision to trust DRBD again (upstream miroir 0.12.0 retest or a LINBIT/drbd#137 fix), and remember local volumes pin their pod to one node — the memory-headroom constraints that shaped the migration apply to anything new that lands here.

2026-09-0301M1MCT5JAD69S0H365GJJH16T