* feat(ceph): import yucca-ceph ansible + terraform infrastructure
Imports the yucca-ceph Ansible tree into ansible/ceph/ and adds the
Terraform stack at tf/ that drives it. Cuts over from ansible-vault
to the hybrid secrets architecture (TF as inventory authority, 1P
as secrets store, op-inject at deploy time) in one atomic move.
Source: internal yucca-ceph working tree; fresh subtree-style
import, history not preserved. Andy continues operating sietch +
painbox post-merge; yucca-team hosts the code and reviews changes.
What it adds:
- sietch (3-node Austin, production Ceph S3 backend, untouched
by this PR)
- painbox (single-node Hetzner SX295 in Helsinki) as a second
deployable cluster
- Future clusters land by appending to clusters.auto.tfvars in
the matching environment stack (tf/deployment/<env>/ceph/) —
no per-cluster TF code required
How it works (full map: ansible/ceph/docs/architecture.md):
- tf/shared/modules/ceph-cluster renders inventory.ini variants
+ secrets.yml.tpl per cluster from clusters.auto.tfvars
- secrets.yml.tpl carries op:// refs; `op inject -f` resolves
them at deploy time from the matching yucca_tf_<env> vault
- State in OVH yucca-tf-state bucket (key ceph/<env>/<stack>/)
- 11 ADRs capture the decisions: ansible/ceph/docs/adr/
Out of scope (intentional):
- LUKS keys not yet in 1P (deferred until hybrid is stable)
- tf/shared/modules/ceph-cluster/secrets.tf.disabled is dormant;
today's 1P items via `op item create` per
ansible/ceph/docs/adding-a-cluster.md
- Talos K8s on sietch is a separate workstream
Atomicity + rollback: TF-rendered inventory + secrets-template
files are gitignored (TF generates them) and ansible-vault removal
is coupled to the op-inject path. Splitting this PR lands in a
non-bootable state — merge as one unit. The merge itself is
reversible via `git revert` until the post-merge `tf:apply` runs;
after apply, full rollback needs state restore or `tofu state mv`
(land + validate before applying).
Dev-env impact: adds opentofu + terragrunt to yucca root mise tools
plus a self-contained ansible/ceph/.mise.toml. No new commands or
prereqs for immich-side contributors who don't touch ceph or run
tf:* tasks.
Verification:
- `mise run lint` (from ansible/ceph/): 130 files, 0 warnings
- `mise run check`: 19 playbooks parse clean
- `mise run tf:plan`: succeeds; 7 expected file path-rename
replacements (3 painbox + 4 sietch). State drift from import,
no cluster-side change.
- painbox deployed 2026-04-26 on the new code path: Bookworm +
Ceph Tentacle, 15 OSDs (14 HDD + 1 SSD) up + in, mon/mgr/rgw
running. HEALTH_WARN is expected on a single-node cluster.
Post-merge: from the yucca root, `mise run tf:apply` flips the
bucket state to the new monorepo paths (the 7 renames above).
* fix(ceph): exempt ansible/ and tf/ subtrees from root prettier
The imported infrastructure subtrees enforce their own format
conventions (yamllint + ansible-lint inside ansible/ceph/; tofu fmt
inside tf/). Prettier on ansible YAML reflows long Jinja2 expressions
and shell command blocks in unwanted ways, so root prettier checks
are skipped for both subtrees.
Also reformat root README.md table column alignment to match prettier
conventions (only the imported subtrees are exempt; yucca-side files
including the root README still follow root prettier rules).
* fix(ceph): clean up secrets tmpfile after ansible-playbook exits
`ansible-play.sh` rendered the resolved secrets file via `op inject`
into a `mktemp` tmpfile, set up a `trap 'rm -f "$TMPFILE"' EXIT INT
TERM`, then `exec`'d ansible-playbook. The `exec` replaced the bash
shell entirely, so the EXIT trap never fired — every play left a
plaintext-secrets file in /tmp.
In practice this was masked because /tmp is tmpfs (RAM only on this
operator's setup), so files evaporate on reboot. But within an
operator session, files accumulated linearly with each playbook
invocation. Recent count on the import-PR session: 38 files.
Drop the `exec`. With `set -euo pipefail` already on, bash:
- propagates ansible-playbook's exit code (set -e)
- fires the EXIT trap before exiting (always)
- cleans up the tmpfile on success, failure, or signal
Verified: `CEPH_ENV=... scripts/ansible-play.sh status.yml
--syntax-check` creates and removes the tmpfile within the same
invocation — /tmp is clean before and after.
`scripts/preflight.sh` uses the same trap pattern but does not
`exec`, so its tmpfile cleanup was already correct (and the suffix
differs: `-secrets-test.yml` vs `-secrets.yml`, confirming
ansible-play.sh as the sole offender).
* feat: initial commit (mise, nest, restic-api)
* ci: should use new nested tasks
* chore: use absolute imports
* chore: format
* fix: absolute imports for jest
* feat: REST API impl. with in-memory storage
* feat: s3 storage backing (wip)
* fix: use Range in S3 get object
* feat: validate path parameters
* chore: clean up checks
* chore: remove TODO
* docs: add descriptions to mise commands
* feat: JWT auth
* feat: load secrets from env (validate w. zod)
* chore: set default compose variables
* chore: add docker commands to mise
* fix: don't eat JWT errors
* fix: obfuscate s3 errors
* chore: validate auth schema
* feat: commit errors.ts
* fix: return partial status if Range provided
* chore: split jwt/payload validation errors
* fix: check name is a sha256 hash
* feat: streaming download
* feat: streaming upload
* feat: WORM / write once support
* fix: don't obscure other S3 errors for checkBucket
* refactor: more concise error types
* test: service unit tests for app/auth; e2e for storage
* chore: prettier ignore .dev folder
* feat: interceptor for Restic JSON routes
* test: app e2e tests (wip)
* test: partial content response
* chore: lint
* refactor: use class-validator for auth dto
* refactor: clean up constants through project
* refactor: also update binary content type
* fix: incorrect Basic parsing
* fix: actually handle validation errors in auth dto
* chore: delete .gitignore
* fix: should use 403 not 405 in worm
* chore: expose minio console for debug
* feat: delete config route
* fix: error consistency with restic reference
* feat: validate sha256 on blob upload
* fix: use bad request exception for mismatch
* feat: initial work on full e2e test
* fix: respond with correct headers for Range
refactor: unify respond logic
* test: backup & restore e2e test
* feat: setup empty yucca-api project
* feat: setup kysely for yucca-api & db repository
* refactor: `utils.ts` -> `utils/s3.ts`
* fix: shut down database after e2e test
* chore: remove stray log
* test: refactor getObject test
* chore: add built wrapper for now
* chore: add e2e/ to mise task
* chore: format
* fix: esm jest
* feat: setup database migrator
* feat: minimal template for frontend project
* chore: add prettier-plugin-tailwindcss to workspace
* chore: configure @immich/ui
* feat: sdk for web, hook up sample
* chore: add @oazapfts/runtime dependency
* chore: ignore `yucca-sdk/src/fetch-client.ts` from eslint
* chore: format
* chore: move all CI check stages into mise config
* test: mock $env/dynamic/publish for ui library
* chore: format code
* fix: correct restic API port in e2e test
* chore: move tasks around
* chore: use restic-wrapper from npm
* chore: string
* test: comprehensive e2e tests for restic API
* test: cover new commands in e2e tests
* test: worm-case for prune
* feat: lingui for svelte
* chore: ensure check runs lingui
* chore: ignore locales from checks
* chore: format/lint
* chore: configure nix-ld for mise
* chore: move `env.ts` to common package
* chore: accept any no. of arguments for docker tasks/e2e
* chore: different strategy for e2e clean up
* chore: configure Git Town
Signed-off-by: izzy <me@insrt.uk>
* chore: add default.nix for mise
Signed-off-by: izzy <me@insrt.uk>
* chore: lost changes from parent
Signed-off-by: izzy <me@insrt.uk>
* fix: use valid language code
* chore: ignore migration errors in dev
* fix: apply fixes to @common/server (again)
* fix: wait for common:build
---------
Signed-off-by: izzy <me@insrt.uk>
* feat: initial commit (mise, nest, restic-api)
* ci: should use new nested tasks
* chore: use absolute imports
* chore: format
* fix: absolute imports for jest
* feat: REST API impl. with in-memory storage
* feat: s3 storage backing (wip)
* fix: use Range in S3 get object
* feat: validate path parameters
* chore: clean up checks
* chore: remove TODO
* docs: add descriptions to mise commands
* feat: JWT auth
* feat: load secrets from env (validate w. zod)
* chore: set default compose variables
* chore: add docker commands to mise
* fix: don't eat JWT errors
* fix: obfuscate s3 errors
* chore: validate auth schema
* feat: commit errors.ts
* fix: return partial status if Range provided
* chore: split jwt/payload validation errors
* fix: check name is a sha256 hash
* feat: streaming download
* feat: streaming upload
* feat: WORM / write once support
* fix: don't obscure other S3 errors for checkBucket
* refactor: more concise error types
* test: service unit tests for app/auth; e2e for storage
* chore: prettier ignore .dev folder
* feat: interceptor for Restic JSON routes
* test: app e2e tests (wip)
* test: partial content response
* chore: lint
* refactor: use class-validator for auth dto
* refactor: clean up constants through project
* refactor: also update binary content type
* fix: incorrect Basic parsing
* fix: actually handle validation errors in auth dto
* chore: delete .gitignore
* fix: should use 403 not 405 in worm
* chore: expose minio console for debug
* feat: delete config route
* fix: error consistency with restic reference
* feat: validate sha256 on blob upload
* fix: use bad request exception for mismatch
* feat: initial work on full e2e test
* fix: respond with correct headers for Range
refactor: unify respond logic
* test: backup & restore e2e test
* feat: setup empty yucca-api project
* feat: setup kysely for yucca-api & db repository
* refactor: `utils.ts` -> `utils/s3.ts`
* fix: shut down database after e2e test
* chore: remove stray log
* test: refactor getObject test
* chore: add built wrapper for now
* chore: add e2e/ to mise task
* chore: format
* fix: esm jest
* feat: setup database migrator
* feat: minimal template for frontend project
* chore: add prettier-plugin-tailwindcss to workspace
* chore: configure @immich/ui
* feat: sdk for web, hook up sample
* chore: add @oazapfts/runtime dependency
* chore: ignore `yucca-sdk/src/fetch-client.ts` from eslint
* chore: format
* chore: move all CI check stages into mise config
* test: mock $env/dynamic/publish for ui library
* chore: format code
* fix: correct restic API port in e2e test
* chore: move tasks around
* chore: use restic-wrapper from npm
* chore: string
* test: comprehensive e2e tests for restic API
* test: cover new commands in e2e tests
* test: worm-case for prune
* feat: lingui for svelte
* chore: ensure check runs lingui
* chore: ignore locales from checks
* chore: format/lint
* chore: configure nix-ld for mise
* chore: accept any no. of arguments for docker tasks/e2e
* chore: different strategy for e2e clean up
* chore: configure Git Town
Signed-off-by: izzy <me@insrt.uk>
* chore: add default.nix for mise
Signed-off-by: izzy <me@insrt.uk>
* chore: lost changes from parent
Signed-off-by: izzy <me@insrt.uk>
* fix: use valid language code
---------
Signed-off-by: izzy <me@insrt.uk>
Antoine Lecompte
Paul Makles
Andy Molenda