Commit Graph
16 Commits
Author SHA1 Message Date
Zack Pollard 5a144405cf feat(docs): end-user documentation site at docs.futo.cloud (#623)
packages/docs is a SvelteKit site where every page is a +page.md compiled
by @immich/svelte-markdown-preprocess into @immich/ui components, deployed
to Cloudflare Pages the way static-pages deploys the immich.app sites
(tf/pages/docs; main -> docs.futo.cloud, every PR gets a preview).

The content is written for beta users: joining the beta, setting up in
Immich or the standalone container, the recovery key, backups, schedules,
restores, the account dashboard, troubleshooting and support. Internal and
developer documentation stays in docs/ and the per-directory READMEs.
2026-09-14 07:41:44 -07:00
Antoine Lecompte b7bbe2e80e chore(yuctl): restructure and make pretty (#485) 2026-08-18 09:53:59 -04:00
Antoine Lecompte ecf969ba3e feat(admin): make the whole thing benchmarkable (#330) 2026-07-24 11:15:09 -04:00
Antoine Lecompte 963364a034 feat(prod): prod (#247) 2026-07-13 13:15:50 +00:00
Antoine Lecompte 75b7808993 chore(netbird): move to kebab naming (#242)
* chore(netbird): move to kebab naming

Render NetBird object names (groups, setup keys, policies, networks,
network-resources) as lowercase-kebab instead of UPPER_SNAKE, e.g.
YUCCA_PROD_HTZ_FSN1_MGMT → yucca-prod-htz-fsn1-mgmt. The 1Password setup-key
item titles stay UPPER_SNAKE (decoupled) so CI/ansible/talos op:// consumers
keep resolving.

Pin the futo-org/netbird provider to 1.0.2, which fixes the group
resources TF→API decode so a resource-tag group (htz-fsn1 `resources`) can be
renamed in place — no name pin needed.

* update locks
2026-06-30 16:20:10 +00:00
Antoine Lecompte 0c1b7f765c chore(fabric): switch over the fabric from the generated provider to … (#231)
* chore(fabric): switch over the fabric from the generated provider to a community provider

* cleanup
2026-06-29 16:03:50 -04:00
Antoine Lecompte aeee19e336 feat(bgp): stand up bgp (#224)
* feat(bgp): stand up bgp

* eergh

* firewall

* fix
2026-06-29 14:50:43 +00:00
Antoine Lecompte c6985d902c feat(all): introduce partition/region/ceph-cluster model across the stack (#222)
* feat: introduce partition/region/ceph-cluster model across the stack

Formalize partition -> region -> {one k8s cluster, many ceph clusters} and
thread it through every layer plus a new yuctl ops CLI.

- tf: deployment/<partition>/<region>/<stack> layout; terragrunt path-parse +
  state key yucca/<partition>/<region>/<stack>; per-region region.hcl (role,
  site_id, datacenter, provider_code, domain); env->partition / site->region
  renames (NetBird object names byte-identical); standardized per-stack
  `discovery` output contract (secrets as op:// refs).
- k8s: clusters/<partition>/<region>/ (staging/austin, prod/htz-fsn1, dev/local);
  role-based kustomize components (primary/secondary); hybrid cluster-settings
  (TF-rendered identity + human fragment); dev-mirror folded into dev/local;
  charts regrouped into charts/{apps,platform,lib,dev}.
- ci: infra.yml partition/region discovery matrix; partition-keyed path filters;
  <partition>-<region> environment gates; image-versions path moves.
- ansible: inventories under <partition>-<region>/<cluster>.
- yuctl: Go/cobra CLI reading the discovery contract from TF state.
- Retire the sietch-talos libvirt VM cluster (dev@local is the k3d cluster);
  ceph inventory_dirname -> <partition>-<region>/<cluster>.

Verified: mise k8s:validate green (3 clusters); yuctl go build/vet; tofu
validate pre-merge (all 9 stacks). Live-staging state migration NOT run.

* fix typo

* commit
2026-06-29 08:40:29 -04:00
Antoine Lecompte 481c5e920a feat(yucca): add full e2e mgmt provisioning maybe (#182)
* feat(yucca): add full e2e mgmt provisioning maybe

* moar !

* prefer tailscale over public ip if availbale

* ignore files

* fix

* more progress
2026-06-26 14:45:20 -04:00
Antoine Lecompte 80d15f23af feat: fabric terraform (#170) 2026-06-25 11:23:59 -04:00
Antoine Lecompte f700b18cd6 feat: staging (#135)
* feat: staging

* more stuff

* pin actions

* adjust

* adjust
2026-06-23 18:21:12 +00:00
Antoine Lecompte 070e22a7bb feat: local k8s (#85)
* impl. local kube

* add support for op injected oidc secrets

* ci: set least-privilege workflow token permissions
2026-06-12 13:17:37 +00:00
Andy Molenda 63087f6850 feat(ceph): import yucca-ceph ansible + terraform infrastructure (#86)
* feat(ceph): import yucca-ceph ansible + terraform infrastructure

Imports the yucca-ceph Ansible tree into ansible/ceph/ and adds the
Terraform stack at tf/ that drives it. Cuts over from ansible-vault
to the hybrid secrets architecture (TF as inventory authority, 1P
as secrets store, op-inject at deploy time) in one atomic move.
Source: internal yucca-ceph working tree; fresh subtree-style
import, history not preserved. Andy continues operating sietch +
painbox post-merge; yucca-team hosts the code and reviews changes.

What it adds:
  - sietch (3-node Austin, production Ceph S3 backend, untouched
    by this PR)
  - painbox (single-node Hetzner SX295 in Helsinki) as a second
    deployable cluster
  - Future clusters land by appending to clusters.auto.tfvars in
    the matching environment stack (tf/deployment/<env>/ceph/) —
    no per-cluster TF code required

How it works (full map: ansible/ceph/docs/architecture.md):
  - tf/shared/modules/ceph-cluster renders inventory.ini variants
    + secrets.yml.tpl per cluster from clusters.auto.tfvars
  - secrets.yml.tpl carries op:// refs; `op inject -f` resolves
    them at deploy time from the matching yucca_tf_<env> vault
  - State in OVH yucca-tf-state bucket (key ceph/<env>/<stack>/)
  - 11 ADRs capture the decisions: ansible/ceph/docs/adr/

Out of scope (intentional):
  - LUKS keys not yet in 1P (deferred until hybrid is stable)
  - tf/shared/modules/ceph-cluster/secrets.tf.disabled is dormant;
    today's 1P items via `op item create` per
    ansible/ceph/docs/adding-a-cluster.md
  - Talos K8s on sietch is a separate workstream

Atomicity + rollback: TF-rendered inventory + secrets-template
files are gitignored (TF generates them) and ansible-vault removal
is coupled to the op-inject path. Splitting this PR lands in a
non-bootable state — merge as one unit. The merge itself is
reversible via `git revert` until the post-merge `tf:apply` runs;
after apply, full rollback needs state restore or `tofu state mv`
(land + validate before applying).

Dev-env impact: adds opentofu + terragrunt to yucca root mise tools
plus a self-contained ansible/ceph/.mise.toml. No new commands or
prereqs for immich-side contributors who don't touch ceph or run
tf:* tasks.

Verification:
  - `mise run lint` (from ansible/ceph/): 130 files, 0 warnings
  - `mise run check`: 19 playbooks parse clean
  - `mise run tf:plan`: succeeds; 7 expected file path-rename
    replacements (3 painbox + 4 sietch). State drift from import,
    no cluster-side change.
  - painbox deployed 2026-04-26 on the new code path: Bookworm +
    Ceph Tentacle, 15 OSDs (14 HDD + 1 SSD) up + in, mon/mgr/rgw
    running. HEALTH_WARN is expected on a single-node cluster.

Post-merge: from the yucca root, `mise run tf:apply` flips the
bucket state to the new monorepo paths (the 7 renames above).

* fix(ceph): exempt ansible/ and tf/ subtrees from root prettier

The imported infrastructure subtrees enforce their own format
conventions (yamllint + ansible-lint inside ansible/ceph/; tofu fmt
inside tf/). Prettier on ansible YAML reflows long Jinja2 expressions
and shell command blocks in unwanted ways, so root prettier checks
are skipped for both subtrees.

Also reformat root README.md table column alignment to match prettier
conventions (only the imported subtrees are exempt; yucca-side files
including the root README still follow root prettier rules).

* fix(ceph): clean up secrets tmpfile after ansible-playbook exits

`ansible-play.sh` rendered the resolved secrets file via `op inject`
into a `mktemp` tmpfile, set up a `trap 'rm -f "$TMPFILE"' EXIT INT
TERM`, then `exec`'d ansible-playbook. The `exec` replaced the bash
shell entirely, so the EXIT trap never fired — every play left a
plaintext-secrets file in /tmp.

In practice this was masked because /tmp is tmpfs (RAM only on this
operator's setup), so files evaporate on reboot. But within an
operator session, files accumulated linearly with each playbook
invocation. Recent count on the import-PR session: 38 files.

Drop the `exec`. With `set -euo pipefail` already on, bash:

  - propagates ansible-playbook's exit code (set -e)
  - fires the EXIT trap before exiting (always)
  - cleans up the tmpfile on success, failure, or signal

Verified: `CEPH_ENV=... scripts/ansible-play.sh status.yml
--syntax-check` creates and removes the tmpfile within the same
invocation — /tmp is clean before and after.

`scripts/preflight.sh` uses the same trap pattern but does not
`exec`, so its tmpfile cleanup was already correct (and the suffix
differs: `-secrets-test.yml` vs `-secrets.yml`, confirming
ansible-play.sh as the sole offender).
2026-05-18 06:17:56 -07:00
Antoine Lecompte 4665f5cdd1 feat: replace restic-api with michael (#57) 2026-04-09 16:47:37 +01:00
Paul Makles c568a50687 feat: configure OpenTelemetry (#29) 2026-02-18 14:49:31 +00:00
Paul Makles 0b99068f23 feat: initial commit (#27)
* feat: initial commit (mise, nest, restic-api)

* ci: should use new nested tasks

* chore: use absolute imports

* chore: format

* fix: absolute imports for jest

* feat: REST API impl. with in-memory storage

* feat: s3 storage backing (wip)

* fix: use Range in S3 get object

* feat: validate path parameters

* chore: clean up checks

* chore: remove TODO

* docs: add descriptions to mise commands

* feat: JWT auth

* feat: load secrets from env (validate w. zod)

* chore: set default compose variables

* chore: add docker commands to mise

* fix: don't eat JWT errors

* fix: obfuscate s3 errors

* chore: validate auth schema

* feat: commit errors.ts

* fix: return partial status if Range provided

* chore: split jwt/payload validation errors

* fix: check name is a sha256 hash

* feat: streaming download

* feat: streaming upload

* feat: WORM / write once support

* fix: don't obscure other S3 errors for checkBucket

* refactor: more concise error types

* test: service unit tests for app/auth; e2e for storage

* chore: prettier ignore .dev folder

* feat: interceptor for Restic JSON routes

* test: app e2e tests (wip)

* test: partial content response

* chore: lint

* refactor: use class-validator for auth dto

* refactor: clean up constants through project

* refactor: also update binary content type

* fix: incorrect Basic parsing

* fix: actually handle validation errors in auth dto

* chore: delete .gitignore

* fix: should use 403 not 405 in worm

* chore: expose minio console for debug

* feat: delete config route

* fix: error consistency with restic reference

* feat: validate sha256 on blob upload

* fix: use bad request exception for mismatch

* feat: initial work on full e2e test

* fix: respond with correct headers for Range
refactor: unify respond logic

* test: backup & restore e2e test

* feat: setup empty yucca-api project

* feat: setup kysely for yucca-api & db repository

* refactor: `utils.ts` -> `utils/s3.ts`

* fix: shut down database after e2e test

* chore: remove stray log

* test: refactor getObject test

* chore: add built wrapper for now

* chore: add e2e/ to mise task

* chore: format

* fix: esm jest

* feat: setup database migrator

* feat: minimal template for frontend project

* chore: add prettier-plugin-tailwindcss to workspace

* chore: configure @immich/ui

* feat: sdk for web, hook up sample

* chore: add @oazapfts/runtime dependency

* chore: ignore `yucca-sdk/src/fetch-client.ts` from eslint

* chore: format

* chore: move all CI check stages into mise config

* test: mock $env/dynamic/publish for ui library

* chore: format code

* fix: correct restic API port in e2e test

* chore: move tasks around

* chore: use restic-wrapper from npm

* chore: string

* test: comprehensive e2e tests for restic API

* test: cover new commands in e2e tests

* test: worm-case for prune

* feat: lingui for svelte

* chore: ensure check runs lingui

* chore: ignore locales from checks

* chore: format/lint

* chore: configure nix-ld for mise

* chore: accept any no. of arguments for docker tasks/e2e

* chore: different strategy for e2e clean up

* chore: configure Git Town

Signed-off-by: izzy <me@insrt.uk>

* chore: add default.nix for mise

Signed-off-by: izzy <me@insrt.uk>

* chore: lost changes from parent

Signed-off-by: izzy <me@insrt.uk>

* fix: use valid language code

---------

Signed-off-by: izzy <me@insrt.uk>
2026-01-27 13:46:14 +00:00