immich-app/yucca
1
0
Fork 0
mirror of https://github.com/immich-app/yucca.git synced 2026-09-30 13:33:00 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(netbird): some issues (#212)

Browse Source
This commit is contained in:
Antoine Lecompte
2026-06-26 18:16:41 +00:00
committed by GitHub
parent 38ce07687e
commit 948b15cf0b
7 changed files with 61 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/infra.yml
+8 -13
View File
@@ -73,24 +73,19 @@ jobs:
- name: Install 1Password CLI
uses: 1password/install-cli-action@a5215d3a7f75c1629216c465ea9ab3ab399c4b71 # v4.0.0
# The Talos stack refreshes state against the nodes; the DNS, ceph and
# netbird stacks are pure Cloudflare/1Password/NetBird-API and need no
# overlay. CI reaches the 10.10.10.0/24 nodes over NetBird: the runner joins
# as a `ci` peer via the minted setup key, and the existing staging route
# advertises the LAN. The key must already exist in 1P (minted by a prior
# netbird apply) — true for every PR after the first bootstrap apply.
- name: Connect to NetBird
if: matrix.stack == 'talos'
uses: ./.github/actions/netbird-connect
with:
setup-key-ref: op://yucca_tf_staging/NETBIRD_YUCCA_STAGING_CI_SETUP_KEY/password
hostname: gha-staging-plan-${{ github.run_id }}
# Plan does NOT join the NetBird overlay. The talos plan would otherwise
# refresh against the 10.10.10.0/24 nodes (reachable only over NetBird),
# whose CI setup key is itself minted by the apply below — and the apply is
# gated on this plan. To avoid that bootstrap deadlock, CI plans talos with
# `-refresh=false` (config-vs-state only, no node contact); the apply job
# does the full refresh + apply over the overlay. The cloud-API stacks
# (dns/ceph/netbird) refresh normally — they need no overlay.
- name: Terragrunt plan
run: >-
tf/op-run.sh terragrunt
--working-dir tf/deployment/staging/${{ matrix.stack }}
--non-interactive plan
${{ matrix.stack == 'talos' && '-refresh=false' || '' }}
apply:
name: Apply (gated)
tf/README.md
+17 -14
View File
@@ -286,14 +286,16 @@ Per env (and per prod site), the baseline groups are:
| group | who | rendered (staging / prod htz-fsn1) |
|---|---|---|
| `ci` | ephemeral CI runners | `yucca_staging_ci` / `yucca_prod_htz_fsn1_ci` |
| `mgmt` | management nodes (configured via Ansible; also the route peers) | `yucca_staging_mgmt` / … |
| `talos` | Talos cluster nodes | `yucca_staging_talos` / … |
| `k8s_operator` | in-cluster kubernetes operator | `yucca_staging_k8s_operator` / … |
| `ci` | ephemeral CI runners | `YUCCA_STAGING_CI` / `YUCCA_PROD_HTZ_FSN1_CI` |
| `mgmt` | management nodes (configured via Ansible; also the route peers) | `YUCCA_STAGING_MGMT` / … |
| `talos` | Talos cluster nodes | `YUCCA_STAGING_TALOS` / … |
| `k8s_operator` | in-cluster kubernetes operator | `YUCCA_STAGING_K8S_OPERATOR` / … |
CI is **per-env** (`yucca_<env>_ci`, reaching only that env's groups) — no
cross-env CI plane. `k8s` is split into `talos` (the nodes) and `k8s_operator`
(the operator identity) so they can carry different policies.
Logical keys (the tfvars map keys, e.g. `ci`) stay lowercase; **rendered NetBird
names are UPPER_SNAKE** (uppercased, hyphens → underscores). CI is **per-env**
(`ci`, reaching only that env's groups) — no cross-env CI plane. `k8s` is split
into `talos` (the nodes) and `k8s_operator` (the operator identity) so they can
carry different policies.
### Stacks & layering
@@ -305,8 +307,8 @@ cross-env CI plane. `k8s` is split into `talos` (the nodes) and `k8s_operator`
Staging is single-layer. **Prod is layered**: a `global` layer (account-wide
groups + policies) above per-site layers. The global layer owns the
**`yucca_resource`** tag group and the account-wide **`yucca → yucca_resource`**
policy (see below). Site groups are site-scoped (`yucca_prod_<site>_<role>`) so a
**`YUCCA_RESOURCE`** tag group and the account-wide **`yucca → yucca_resource`**
policy (see below). Site groups are site-scoped (`YUCCA_PROD_<SITE>_<ROLE>`) so a
network router's peers are unambiguously *that site's* mgmt nodes. A site layer
consumes a global group via a terragrunt `dependency` on `prod/global` → the
module's `external_groups` input (htz-fsn1 does this for `yucca_resource`). The
@@ -315,12 +317,13 @@ gets its own state key without colliding with the `prod/htz-fsn1` fabric stack.
### The `yucca` / `yucca_resource` access model
Two pre-existing-or-managed groups drive account-wide access to routed subnets:
Two groups drive account-wide access to routed subnets (rendered names in caps):
- **`yucca`** — the existing **users** group (people). External (looked up by
name); never managed here.
- **`yucca_resource`** — the shared **resource tag**, managed in `prod/global`.
Every routed `netbird_network_resource` (across sites) is tagged into it.
- **`yucca`** — the existing **users** group (people). External (looked up by its
actual name `yucca`); never managed here.
- **`YUCCA_RESOURCE`** — the shared **resource tag**, managed in `prod/global`
(logical key `yucca_resource`). Every routed `netbird_network_resource` (across
sites) is tagged into it.
One global policy in `prod/global` — `yucca → yucca_resource`, `bidirectional =
false` — lets users reach every tagged resource. Because `yucca_resource` is only
tf/deployment/prod/global/netbird.auto.tfvars
+3 -2
View File
@@ -9,9 +9,10 @@
# The shared resource tag. Site layers tag every routed network resource into
# this group (via a terragrunt dependency on this stack), so one account-wide
# policy governs access to all of them. Explicit name → not prefixed "yucca_prod_".
# policy governs access to all of them. Explicit name → the unprefixed shared tag
# (without it the module would render "YUCCA_PROD_YUCCA_RESOURCE").
groups = {
yucca_resource = { name = "yucca_resource" }
yucca_resource = { name = "YUCCA_RESOURCE" }
}
setup_keys = {}
tf/deployment/prod/htz-fsn1/netbird/netbird.auto.tfvars
+6 -6
View File
@@ -1,12 +1,12 @@
# htz-fsn1 site NetBird objects (auto-prefixed "yucca_prod_htz_fsn1_").
# Setup-key plaintext → the yucca_tf_prod vault. NetBird is default-deny: a peer
# gets only the access its groups' policies grant.
# htz-fsn1 site NetBird objects (rendered UPPER_SNAKE, auto-prefixed
# "YUCCA_PROD_HTZ_FSN1_"). Setup-key plaintext → the yucca_tf_prod vault. NetBird
# is default-deny: a peer gets only the access its groups' policies grant.
groups = {
ci = {} # ephemeral CI runners → yucca_prod_htz_fsn1_ci
ci = {} # ephemeral CI runners → YUCCA_PROD_HTZ_FSN1_CI
mgmt = {} # management nodes (configured via ansible); also the route peers
talos = {} # Talos cluster nodes → yucca_prod_htz_fsn1_talos
k8s_operator = {} # in-cluster kubernetes operator → yucca_prod_htz_fsn1_k8s_operator
talos = {} # Talos cluster nodes → YUCCA_PROD_HTZ_FSN1_TALOS
k8s_operator = {} # in-cluster kubernetes operator → YUCCA_PROD_HTZ_FSN1_K8S_OPERATOR
}
setup_keys = {
tf/deployment/staging/netbird/netbird.auto.tfvars
+6 -6
View File
@@ -1,15 +1,15 @@
# NetBird Cloud objects for yucca-staging. Names are auto-prefixed "yucca_staging_"
# (all underscores); setup-key plaintext is written to the yucca_tf_staging vault.
# NetBird Cloud objects for yucca-staging. Names render UPPER_SNAKE, auto-prefixed
# "YUCCA_STAGING_"; setup-key plaintext is written to the yucca_tf_staging vault.
#
# Groups start empty — a peer joins a group by registering with a setup key whose
# auto_groups include it. NetBird is default-deny: a peer gets only the access its
# groups' policies grant.
groups = {
ci = {} # ephemeral CI runners → yucca_staging_ci
mgmt = {} # management nodes (configured via ansible) → yucca_staging_mgmt
talos = {} # Talos cluster nodes → yucca_staging_talos
k8s_operator = {} # in-cluster kubernetes operator → yucca_staging_k8s_operator
ci = {} # ephemeral CI runners → YUCCA_STAGING_CI
mgmt = {} # management nodes (configured via ansible) → YUCCA_STAGING_MGMT
talos = {} # Talos cluster nodes → YUCCA_STAGING_TALOS
k8s_operator = {} # in-cluster kubernetes operator → YUCCA_STAGING_K8S_OPERATOR
}
setup_keys = {
tf/shared/modules/netbird-env/main.tf
+19 -16
View File
@@ -1,5 +1,5 @@
# Per-environment NetBird (Cloud) objects. One NetBird Cloud account backs every
# env/site; objects are namespaced "<name_prefix>_<key>" (all underscores) so they
# env/site; objects are namespaced "<NAME_PREFIX>_<KEY>" (UPPER_SNAKE) so they
# coexist. Groups are created first; setup keys, policies, and networks resolve
# their logical group keys to NetBird-assigned group IDs.
#
@@ -7,11 +7,14 @@
# module only declares the dependency in versions.tf).
locals {
# Names are normalized to underscores (no hyphens) per the repo convention,
# e.g. name_prefix "yucca_prod_htz-fsn1" + key "mgmt" → "yucca_prod_htz_fsn1_mgmt".
# An explicit `name` override on a group is respected verbatim.
# All generated NetBird names are normalized to UPPER_SNAKE: uppercased, with
# hyphens → underscores. e.g. name_prefix "yucca_prod_htz-fsn1" + key "mgmt" →
# "YUCCA_PROD_HTZ_FSN1_MGMT"; an explicit group `name` override is normalized
# the same way (so "yucca_resource" → "YUCCA_RESOURCE"). The one exception is a
# network's display name (see netbird_network below), which keeps its casing/
# hyphens so labels like "HTZ-FSN1" survive.
group_names = {
for k, g in var.groups : k => coalesce(g.name, replace("${var.name_prefix}_${k}", "-", "_"))
for k, g in var.groups : k => upper(replace(coalesce(g.name, "${var.name_prefix}_${k}"), "-", "_"))
}
# Logical key → NetBird group ID, covering both the groups this layer owns and
@@ -31,7 +34,7 @@ locals {
description = r.description
groups = r.groups
enabled = r.enabled
name = coalesce(r.name, replace("${nk}_${rk}", "-", "_"))
name = upper(replace(coalesce(r.name, "${nk}_${rk}"), "-", "_"))
}
}
]...)
@@ -46,7 +49,7 @@ resource "netbird_group" "this" {
resource "netbird_setup_key" "this" {
for_each = var.setup_keys
name = replace("${var.name_prefix}_${each.key}", "-", "_")
name = upper(replace("${var.name_prefix}_${each.key}", "-", "_"))
type = each.value.type
expiry_seconds = each.value.expiry_seconds
usage_limit = each.value.usage_limit
@@ -59,14 +62,14 @@ resource "netbird_setup_key" "this" {
resource "netbird_policy" "this" {
for_each = var.policies
name = replace("${var.name_prefix}_${each.key}", "-", "_")
name = upper(replace("${var.name_prefix}_${each.key}", "-", "_"))
description = each.value.description
enabled = each.value.enabled
dynamic "rule" {
for_each = each.value.rules
content {
name = rule.value.name
name = upper(replace(rule.value.name, "-", "_"))
action = rule.value.action
protocol = rule.value.protocol
bidirectional = rule.value.bidirectional
@@ -83,11 +86,11 @@ resource "netbird_policy" "this" {
# A Network groups one or more resources (subnets/hosts) reachable through a set
# of routing peers (router.peer_groups — e.g. a site's mgmt nodes). resources[*]
# .groups controls which peers may reach that subnet. The Network's display name
# is the map key (or `name` override) verbatim — NOT underscore-normalized — so
# human labels like "HTZ-FSN1" survive.
# is the map key (or `name` override), uppercased but NOT underscore-normalized —
# so human labels like "HTZ-FSN1" survive (hyphen kept).
resource "netbird_network" "this" {
for_each = var.networks
name = coalesce(each.value.name, each.key)
name = upper(coalesce(each.value.name, each.key))
description = each.value.description
}
@@ -123,10 +126,10 @@ resource "onepassword_item" "setup_key" {
for_each = var.setup_keys
vault = data.onepassword_vault.env.uuid
# Title derived from the namespaced setup-key name so multiple sites writing to
# the SAME vault (prod: global + every site → yucca_tf_prod) never collide, e.g.
# "yucca_prod_htz_fsn1_mgmt" → NETBIRD_YUCCA_PROD_HTZ_FSN1_MGMT_SETUP_KEY.
title = "NETBIRD_${upper(netbird_setup_key.this[each.key].name)}_SETUP_KEY"
# Title derived from the (already UPPER_SNAKE) setup-key name so multiple sites
# writing to the SAME vault (prod: global + every site → yucca_tf_prod) never
# collide, e.g. "YUCCA_PROD_HTZ_FSN1_MGMT" → NETBIRD_YUCCA_PROD_HTZ_FSN1_MGMT_SETUP_KEY.
title = "NETBIRD_${netbird_setup_key.this[each.key].name}_SETUP_KEY"
category = "password"
password = netbird_setup_key.this[each.key].key
tf/shared/modules/netbird-env/variables.tf
+2 -2
View File
@@ -9,7 +9,7 @@ variable "env" {
}
variable "name_prefix" {
description = "Prefix for every NetBird object name so all envs/sites coexist in one NetBird Cloud account (e.g. \"yucca_staging\" → group \"yucca_staging_mgmt\"; \"yucca_prod_htz-fsn1\" → \"yucca_prod_htz_fsn1_mgmt\"). Hyphens in the prefix are normalized to underscores."
description = "Prefix for every NetBird object name so all envs/sites coexist in one NetBird Cloud account. Names render UPPER_SNAKE (uppercased, hyphens → underscores): e.g. \"yucca_staging\" → group \"YUCCA_STAGING_MGMT\"; \"yucca_prod_htz-fsn1\" → \"YUCCA_PROD_HTZ_FSN1_MGMT\"."
type = string
}
@@ -19,7 +19,7 @@ variable "vault" {
}
variable "groups" {
description = "NetBird groups keyed by logical name. Groups start empty — membership comes from setup keys' auto_groups as peers register. Set `name` only to override the derived \"<name_prefix>_<key>\"."
description = "NetBird groups keyed by logical name. Groups start empty — membership comes from setup keys' auto_groups as peers register. Set `name` only to override the derived \"<NAME_PREFIX>_<KEY>\" (an override is normalized to UPPER_SNAKE too)."
type = map(object({
name = optional(string)
}))