diff --git a/README.md b/README.md new file mode 100644 index 0000000..1e03eae --- /dev/null +++ b/README.md @@ -0,0 +1,413 @@ +# Yucca O11y + +Multi-environment observability platform running on bare-metal OVH servers with Talos Linux, managed by Flux CD. + +## Table of Contents + +- [Yucca O11y](#yucca-o11y) + - [Table of Contents](#table-of-contents) + - [Architecture Overview](#architecture-overview) + - [Multi-Environment Cluster Architecture](#multi-environment-cluster-architecture) + - [Environments](#environments) + - [Per-Node Stack](#per-node-stack) + - [Node Specifications (Staging)](#node-specifications-staging) + - [Flux CD GitOps Structure](#flux-cd-gitops-structure) + - [How It Works](#how-it-works) + - [Example Dependency Chain](#example-dependency-chain) + - [Renovate Integration](#renovate-integration) + - [Infrastructure Provisioning](#infrastructure-provisioning) + - [Dependency Flow](#dependency-flow) + - [Victoria Metrics Architecture](#victoria-metrics-architecture) + - [Components](#components) + - [Data Flow](#data-flow) + - [Authentication](#authentication) + - [Ingress: Envoy Gateway with TLS](#ingress-envoy-gateway-with-tls) + - [Traffic Flow](#traffic-flow) + - [DNS Management](#dns-management) + - [Wildcard DNS Records](#wildcard-dns-records) + - [Cert-Manager with OVH DNS-01](#cert-manager-with-ovh-dns-01) + - [OVH Load Balancer](#ovh-load-balancer) + - [Configuration](#configuration) + - [Load Balancer → Node Mapping](#load-balancer--node-mapping) + - [DNS Resolution](#dns-resolution) + - [Repository Layout](#repository-layout) + +--- + +## Architecture Overview + +```mermaid +graph TD + subgraph PROD ["Production"] + PLB["OVH Load Balancer
(Production)"] + PLON["Node: LON
Talos K8s
(single-node CP)"] + PRBX["Node: RBX
Talos K8s
(single-node CP)"] + PFRA["Node: FRA
Talos K8s
(single-node CP)"] + PLB --> PLON + PLB --> PRBX + PLB --> PFRA + end + + subgraph STG ["Staging"] + SLB["OVH Load Balancer
(Staging)"] + SLON["Node: LON
Talos K8s
(single-node CP)"] + SRBX["Node: RBX
Talos K8s
(single-node CP)"] + SFRA["Node: FRA
Talos K8s
(single-node CP)"] + SLB --> SLON + SLB --> SRBX + SLB --> SFRA + end + + TS(["Tailscale Mesh VPN
(cross-zone VM replication)"]) + PLON -.- TS + PRBX -.- TS + PFRA -.- TS + SLON -.- TS + SRBX -.- TS + SFRA -.- TS +``` + +Each environment (staging, production) runs **three independent single-node Talos Linux clusters** on OVH bare-metal servers in different European datacenters (London, Roubaix, Frankfurt). Each node is a full controlplane + worker. Each environment has its own **OVH Load Balancer** for traffic isolation. The clusters are connected via **Tailscale mesh VPN** for cross-zone Victoria Metrics replication. + +--- + +## Multi-Environment Cluster Architecture + +### Environments + +| Environment | Nodes | Datacenters | Description | +| --- | --- | --- | --- | +| **Staging** | 3 | LON, RBX, FRA | Full replica of production for testing | +| **Production** | 3 | LON, RBX, FRA | Production observability platform | + +### Per-Node Stack + +Each node runs an identical Kubernetes stack deployed via Flux: + +| Component | Purpose | +| --- | --- | +| **Flux Operator + Instance** | GitOps reconciliation from this repo | +| **Envoy Gateway** | Ingress controller (Gateway API) | +| **Envoy Proxy** | Data plane, TLS termination inside the cluster | +| **Victoria Metrics** | Metrics storage (VMSingle), collection (VMAgent), auth (VMAuth) | +| **cert-manager** | Automated TLS wildcard certificate management via OVH DNS-01 | +| **cert-manager-webhook-ovh** | OVH DNS-01 solver for cert-manager | +| **OpenEBS** | Local hostpath persistent volumes | +| **Prometheus Operator CRDs** | ServiceMonitor/PodMonitor CRDs for VM operator compatibility | + +### Node Specifications (Staging) + +| Node | Datacenter | Plan | Storage | RAM | VLAN IP | +| --- | --- | --- | --- | --- | --- | +| LON | London | 24sys012 | 2x512GB NVMe (RAID) | 32GB ECC | 10.150.200.10 | +| RBX | Roubaix | 24sys012 | 2x512GB NVMe (RAID) | 32GB ECC | 10.150.200.11 | +| FRA | Frankfurt | 24sys012 | 2x512GB NVMe (RAID) | 32GB ECC | 10.150.200.12 | + +Nodes are connected via OVH vRack (private VLAN 2600) where available, and Tailscale for cross-datacenter communication. + +--- + +## Flux CD GitOps Structure + +The repository uses a **base + overlay** pattern with Flux Kustomizations: + +```text +kubernetes/ +├── apps/ +│ ├── base/ # Shared manifests (HelmReleases, CRDs, configs) +│ │ ├── cert-manager/ +│ │ ├── cert-manager-webhook/ +│ │ ├── envoy-gateway/ +│ │ ├── envoy-proxy/ +│ │ ├── openebs/ +│ │ ├── prometheus-operator-crds/ +│ │ ├── victoria-metrics/ +│ │ ├── victoria-metrics-operator/ +│ │ └── victoria-metrics-operator-crds/ +│ ├── staging/ # Staging-specific overlays +│ │ ├── cert-manager/ +│ │ ├── envoy-system/ +│ │ ├── o11y/ +│ │ └── openebs-system/ +│ └── production/ # Production-specific overlays (mirrors staging) +└── clusters/ + ├── staging/ + │ └── apps.yaml # Cluster entrypoint for Flux + └── production/ + └── apps.yaml +``` + +### How It Works + +1. **Cluster bootstrap**: Terragrunt deploys the Flux Operator and Flux Instance via Helm into each node's cluster. The Flux Instance syncs from `kubernetes/clusters//`. + +2. **Entrypoint** (`clusters//apps.yaml`): A top-level Flux Kustomization that points at `kubernetes/apps//` and applies global patches (CRD install strategy, upgrade remediation). + +3. **Environment overlays** (`apps//`): Each subdirectory contains Flux Kustomizations that reference `base/` manifests and apply environment-specific patches (version pins for Renovate, dependency ordering). + +4. **Base manifests** (`apps/base/`): Contains the actual HelmReleases, OCI repositories, and raw Kubernetes resources. These are reusable across environments. + +5. **Variable substitution**: Environment-specific values (like cross-zone node IPs) are injected via `postBuild.substituteFrom` referencing ConfigMaps created by Terraform (e.g., `vm-zone-endpoints`). + +### Example Dependency Chain + +```mermaid +graph TD + POC[prometheus-operator-crds] --> VMOC[victoria-metrics-operator-crds] + VMOC --> VMO[victoria-metrics-operator] + VMO --> VM[victoria-metrics] + OE[openebs] --> VM + + CM[cert-manager] --> CMW[cert-manager-webhook-ovh] + + EG[envoy-gateway] --> EP[envoy-proxy] +``` + +### Renovate Integration + +OCI repository tags in staging overlays are annotated with Renovate comments for automated version bumps: + +```yaml +# renovate: datasource=docker depName=ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator +tag: 0.58.1 +``` + +--- + +## Infrastructure Provisioning + +Infrastructure is managed with **Terraform + Terragrunt** in four layers: + +```text +deployment/modules/ +├── ovh/account/ # 1. OVH servers, vRack, DNS records +├── tailscale/account/ # 2. Tailscale ACLs and tailnet settings +├── talos/cluster/ # 3. Talos Linux cluster bootstrap per node +└── kubernetes/helm/ # 4. Flux operator + secrets into each cluster +``` + +### Dependency Flow + +```mermaid +graph LR + OVH[ovh/account] --> TALOS[talos/cluster] + TS[tailscale/account] --> TALOS + TALOS --> K8S[kubernetes/helm] +``` + +1. **ovh/account**: Provisions bare-metal servers with Talos qcow2 images, creates vRack networking, and manages wildcard DNS records in OVH (e.g., `*.futostat.us`, `*.staging.futostat.us`, `*.futostatus.com`, `*.staging.futostatus.com`). + +2. **tailscale/account**: Configures Tailscale ACLs, device approval policies, and tailnet settings. + +3. **talos/cluster**: Bootstraps each node as a single-node Talos cluster with Tailscale extension. Creates auth keys, waits for Tailscale device registration, and generates kubeconfigs. + +4. **kubernetes/helm**: Deploys Flux Operator + Instance into each cluster using the kubeconfig from the Talos module. Creates secrets (VMAuth credentials) and ConfigMaps (cross-zone node IPs) needed by the workloads. + +--- + +## Victoria Metrics Architecture + +Each zone runs a full Victoria Metrics stack. Data is replicated across all three zones for durability, and reads can fan out across zones for availability. + +```mermaid +graph TD + subgraph ZONE1 ["ZONE 1 (LON) — Zone 2 (RBX) and Zone 3 (FRA) run identical stacks"] + EXT["All External Traffic
(remote write, dashboards, Grafana UI)"] --> EP["Envoy Proxy
TLS termination + routing
(NodePort via OVH LB :443)"] + + EP -->|"HTTPRoute
vmauth.futo.."| VMAE["VMAuth External
(ClusterIP)
read → all zones
write → VMAgent"] + EP -->|"HTTPRoute
grafana.futo.."| GF["Grafana
(ClusterIP)"] + + GF -->|reads from| VMAI["VMAuth Internal
(NodePort :30426)"] + + VMAE -->|write| VMA["VMAgent
(2 replicas)
remoteWrite to all zones"] + VMAE -->|"read → all zones"| VMAI + + VMAI --> VMS["VMSingle
(300Gi disk, 60d retention)"] + VMA -->|local| VMAI + end + + VMA -->|"Tailscale"| Z2["Zone 2 :30426
VMAuth Internal → VMSingle"] + VMA -->|"Tailscale"| Z3["Zone 3 :30426
VMAuth Internal → VMSingle"] +``` + +### Components + +| Component | Role | Service Type | Port | +| --- | --- | --- | --- | +| **VMSingle** | Time-series storage | ClusterIP | 8428 | +| **VMAgent** | Scrapes & replicates metrics (2 replicas, 50Gi buffer) | ClusterIP | 8429 | +| **VMAuth Internal** | Routes cross-zone read/write to local VMSingle | NodePort | 30426 | +| **VMAuth External** | Routes external reads across all zones, writes to VMAgent | ClusterIP | 8427 | + +### Data Flow + +1. **Ingestion (write path)**: + - External writers → OVH LB :443 → Envoy (TLS termination) → HTTPRoute → VMAuth External → VMAgent + - VMAgent writes to: + - Local zone via VMAuth Internal (ClusterIP → VMSingle) + - Zone 2 via Tailscale IP (:30426 → VMAuth Internal → VMSingle) + - Zone 3 via Tailscale IP (:30426 → VMAuth Internal → VMSingle) + +2. **Query (read path)**: + - **Grafana (in-cluster)**: Grafana → VMAuth Internal (ClusterIP) — no external hop, reads from the local zone's VMSingle directly + - **External API consumers**: → OVH LB :443 → Envoy → HTTPRoute → VMAuth External → fans out reads to all zones via Tailscale IPs + - VMAuth External uses `first_available` load balancing with retry on 429/5xx + +3. **Cross-zone communication**: All inter-zone traffic flows over Tailscale mesh VPN using private IPs. VMAgent's `remoteWrite` targets and VMAuth External's read fanout use Tailscale IPs injected via the `vm-zone-endpoints` ConfigMap. + +### Authentication + +Credentials are generated by Terraform (`random_password`) and injected as Kubernetes Secrets: + +- `vmauth-external-credentials`: reader/writer passwords for external VMAuth +- `vmauth-internal-credentials`: reader/writer passwords for internal VMAuth + +--- + +## Ingress: Envoy Gateway with TLS + +Envoy Gateway is the **single external ingress point** for all traffic using the **Gateway API**. TLS termination happens **inside the cluster** at Envoy — the OVH load balancer does TCP passthrough only. + +All services (Grafana, VMAuth, dashboards, etc.) are exposed as ClusterIP services with HTTPRoutes directing traffic through Envoy. This means: + +- One ingress point to secure, monitor, and rate-limit +- TLS managed entirely in-cluster by cert-manager +- No need to open additional NodePorts for individual services +- End-to-end encryption from client to Envoy + +### Traffic Flow + +```mermaid +graph LR + C["Client
(TLS)"] -->|":443"| LB["OVH LB
(TCP passthrough)"] + + subgraph NODE ["Cluster Node"] + EP["Envoy Proxy
(TLS termination)"] --> HR{"HTTPRoute
rules"} + HR -->|"vmauth.futostat.us"| VMA[VMAuth External] + HR -->|"grafana.futostat.us"| GF[Grafana] + HR -->|"*.futostat.us"| APP[App Service] + end + + LB -->|"NodePort"| EP +``` + +--- + +## DNS Management + +All DNS records are managed by **Terraform** in the `ovh/account` module using wildcard records. No in-cluster DNS controller (e.g., external-dns) is used. + +### Wildcard DNS Records + +Terraform creates A records and wildcard CNAME records pointing to the appropriate OVH Load Balancer for each environment. Production and staging each have their own dedicated load balancer: + +```text +Terraform: futostat.us → A → Production OVH LB IP +Terraform: futostatus.com → A → Production OVH LB IP +Terraform: *.futostat.us → CNAME → futostat.us (production) +Terraform: *.futostatus.com → CNAME → futostatus.com (production) +Terraform: staging.futostat.us → A → Staging OVH LB IP +Terraform: staging.futostatus.com → A → Staging OVH LB IP +Terraform: *.staging.futostat.us → CNAME → staging.futostat.us (staging) +Terraform: *.staging.futostatus.com → CNAME → staging.futostatus.com (staging) +``` + +This means any subdomain (e.g., `vmauth.futostat.us`, `grafana.staging.futostat.us`) automatically resolves to the correct environment's load balancer without per-service DNS records. Envoy handles routing to the correct backend based on the `Host` header via HTTPRoute rules. + +--- + +## Cert-Manager with OVH DNS-01 + +TLS certificates are issued automatically via **cert-manager** using **DNS-01 challenges** solved against OVH DNS. + +cert-manager with the [cert-manager-webhook-ovh](https://github.com/aureq/cert-manager-webhook-ovh) solver handles ACME DNS-01 challenges via the OVH API. Wildcard certificates are issued for each environment: + +| Environment | Certificate | Domains | +| --- | --- | --- | +| **Production** | `*.futostat.us` | `vmauth.futostat.us`, `grafana.futostat.us`, etc. | +| **Production** | `*.futostatus.com` | `vmauth.futostatus.com`, `grafana.futostatus.com`, etc. | +| **Staging** | `*.staging.futostat.us` | `vmauth.staging.futostat.us`, `grafana.staging.futostat.us`, etc. | +| **Staging** | `*.staging.futostatus.com` | `vmauth.staging.futostatus.com`, `grafana.staging.futostatus.com`, etc. | + +DNS-01 is used over HTTP-01 because it supports wildcard certificates, works regardless of load balancer configuration, and doesn't require exposing HTTP endpoints. + +--- + +## OVH Load Balancer + +Each environment (staging, production) has its own dedicated **OVH IP Load Balancing** instance, providing a stable public IP per environment and distributing traffic across that environment's three nodes. + +### Configuration + +| Setting | Value | +| --- | --- | +| **Instances** | 2 (one per environment: production, staging) | +| **Region** | Europe | +| **Mode** | TCP passthrough (for TLS passthrough to Envoy) | +| **Backend nodes** | 3 per LB (LON, RBX, FRA) | +| **Balance** | Round-robin | +| **Health check** | HTTP probe on backend ports | + +### Load Balancer → Node Mapping + +Each environment's load balancer routes to its own set of nodes: + +```mermaid +graph LR + subgraph PROD ["Production"] + PLB["Production OVH LB :443"] -->|"TCP passthrough"| PLON["LON :NodePort
(envoy proxy)"] + PLB -->|"TCP passthrough"| PRBX["RBX :NodePort
(envoy proxy)"] + PLB -->|"TCP passthrough"| PFRA["FRA :NodePort
(envoy proxy)"] + end + + subgraph STG ["Staging"] + SLB["Staging OVH LB :443"] -->|"TCP passthrough"| SLON["LON :NodePort
(envoy proxy)"] + SLB -->|"TCP passthrough"| SRBX["RBX :NodePort
(envoy proxy)"] + SLB -->|"TCP passthrough"| SFRA["FRA :NodePort
(envoy proxy)"] + end +``` + +Envoy then routes to the appropriate ClusterIP service based on HTTPRoute rules (e.g., `vmauth.futostat.us` → VMAuth External, `grafana.staging.futostat.us` → Grafana). + +### DNS Resolution + +All DNS is managed by Terraform using wildcard records. Production and staging domains point to their respective load balancers: + +| Record | Type | Target | Purpose | +| --- | --- | --- | --- | +| `futostat.us` | A | Production OVH LB IP | Production base record | +| `futostatus.com` | A | Production OVH LB IP | Production base record | +| `*.futostat.us` | CNAME | `futostat.us` | Production services | +| `*.futostatus.com` | CNAME | `futostatus.com` | Production services | +| `staging.futostat.us` | A | Staging OVH LB IP | Staging base record | +| `staging.futostatus.com` | A | Staging OVH LB IP | Staging base record | +| `*.staging.futostat.us` | CNAME | `staging.futostat.us` | Staging services | +| `*.staging.futostatus.com` | CNAME | `staging.futostatus.com` | Staging services | + +No per-service DNS records are needed — wildcard records cover all subdomains and Envoy routes traffic based on the `Host` header. + +--- + +## Repository Layout + +```text +. +├── deployment/ # Infrastructure as Code +│ └── modules/ +│ ├── ovh/account/ # OVH servers, vRack, DNS, load balancer +│ ├── tailscale/account/ # Tailscale ACLs and settings +│ ├── talos/cluster/ # Talos Linux cluster per node +│ │ └── modules/node/ # Per-node: Talos config, bootstrap, Tailscale +│ └── kubernetes/helm/ # Flux operator, secrets, ConfigMaps +│ └── modules/cluster/ # Per-cluster Helm releases and resources +├── kubernetes/ # Kubernetes manifests (GitOps source) +│ ├── apps/ +│ │ ├── base/ # Shared component definitions +│ │ ├── staging/ # Staging overlay + patches +│ │ └── production/ # Production overlay + patches +│ └── clusters/ +│ ├── staging/apps.yaml # Flux entrypoint for staging +│ └── production/apps.yaml # Flux entrypoint for production +├── renovate.json # Automated dependency updates +└── README.md +``` diff --git a/deployment/.env b/deployment/.env index faba38c..bae7675 100644 --- a/deployment/.env +++ b/deployment/.env @@ -1,14 +1,18 @@ export TF_VAR_env="${ENVIRONMENT:-dev}" export TF_VAR_stage=$STAGE -export TF_VAR_ovh_application_key=op://yucca_tf/OVH_APPLICATION_KEY/password -export TF_VAR_ovh_application_secret=op://yucca_tf/OVH_APPLICATION_SECRET/password -export TF_VAR_ovh_consumer_key=op://yucca_tf/OVH_CONSUMER_KEY/password +export TF_VAR_ovh_application_key=op://o11y_tf/OVH_APPLICATION_KEY/password +export TF_VAR_ovh_application_secret=op://o11y_tf/OVH_APPLICATION_SECRET/password +export TF_VAR_ovh_consumer_key=op://o11y_tf/OVH_CONSUMER_KEY/password -export TF_VAR_tf_state_s3_endpoint=op://yucca_tf/TF_STATE_S3_ENDPOINT/password -export TF_VAR_tf_state_s3_bucket=op://yucca_tf/TF_STATE_S3_BUCKET/password -export TF_VAR_tf_state_s3_region=op://yucca_tf/TF_STATE_S3_REGION/password -export TF_VAR_tf_state_s3_access_key=op://yucca_tf/TF_STATE_S3_ACCESS_KEY/password -export TF_VAR_tf_state_s3_secret_key=op://yucca_tf/TF_STATE_S3_SECRET_KEY/password +export TF_VAR_tf_state_s3_endpoint=op://o11y_tf/TF_STATE_S3_ENDPOINT/password +export TF_VAR_tf_state_s3_bucket=op://o11y_tf/TF_STATE_S3_BUCKET/password +export TF_VAR_tf_state_s3_region=op://o11y_tf/TF_STATE_S3_REGION/password +export TF_VAR_tf_state_s3_access_key=op://o11y_tf/TF_STATE_S3_ACCESS_KEY/password +export TF_VAR_tf_state_s3_secret_key=op://o11y_tf/TF_STATE_S3_SECRET_KEY/password -export TF_VAR_tailscale_api_key=op://yucca_tf/TAILSCALE_API_KEY/password -export TF_VAR_tailscale_tailnet_id=op://yucca_tf/TAILSCALE_TAILNET_ID/password +export TF_VAR_tailscale_api_key=op://o11y_tf/TAILSCALE_API_KEY/password +export TF_VAR_tailscale_tailnet_id=op://o11y_tf/TAILSCALE_TAILNET_ID/password + +export TF_VAR_op_credentials_file=op://o11y_tf/1PASS_CONNECT_SERVER_CREDENTIALS_FILE/password +export TF_VAR_op_connect_token=op://o11y_tf/1PASS_CONNECT_O11Y_SUPERUSER/password +export TF_VAR_op_connect_token_env=op://o11y_tf_${ENVIRONMENT:-dev}/1PASS_CONNECT_O11Y_READ/password diff --git a/deployment/modules/kubernetes/helm/.terraform.lock.hcl b/deployment/modules/kubernetes/helm/.terraform.lock.hcl index aabb3c6..6ee9ca9 100644 --- a/deployment/modules/kubernetes/helm/.terraform.lock.hcl +++ b/deployment/modules/kubernetes/helm/.terraform.lock.hcl @@ -6,6 +6,7 @@ provider "registry.opentofu.org/hashicorp/helm" { constraints = ">= 3.0.0, 3.1.1" hashes = [ "h1:8SOQHxpTUK0rYBsCoxqrvDRc75KZl9hBt1m7QLrs+QM=", + "h1:brfn5YltnzexsfqpWKw+5gS9U/m77e0An3hZQamlEZk=", "zh:09b38905e234c2e0b185332819614224660050b7e4b25e9e858b593ab01adafe", "zh:09fed1b19b8bcded169fb76304e06c5b1216d5ceba92948c23384f34ddbf1fac", "zh:2e0af220f3fe79048d82f6de91752ba9929c215819d3de4f82ccb473bcd9e5df", @@ -22,6 +23,7 @@ provider "registry.opentofu.org/hashicorp/kubernetes" { version = "3.0.1" constraints = ">= 3.0.0, 3.0.1" hashes = [ + "h1:e0dSpTDhKjin6KYIwLWTR+AHVC7wWlU3VfIx27n1bec=", "h1:idu+cVjePQ4hnl7zlOio+h1Gc5tQybs9KgPKWmyRd/A=", "zh:0a6aff192781cfd062efe814d87ec21c84273005a685c818fb3c771ec9fd7051", "zh:129f10760e8c727f7b593111e0026aa36aeb28c98f6500c749007aabba402332", @@ -40,6 +42,7 @@ provider "registry.opentofu.org/hashicorp/random" { constraints = "3.8.1" hashes = [ "h1:EHn3jsqOKhWjbg0X+psk0Ww96yz3N7ASqEKKuFvDFwo=", + "h1:LsYuJLZcYl1RiH7Hd3w90Ra5+k5cNqfdRUQXItkTI8Y=", "zh:25c458c7c676f15705e872202dad7dcd0982e4a48e7ea1800afa5fc64e77f4c8", "zh:2edeaf6f1b20435b2f81855ad98a2e70956d473be9e52a5fdf57ccd0098ba476", "zh:44becb9d5f75d55e36dfed0c5beabaf4c92e0a2bc61a3814d698271c646d48e7", diff --git a/deployment/modules/kubernetes/helm/helm.tf b/deployment/modules/kubernetes/helm/helm.tf index 4107cce..2ed414d 100644 --- a/deployment/modules/kubernetes/helm/helm.tf +++ b/deployment/modules/kubernetes/helm/helm.tf @@ -9,13 +9,20 @@ module "cluster" { cluster_name = each.value.name flux_operator_version = var.flux_operator_version - flux_instance_values_file = "${path.module}/values.yml" + flux_instance_values_file = "${path.module}/values.yaml" + env = var.env other_node_ips = local.other_node_ips[each.key] vmauth_external_reader_password = random_password.vmauth_external_reader.result vmauth_external_writer_password = random_password.vmauth_external_writer.result vmauth_internal_reader_password = random_password.vmauth_internal_reader.result vmauth_internal_writer_password = random_password.vmauth_internal_writer.result + ovh_application_key = var.ovh_application_key + ovh_application_secret = var.ovh_application_secret + ovh_consumer_key = var.ovh_consumer_key + op_credentials_file = var.op_credentials_file + op_connect_token = var.op_connect_token + op_connect_token_env = var.op_connect_token_env } output "cluster_deployments" { diff --git a/deployment/modules/kubernetes/helm/modules/cluster/main.tf b/deployment/modules/kubernetes/helm/modules/cluster/main.tf index 61ca96f..ec3f709 100644 --- a/deployment/modules/kubernetes/helm/modules/cluster/main.tf +++ b/deployment/modules/kubernetes/helm/modules/cluster/main.tf @@ -15,7 +15,7 @@ resource "helm_release" "flux_instance" { repository = "oci://ghcr.io/controlplaneio-fluxcd/charts" chart = "flux-instance" version = var.flux_operator_version - values = [file(var.flux_instance_values_file)] + values = [templatefile(var.flux_instance_values_file, { env = var.env })] cleanup_on_fail = true wait_for_jobs = true depends_on = [helm_release.flux_operator] diff --git a/deployment/modules/kubernetes/helm/modules/cluster/resources.tf b/deployment/modules/kubernetes/helm/modules/cluster/resources.tf index 3244df8..860e7b1 100644 --- a/deployment/modules/kubernetes/helm/modules/cluster/resources.tf +++ b/deployment/modules/kubernetes/helm/modules/cluster/resources.tf @@ -39,3 +39,87 @@ resource "kubernetes_secret_v1" "vmauth_internal_credentials" { "writer-password" = var.vmauth_internal_writer_password } } + +resource "kubernetes_namespace_v1" "cert_manager" { + depends_on = [helm_release.flux_operator] + + metadata { + annotations = { + "kustomize.toolkit.fluxcd.io/prune" = "disabled" + } + labels = { + "kustomize.toolkit.fluxcd.io/name" = "cluster-apps" + "kustomize.toolkit.fluxcd.io/namespace" = "flux-system" + } + name = "cert-manager" + } +} + +resource "kubernetes_secret_v1" "ovh_credentials" { + depends_on = [kubernetes_namespace_v1.cert_manager] + + metadata { + name = "ovh-credentials" + namespace = "cert-manager" + } + + data = { + applicationKey = var.ovh_application_key + applicationSecret = var.ovh_application_secret + applicationConsumerKey = var.ovh_consumer_key + } +} + +resource "kubernetes_namespace_v1" "external_secrets" { + depends_on = [helm_release.flux_operator] + + metadata { + annotations = { + "kustomize.toolkit.fluxcd.io/prune" = "disabled" + } + labels = { + "kustomize.toolkit.fluxcd.io/name" = "cluster-apps" + "kustomize.toolkit.fluxcd.io/namespace" = "flux-system" + } + name = "external-secrets" + } +} + +resource "kubernetes_secret_v1" "onepassword_connect_credentials" { + depends_on = [kubernetes_namespace_v1.external_secrets] + + metadata { + name = "onepassword-connect-credentials" + namespace = "external-secrets" + } + + data = { + "1password-credentials.json" = var.op_credentials_file + } +} + +resource "kubernetes_secret_v1" "onepassword_connect_token" { + depends_on = [kubernetes_namespace_v1.external_secrets] + + metadata { + name = "onepassword-connect" + namespace = "external-secrets" + } + + data = { + token = var.op_connect_token + } +} + +resource "kubernetes_secret_v1" "onepassword_connect_environment" { + depends_on = [kubernetes_namespace_v1.external_secrets] + + metadata { + name = "onepassword-connect-environment" + namespace = "external-secrets" + } + + data = { + token = var.op_connect_token_env + } +} diff --git a/deployment/modules/kubernetes/helm/modules/cluster/variables.tf b/deployment/modules/kubernetes/helm/modules/cluster/variables.tf index ee10c33..cd7ff2a 100644 --- a/deployment/modules/kubernetes/helm/modules/cluster/variables.tf +++ b/deployment/modules/kubernetes/helm/modules/cluster/variables.tf @@ -3,9 +3,14 @@ variable "cluster_name" { description = "Name of the cluster" } +variable "env" { + type = string + description = "Environment name (e.g. staging, production)" +} + variable "flux_operator_version" { type = string - default = "0.37.1" + default = "0.45.0" description = "Flux operator chart version" } @@ -38,3 +43,33 @@ variable "vmauth_internal_writer_password" { type = string sensitive = true } + +variable "ovh_application_key" { + type = string + sensitive = true +} + +variable "ovh_application_secret" { + type = string + sensitive = true +} + +variable "ovh_consumer_key" { + type = string + sensitive = true +} + +variable "op_credentials_file" { + type = string + sensitive = true +} + +variable "op_connect_token" { + type = string + sensitive = true +} + +variable "op_connect_token_env" { + type = string + sensitive = true +} diff --git a/deployment/modules/kubernetes/helm/values.yml b/deployment/modules/kubernetes/helm/values.yaml similarity index 94% rename from deployment/modules/kubernetes/helm/values.yml rename to deployment/modules/kubernetes/helm/values.yaml index 63882f2..72ace5d 100644 --- a/deployment/modules/kubernetes/helm/values.yml +++ b/deployment/modules/kubernetes/helm/values.yaml @@ -1,7 +1,6 @@ instance: distribution: - artifact: oci://ghcr.io/controlplaneio-fluxcd/flux-operator-manifests:v0.37.1 - version: 2.x + artifact: oci://ghcr.io/controlplaneio-fluxcd/flux-operator-manifests:v0.45.0 cluster: networkPolicy: false components: @@ -11,12 +10,10 @@ instance: - notification-controller sync: kind: GitRepository - url: "https://github.com/immich-app/yucca-o11y.git" - ref: "refs/heads/main" - path: kubernetes/flux/cluster - commonMetadata: - labels: - app.kubernetes.io/name: flux + url: https://github.com/immich-app/yucca-o11y.git + ref: refs/heads/multi-env-flux # TODO: change back to refs/heads/main before merge + path: kubernetes/clusters/${env} + interval: 1h kustomize: patches: - # Increase the number of workers diff --git a/deployment/modules/kubernetes/helm/variables.tf b/deployment/modules/kubernetes/helm/variables.tf index 659ba18..bed43c2 100644 --- a/deployment/modules/kubernetes/helm/variables.tf +++ b/deployment/modules/kubernetes/helm/variables.tf @@ -22,6 +22,36 @@ variable "clusters" { variable "flux_operator_version" { type = string - default = "0.37.1" + default = "0.45.0" description = "Flux operator chart version" } + +variable "ovh_application_key" { + type = string + sensitive = true +} + +variable "ovh_application_secret" { + type = string + sensitive = true +} + +variable "ovh_consumer_key" { + type = string + sensitive = true +} + +variable "op_credentials_file" { + type = string + sensitive = true +} + +variable "op_connect_token" { + type = string + sensitive = true +} + +variable "op_connect_token_env" { + type = string + sensitive = true +} diff --git a/deployment/modules/ovh/account/.terraform.lock.hcl b/deployment/modules/ovh/account/.terraform.lock.hcl index e9d4510..54a1070 100644 --- a/deployment/modules/ovh/account/.terraform.lock.hcl +++ b/deployment/modules/ovh/account/.terraform.lock.hcl @@ -6,6 +6,7 @@ provider "registry.opentofu.org/ovh/ovh" { constraints = "2.11.0" hashes = [ "h1:XlPqU8iVvTM+TR0cgkTh93PBDeWayoBo709kewzP9II=", + "h1:n72jkLuNqAztr+lSggZe7k1kUqunVOmym10Uk1qVSt0=", "zh:1991e3d0c663e1b6a5a886f0a97cf71e9275f39e60e73e5d83bd83d945c9feee", "zh:1d2ed8d9ed0205677151179a8d385a9e5df656fa70b29efdc5b776eb5d6b6c9c", "zh:25d8fa3c70e27cac11ac69f321ebc303ed5fa7febf833736f8a96279b14644ab", @@ -28,6 +29,7 @@ provider "registry.opentofu.org/siderolabs/talos" { constraints = "0.10.1" hashes = [ "h1:1/HTp6cDJWQJzRj8preKQvw3x/qffivOJhZx27OmAig=", + "h1:fc7ekyeFDNNvScqgHgowGjM9jnKFyUOMGfnEKJwuf1c=", "zh:0fa82a384b25a58b65523e0ea4768fa1212b1f5cfc0c9379d31162454fedcc9d", "zh:349463cdd4cdb36e03276fdb855e687242237c7cf0bd5871aea995a83838c52e", "zh:3885026ef7c1c7012d312fc37a35af70821650b10cef03b8ffd08d22145c117d", diff --git a/deployment/modules/ovh/account/dns.tf b/deployment/modules/ovh/account/dns.tf index 9080dc9..ddd9afa 100644 --- a/deployment/modules/ovh/account/dns.tf +++ b/deployment/modules/ovh/account/dns.tf @@ -12,7 +12,11 @@ locals { ovh_domain_name.futostatus_com.domain_name, ] - dns_records = flatten([ + # Wildcard subdomain: staging → *.staging, prod → * + wildcard_subdomain = var.env == "staging" ? "*.staging" : "*" + + # Per-node A records for direct node access + node_dns_records = flatten([ for domain in local.domains : [ for key, node in var.nodes : [ { @@ -27,19 +31,14 @@ locals { subdomain = "o11y-${var.env}-${key}.internal" target = node.vlan_ip }, - { - key = "${domain}-${key}-wildcard" - zone = domain - subdomain = "*.o11y-${var.env}-${key}" - target = ovh_dedicated_server.node[key].ip - }, ] ] ]) } +# Per-node A records (direct node access) resource "ovh_domain_zone_record" "nodes" { - for_each = { for record in local.dns_records : record.key => record } + for_each = { for record in local.node_dns_records : record.key => record } zone = each.value.zone subdomain = each.value.subdomain @@ -47,3 +46,25 @@ resource "ovh_domain_zone_record" "nodes" { ttl = 3600 target = each.value.target } + +resource "ovh_domain_zone_record" "lb" { + for_each = toset(local.domains) + + zone = each.value + subdomain = var.env == "staging" ? "staging" : "" + fieldtype = "A" + ttl = 3600 + target = ovh_iploadbalancing.this.ipv4 +} + +# Wildcard CNAMEs: *.staging.futostat.us → staging.futostat.us (staging) +# *.futostat.us → futostat.us (production) +resource "ovh_domain_zone_record" "wildcard" { + for_each = toset(local.domains) + + zone = each.value + subdomain = local.wildcard_subdomain + fieldtype = "CNAME" + ttl = 3600 + target = var.env == "staging" ? "staging.${each.value}." : "${each.value}." +} diff --git a/deployment/modules/ovh/account/loadbalancer.tf b/deployment/modules/ovh/account/loadbalancer.tf new file mode 100644 index 0000000..2033993 --- /dev/null +++ b/deployment/modules/ovh/account/loadbalancer.tf @@ -0,0 +1,74 @@ +data "ovh_order_cart_product_plan" "iplb" { + cart_id = data.ovh_order_cart.mycart.id + price_capacity = "renew" + product = "ipLoadbalancing" + plan_code = var.ovh_iplb_plan_code +} + +data "ovh_order_cart_product_options_plan" "iplb_zone" { + cart_id = data.ovh_order_cart_product_plan.iplb.cart_id + price_capacity = data.ovh_order_cart_product_plan.iplb.price_capacity + product = data.ovh_order_cart_product_plan.iplb.product + plan_code = data.ovh_order_cart_product_plan.iplb.plan_code + options_plan_code = "iplb-zone-lb1-${var.ovh_iplb_zone}" +} + +resource "ovh_iploadbalancing" "this" { + ovh_subsidiary = data.ovh_me.account.ovh_subsidiary + display_name = "o11y${local.resource_suffix}" + + plan { + duration = data.ovh_order_cart_product_plan.iplb.selected_price.0.duration + plan_code = data.ovh_order_cart_product_plan.iplb.plan_code + pricing_mode = data.ovh_order_cart_product_plan.iplb.selected_price.0.pricing_mode + } + + plan_option { + duration = data.ovh_order_cart_product_options_plan.iplb_zone.selected_price.0.duration + plan_code = data.ovh_order_cart_product_options_plan.iplb_zone.plan_code + pricing_mode = data.ovh_order_cart_product_options_plan.iplb_zone.selected_price.0.pricing_mode + } +} + +resource "ovh_iploadbalancing_tcp_farm" "envoy" { + service_name = ovh_iploadbalancing.this.service_name + display_name = "envoy-proxy" + zone = "all" + port = 30443 + balance = "roundrobin" + + probe { + type = "tcp" + port = 30443 + interval = 30 + } +} + +resource "ovh_iploadbalancing_tcp_farm_server" "envoy" { + for_each = ovh_dedicated_server.node + + service_name = ovh_iploadbalancing.this.service_name + farm_id = ovh_iploadbalancing_tcp_farm.envoy.id + display_name = "o11y-${var.env}-${each.key}" + address = each.value.ip + port = 30443 + status = "active" + weight = 1 +} + +resource "ovh_iploadbalancing_tcp_frontend" "https" { + service_name = ovh_iploadbalancing.this.service_name + display_name = "https" + zone = "all" + port = "443" + default_farm_id = ovh_iploadbalancing_tcp_farm.envoy.id +} + +resource "ovh_iploadbalancing_refresh" "this" { + service_name = ovh_iploadbalancing.this.service_name + keepers = [ + ovh_iploadbalancing_tcp_farm.envoy.id, + ovh_iploadbalancing_tcp_frontend.https.id, + join(",", [for k, v in ovh_iploadbalancing_tcp_farm_server.envoy : v.id]), + ] +} diff --git a/deployment/modules/ovh/account/outputs.tf b/deployment/modules/ovh/account/outputs.tf index 86ae77b..c1f2e5b 100644 --- a/deployment/modules/ovh/account/outputs.tf +++ b/deployment/modules/ovh/account/outputs.tf @@ -12,3 +12,13 @@ output "nodes" { description = "Node configuration map passed through for downstream modules" value = var.nodes } + +output "loadbalancer_ip" { + description = "Public IPv4 of the OVH IP Load Balancer" + value = ovh_iploadbalancing.this.ipv4 +} + +output "loadbalancer_service_name" { + description = "Service name of the OVH IP Load Balancer" + value = ovh_iploadbalancing.this.service_name +} diff --git a/deployment/modules/ovh/account/variables.tf b/deployment/modules/ovh/account/variables.tf index c33dc85..f3c9d71 100644 --- a/deployment/modules/ovh/account/variables.tf +++ b/deployment/modules/ovh/account/variables.tf @@ -40,3 +40,15 @@ variable "talos_schematic_id" { default = "4a0d65c669d46663f377e7161e50cfd570c401f26fd9e7bda34a0216b6f1922b" description = "Talos image factory schematic ID" } + +variable "ovh_iplb_plan_code" { + type = string + default = "iplb-lb1" + description = "OVH IP Load Balancing plan code" +} + +variable "ovh_iplb_zone" { + type = string + default = "rbx" + description = "OVH IP Load Balancing zone (datacenter, e.g. rbx, gra, fra, lon, bhs)" +} diff --git a/kubernetes/apps/base/cert-manager-webhook-ovh/helmrelease.yaml b/kubernetes/apps/base/cert-manager-webhook-ovh/helmrelease.yaml new file mode 100644 index 0000000..745393f --- /dev/null +++ b/kubernetes/apps/base/cert-manager-webhook-ovh/helmrelease.yaml @@ -0,0 +1,42 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: cert-manager-webhook-ovh +spec: + chartRef: + kind: OCIRepository + name: cert-manager-webhook-ovh + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + groupName: acme.futostat.us + issuers: + - name: letsencrypt-production + create: true + kind: ClusterIssuer + acmeServerUrl: https://acme-v02.api.letsencrypt.org/directory + email: acme@futostat.us + ovhEndpointName: ovh-eu + ovhAuthenticationMethod: application + ovhAuthenticationRef: + applicationKeyRef: + name: ovh-credentials + key: applicationKey + applicationSecretRef: + name: ovh-credentials + key: applicationSecret + applicationConsumerKeyRef: + name: ovh-credentials + key: applicationConsumerKey diff --git a/kubernetes/apps/infra/victoria-metrics-operator/app/kustomization.yaml b/kubernetes/apps/base/cert-manager-webhook-ovh/kustomization.yaml similarity index 100% rename from kubernetes/apps/infra/victoria-metrics-operator/app/kustomization.yaml rename to kubernetes/apps/base/cert-manager-webhook-ovh/kustomization.yaml index 10bfeb9..2ccd5a8 100644 --- a/kubernetes/apps/infra/victoria-metrics-operator/app/kustomization.yaml +++ b/kubernetes/apps/base/cert-manager-webhook-ovh/kustomization.yaml @@ -2,5 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./ocirepository.yaml - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/cert-manager-webhook-ovh/ocirepository.yaml b/kubernetes/apps/base/cert-manager-webhook-ovh/ocirepository.yaml new file mode 100644 index 0000000..27d422b --- /dev/null +++ b/kubernetes/apps/base/cert-manager-webhook-ovh/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: cert-manager-webhook-ovh +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 0.9.4 + url: oci://ghcr.io/aureq/charts/cert-manager-webhook-ovh diff --git a/kubernetes/apps/base/cert-manager/helmrelease.yaml b/kubernetes/apps/base/cert-manager/helmrelease.yaml new file mode 100644 index 0000000..7c253ac --- /dev/null +++ b/kubernetes/apps/base/cert-manager/helmrelease.yaml @@ -0,0 +1,36 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: cert-manager +spec: + chartRef: + kind: OCIRepository + name: cert-manager + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + crds: + enabled: true + dns01RecursiveNameservers: https://1.1.1.1:443/dns-query,https://1.0.0.1:443/dns-query + dns01RecursiveNameserversOnly: true + # TODO: Enable prometheus monitoring once we have prom crds installed and configured + # prometheus: + # enabled: true + # servicemonitor: + # enabled: true + webhook: + replicaCount: 2 + podDisruptionBudget: + enabled: true diff --git a/kubernetes/apps/infra/openebs/app/kustomization.yaml b/kubernetes/apps/base/cert-manager/kustomization.yaml similarity index 100% rename from kubernetes/apps/infra/openebs/app/kustomization.yaml rename to kubernetes/apps/base/cert-manager/kustomization.yaml index 10bfeb9..2ccd5a8 100644 --- a/kubernetes/apps/infra/openebs/app/kustomization.yaml +++ b/kubernetes/apps/base/cert-manager/kustomization.yaml @@ -2,5 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./ocirepository.yaml - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/infra/cert-manager/app/ocirepository.yaml b/kubernetes/apps/base/cert-manager/ocirepository.yaml similarity index 87% rename from kubernetes/apps/infra/cert-manager/app/ocirepository.yaml rename to kubernetes/apps/base/cert-manager/ocirepository.yaml index b740173..b9afd2b 100644 --- a/kubernetes/apps/infra/cert-manager/app/ocirepository.yaml +++ b/kubernetes/apps/base/cert-manager/ocirepository.yaml @@ -3,13 +3,11 @@ apiVersion: source.toolkit.fluxcd.io/v1 kind: OCIRepository metadata: name: cert-manager - namespace: network spec: interval: 15m - url: oci://quay.io/jetstack/charts/cert-manager - ref: - tag: v1.19.3 - layerSelector: mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip operation: copy + ref: + tag: v1.19.4 + url: oci://quay.io/jetstack/charts/cert-manager diff --git a/kubernetes/apps/base/cluster-secret-store/clustersecretstore.yaml b/kubernetes/apps/base/cluster-secret-store/clustersecretstore.yaml new file mode 100644 index 0000000..07618d7 --- /dev/null +++ b/kubernetes/apps/base/cluster-secret-store/clustersecretstore.yaml @@ -0,0 +1,32 @@ +--- +apiVersion: external-secrets.io/v1 +kind: ClusterSecretStore +metadata: + name: onepassword +spec: + provider: + onepassword: + connectHost: http://onepassword-connect.external-secrets.svc.cluster.local:8080 + vaults: + o11y_tf: 1 + auth: + secretRef: + connectTokenSecretRef: + name: onepassword-connect + namespace: external-secrets + key: token +--- +apiVersion: external-secrets.io/v1 +kind: ClusterSecretStore +metadata: + name: onepassword-environment +spec: + provider: + onepassword: + connectHost: http://onepassword-connect.external-secrets.svc.cluster.local:8080 + auth: + secretRef: + connectTokenSecretRef: + name: onepassword-connect-environment + namespace: external-secrets + key: token diff --git a/kubernetes/apps/kustomization.yaml b/kubernetes/apps/base/cluster-secret-store/kustomization.yaml similarity index 72% rename from kubernetes/apps/kustomization.yaml rename to kubernetes/apps/base/cluster-secret-store/kustomization.yaml index c4db29f..d4e6ae1 100644 --- a/kubernetes/apps/kustomization.yaml +++ b/kubernetes/apps/base/cluster-secret-store/kustomization.yaml @@ -2,5 +2,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./infra/ - - ./o11y/ + - ./clustersecretstore.yaml diff --git a/kubernetes/apps/base/echo/helmrelease.yaml b/kubernetes/apps/base/echo/helmrelease.yaml new file mode 100644 index 0000000..e269092 --- /dev/null +++ b/kubernetes/apps/base/echo/helmrelease.yaml @@ -0,0 +1,78 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: echo +spec: + chartRef: + kind: OCIRepository + name: echo + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + controllers: + echo: + strategy: RollingUpdate + containers: + app: + image: + repository: ghcr.io/mendhak/http-https-echo + tag: 40 + env: + HTTP_PORT: &port 8080 + LOG_WITHOUT_NEWLINE: true + LOG_IGNORE_PATH: /healthz + PROMETHEUS_ENABLED: true + probes: + liveness: &probes + enabled: true + custom: true + spec: + httpGet: + path: /healthz + port: *port + initialDelaySeconds: 0 + periodSeconds: 10 + timeoutSeconds: 1 + failureThreshold: 3 + readiness: *probes + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: {drop: ["ALL"]} + resources: + requests: + cpu: 10m + limits: + memory: 64Mi + defaultPodOptions: + securityContext: + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + service: + app: + ports: + http: + port: *port + serviceMonitor: + app: + endpoints: + - port: http + route: + app: + hostnames: [] + parentRefs: + - name: envoy + namespace: envoy-system diff --git a/kubernetes/apps/infra/cert-manager/app/kustomization.yaml b/kubernetes/apps/base/echo/kustomization.yaml similarity index 100% rename from kubernetes/apps/infra/cert-manager/app/kustomization.yaml rename to kubernetes/apps/base/echo/kustomization.yaml index 10bfeb9..2ccd5a8 100644 --- a/kubernetes/apps/infra/cert-manager/app/kustomization.yaml +++ b/kubernetes/apps/base/echo/kustomization.yaml @@ -2,5 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./ocirepository.yaml - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/echo/ocirepository.yaml b/kubernetes/apps/base/echo/ocirepository.yaml new file mode 100644 index 0000000..ff5d836 --- /dev/null +++ b/kubernetes/apps/base/echo/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: echo +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 4.6.2 + url: oci://ghcr.io/bjw-s-labs/helm/app-template diff --git a/kubernetes/apps/base/envoy-gateway/certificate.yaml b/kubernetes/apps/base/envoy-gateway/certificate.yaml new file mode 100644 index 0000000..ae9f551 --- /dev/null +++ b/kubernetes/apps/base/envoy-gateway/certificate.yaml @@ -0,0 +1,22 @@ +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: futostat-us +spec: + dnsNames: [] + issuerRef: + kind: ClusterIssuer + name: letsencrypt-production + secretName: futostat-us-tls +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: futostatus-com +spec: + dnsNames: [] + issuerRef: + kind: ClusterIssuer + name: letsencrypt-production + secretName: futostatus-com-tls diff --git a/kubernetes/apps/base/envoy-gateway/helmrelease.yaml b/kubernetes/apps/base/envoy-gateway/helmrelease.yaml new file mode 100644 index 0000000..bc07b55 --- /dev/null +++ b/kubernetes/apps/base/envoy-gateway/helmrelease.yaml @@ -0,0 +1,32 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: envoy-gateway +spec: + chartRef: + kind: OCIRepository + name: envoy-gateway + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + global: + imageRegistry: mirror.gcr.io + config: + envoyGateway: + provider: + type: Kubernetes + kubernetes: + deploy: + type: GatewayNamespace diff --git a/kubernetes/apps/base/envoy-gateway/kustomization.yaml b/kubernetes/apps/base/envoy-gateway/kustomization.yaml new file mode 100644 index 0000000..d9ab898 --- /dev/null +++ b/kubernetes/apps/base/envoy-gateway/kustomization.yaml @@ -0,0 +1,7 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./certificate.yaml + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/envoy-gateway/ocirepository.yaml b/kubernetes/apps/base/envoy-gateway/ocirepository.yaml new file mode 100644 index 0000000..3679fd47 --- /dev/null +++ b/kubernetes/apps/base/envoy-gateway/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: envoy-gateway +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 1.7.0 + url: oci://mirror.gcr.io/envoyproxy/gateway-helm diff --git a/kubernetes/apps/base/envoy-proxy/backendtrafficpolicy.yaml b/kubernetes/apps/base/envoy-proxy/backendtrafficpolicy.yaml new file mode 100644 index 0000000..df394b7 --- /dev/null +++ b/kubernetes/apps/base/envoy-proxy/backendtrafficpolicy.yaml @@ -0,0 +1,25 @@ +--- +apiVersion: gateway.envoyproxy.io/v1alpha1 +kind: BackendTrafficPolicy +metadata: + name: envoy +spec: + compressor: + - type: Zstd + zstd: {} + - type: Brotli + brotli: {} + - type: Gzip + gzip: {} + retry: + numRetries: 2 + retryOn: + triggers: + - reset + targetSelectors: + - group: gateway.networking.k8s.io + kind: Gateway + tcpKeepalive: {} + timeout: + http: + requestTimeout: 0s diff --git a/kubernetes/apps/base/envoy-proxy/clienttrafficpolicy.yaml b/kubernetes/apps/base/envoy-proxy/clienttrafficpolicy.yaml new file mode 100644 index 0000000..d796e73 --- /dev/null +++ b/kubernetes/apps/base/envoy-proxy/clienttrafficpolicy.yaml @@ -0,0 +1,21 @@ +apiVersion: gateway.envoyproxy.io/v1alpha1 +kind: ClientTrafficPolicy +metadata: + name: envoy +spec: + # clientIPDetection: + # xForwardedFor: + # trustedCIDRs: + # - 10.42.0.0/16 + http2: + onInvalidMessage: TerminateStream + http3: {} + targetSelectors: + - group: gateway.networking.k8s.io + kind: Gateway + tcpKeepalive: {} + tls: + minVersion: "1.2" + alpnProtocols: + - h2 + - http/1.1 diff --git a/kubernetes/apps/base/envoy-proxy/envoyproxy.yaml b/kubernetes/apps/base/envoy-proxy/envoyproxy.yaml new file mode 100644 index 0000000..ea6233c --- /dev/null +++ b/kubernetes/apps/base/envoy-proxy/envoyproxy.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: gateway.envoyproxy.io/v1alpha1 +kind: EnvoyProxy +metadata: + name: envoy +spec: + logging: + level: + default: info + provider: + type: Kubernetes + kubernetes: + envoyDeployment: + replicas: 2 + container: + imageRepository: mirror.gcr.io/envoyproxy/envoy + resources: + requests: + cpu: 100m + limits: + memory: 1Gi + envoyService: + type: NodePort + patch: + type: StrategicMerge + value: + spec: + ports: + - port: 30443 + nodePort: 30443 + telemetry: + metrics: + prometheus: + compression: + type: Zstd diff --git a/kubernetes/apps/base/envoy-proxy/gateway.yaml b/kubernetes/apps/base/envoy-proxy/gateway.yaml new file mode 100644 index 0000000..9743b22 --- /dev/null +++ b/kubernetes/apps/base/envoy-proxy/gateway.yaml @@ -0,0 +1,20 @@ +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: envoy +spec: + gatewayClassName: envoy + listeners: + - name: https + protocol: HTTPS + port: 30443 + allowedRoutes: + namespaces: + from: All + tls: + certificateRefs: + - kind: Secret + name: futostat-us-tls + - kind: Secret + name: futostatus-com-tls diff --git a/kubernetes/apps/base/envoy-proxy/gatewayclass.yaml b/kubernetes/apps/base/envoy-proxy/gatewayclass.yaml new file mode 100644 index 0000000..f4410c2 --- /dev/null +++ b/kubernetes/apps/base/envoy-proxy/gatewayclass.yaml @@ -0,0 +1,12 @@ +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: GatewayClass +metadata: + name: envoy +spec: + controllerName: gateway.envoyproxy.io/gatewayclass-controller + parametersRef: + group: gateway.envoyproxy.io + kind: EnvoyProxy + name: envoy + namespace: envoy-system diff --git a/kubernetes/apps/base/envoy-proxy/kustomization.yaml b/kubernetes/apps/base/envoy-proxy/kustomization.yaml new file mode 100644 index 0000000..4fc3383 --- /dev/null +++ b/kubernetes/apps/base/envoy-proxy/kustomization.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./backendtrafficpolicy.yaml + - ./clienttrafficpolicy.yaml + - ./envoyproxy.yaml + - ./gateway.yaml + - ./gatewayclass.yaml diff --git a/kubernetes/apps/base/external-secrets/helmrelease.yaml b/kubernetes/apps/base/external-secrets/helmrelease.yaml new file mode 100644 index 0000000..e6a4508 --- /dev/null +++ b/kubernetes/apps/base/external-secrets/helmrelease.yaml @@ -0,0 +1,25 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: external-secrets +spec: + chartRef: + kind: OCIRepository + name: external-secrets + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + serviceMonitor: + enabled: true diff --git a/kubernetes/apps/infra/victoria-metrics-operator-crds/app/kustomization.yaml b/kubernetes/apps/base/external-secrets/kustomization.yaml similarity index 100% rename from kubernetes/apps/infra/victoria-metrics-operator-crds/app/kustomization.yaml rename to kubernetes/apps/base/external-secrets/kustomization.yaml index 10bfeb9..2ccd5a8 100644 --- a/kubernetes/apps/infra/victoria-metrics-operator-crds/app/kustomization.yaml +++ b/kubernetes/apps/base/external-secrets/kustomization.yaml @@ -2,5 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./ocirepository.yaml - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/external-secrets/ocirepository.yaml b/kubernetes/apps/base/external-secrets/ocirepository.yaml new file mode 100644 index 0000000..3149829 --- /dev/null +++ b/kubernetes/apps/base/external-secrets/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: external-secrets +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 2.2.0 + url: oci://ghcr.io/external-secrets/charts/external-secrets diff --git a/kubernetes/apps/base/grafana-operator/helmrelease.yaml b/kubernetes/apps/base/grafana-operator/helmrelease.yaml new file mode 100644 index 0000000..efeb02f --- /dev/null +++ b/kubernetes/apps/base/grafana-operator/helmrelease.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: grafana-operator +spec: + chartRef: + kind: OCIRepository + name: grafana-operator + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + dashboard: + enabled: true + serviceMonitor: + enabled: true diff --git a/kubernetes/apps/base/grafana-operator/kustomization.yaml b/kubernetes/apps/base/grafana-operator/kustomization.yaml new file mode 100644 index 0000000..2ccd5a8 --- /dev/null +++ b/kubernetes/apps/base/grafana-operator/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/grafana-operator/ocirepository.yaml b/kubernetes/apps/base/grafana-operator/ocirepository.yaml new file mode 100644 index 0000000..a5537f7 --- /dev/null +++ b/kubernetes/apps/base/grafana-operator/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: grafana-operator +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 5.22.2 + url: oci://ghcr.io/grafana/helm-charts/grafana-operator diff --git a/kubernetes/apps/base/grafana/externalsecret.yaml b/kubernetes/apps/base/grafana/externalsecret.yaml new file mode 100644 index 0000000..10f0627 --- /dev/null +++ b/kubernetes/apps/base/grafana/externalsecret.yaml @@ -0,0 +1,17 @@ +--- +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: grafana-admin-password +spec: + secretStoreRef: + kind: ClusterSecretStore + name: onepassword + target: + name: grafana-admin-password + template: + data: + GF_SECURITY_ADMIN_PASSWORD: "{{ .password }}" + dataFrom: + - extract: + key: GRAFANA_ADMIN_PASSWORD diff --git a/kubernetes/apps/base/grafana/grafana.yaml b/kubernetes/apps/base/grafana/grafana.yaml new file mode 100644 index 0000000..f66b636 --- /dev/null +++ b/kubernetes/apps/base/grafana/grafana.yaml @@ -0,0 +1,77 @@ +--- +apiVersion: grafana.integreatly.org/v1beta1 +kind: Grafana +metadata: + name: grafana + labels: + dashboards: grafana +spec: + config: + analytics: + check_for_updates: "false" + check_for_plugin_updates: "false" + feedback_links_enabled: "false" + reporting_enabled: "false" + auth: + disable_login_form: "false" + auth.anonymous: + enabled: "true" + log: + mode: console + metrics: + enabled: "true" + news: + news_feed_enabled: "false" + plugins: + plugin_admin_enabled: "false" + security: + angular_support_enabled: "true" + server: + enable_gzip: "true" + deployment: + spec: + strategy: + type: Recreate + template: + spec: + containers: + - name: grafana + env: + - name: GF_SECURITY_ADMIN_PASSWORD + valueFrom: + secretKeyRef: + name: grafana-admin-password + key: GF_SECURITY_ADMIN_PASSWORD + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: {drop: ["ALL"]} + securityContext: + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + fsGroupChangePolicy: OnRootMismatch + volumes: + - name: grafana-data + persistentVolumeClaim: + claimName: grafana-pvc + httpRoute: + spec: + hostnames: [] + parentRefs: + - name: envoy + namespace: envoy-system + rules: + - backendRefs: + - name: grafana-service + port: 3000 + persistentVolumeClaim: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + storageClassName: openebs-hostpath + disableDefaultSecurityContext: All diff --git a/kubernetes/apps/base/grafana/kustomization.yaml b/kubernetes/apps/base/grafana/kustomization.yaml new file mode 100644 index 0000000..df57ef7 --- /dev/null +++ b/kubernetes/apps/base/grafana/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./externalsecret.yaml + - ./grafana.yaml diff --git a/kubernetes/apps/base/onepassword-connect/helmrelease.yaml b/kubernetes/apps/base/onepassword-connect/helmrelease.yaml new file mode 100644 index 0000000..b13fdd0 --- /dev/null +++ b/kubernetes/apps/base/onepassword-connect/helmrelease.yaml @@ -0,0 +1,28 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: onepassword-connect +spec: + chartRef: + kind: OCIRepository + name: onepassword-connect + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: + api: + serviceMonitor: + enabled: true + connect: + credentialsName: onepassword-connect-credentials diff --git a/kubernetes/apps/base/onepassword-connect/kustomization.yaml b/kubernetes/apps/base/onepassword-connect/kustomization.yaml new file mode 100644 index 0000000..2ccd5a8 --- /dev/null +++ b/kubernetes/apps/base/onepassword-connect/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/onepassword-connect/ocirepository.yaml b/kubernetes/apps/base/onepassword-connect/ocirepository.yaml new file mode 100644 index 0000000..6e25ed1 --- /dev/null +++ b/kubernetes/apps/base/onepassword-connect/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: onepassword-connect +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 2.4.1 + url: oci://ghcr.io/1password/connect diff --git a/kubernetes/apps/infra/openebs/app/helmrelease.yaml b/kubernetes/apps/base/openebs/helmrelease.yaml similarity index 58% rename from kubernetes/apps/infra/openebs/app/helmrelease.yaml rename to kubernetes/apps/base/openebs/helmrelease.yaml index 6e76630..4df71bb 100644 --- a/kubernetes/apps/infra/openebs/app/helmrelease.yaml +++ b/kubernetes/apps/base/openebs/helmrelease.yaml @@ -2,36 +2,52 @@ apiVersion: helm.toolkit.fluxcd.io/v2 kind: HelmRelease metadata: name: openebs - namespace: storage spec: chartRef: kind: OCIRepository name: openebs + install: + crds: CreateReplace interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 values: + preUpgradeHook: + enabled: false localpv-provisioner: + analytics: + enabled: false localpv: + image: + registry: quay.io/ basePath: &hostPath /var/mnt/u-hostpath - replicas: 1 - enableLeaderElection: true - requests: - cpu: 20m - memory: 64Mi - limits: - memory: 128Mi - hostpathClass: enabled: true name: openebs-hostpath reclaimPolicy: Delete isDefaultClass: true basePath: *hostPath - - analytics: + helperPod: + image: + registry: quay.io/ + openebs-crds: + csi: + volumeSnapshots: + enabled: false + keep: false + zfs-localpv: enabled: false - alloy: + lvm-localpv: enabled: false - loki: + mayastor: enabled: false engines: local: @@ -42,8 +58,9 @@ spec: replicated: mayastor: enabled: false - openebs-crds: - csi: - volumeSnapshots: - enabled: false - keep: false + loki: + enabled: false + alloy: + enabled: false + minio: + enabled: false diff --git a/kubernetes/apps/base/openebs/kustomization.yaml b/kubernetes/apps/base/openebs/kustomization.yaml new file mode 100644 index 0000000..2ccd5a8 --- /dev/null +++ b/kubernetes/apps/base/openebs/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/infra/openebs/app/ocirepository.yaml b/kubernetes/apps/base/openebs/ocirepository.yaml similarity index 74% rename from kubernetes/apps/infra/openebs/app/ocirepository.yaml rename to kubernetes/apps/base/openebs/ocirepository.yaml index dfef564..5dc38d2 100644 --- a/kubernetes/apps/infra/openebs/app/ocirepository.yaml +++ b/kubernetes/apps/base/openebs/ocirepository.yaml @@ -3,13 +3,11 @@ apiVersion: source.toolkit.fluxcd.io/v1 kind: OCIRepository metadata: name: openebs - namespace: storage spec: interval: 15m - url: oci://ghcr.io/home-operations/charts-mirror/openebs - ref: - tag: 4.4.0 - layerSelector: mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip operation: copy + ref: + tag: 4.4.0 + url: oci://ghcr.io/openebs/charts/openebs diff --git a/kubernetes/apps/base/prometheus-operator-crds/helmrelease.yaml b/kubernetes/apps/base/prometheus-operator-crds/helmrelease.yaml new file mode 100644 index 0000000..958708b --- /dev/null +++ b/kubernetes/apps/base/prometheus-operator-crds/helmrelease.yaml @@ -0,0 +1,23 @@ +--- +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: prometheus-operator-crds +spec: + chartRef: + kind: OCIRepository + name: prometheus-operator-crds + install: + crds: CreateReplace + interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 + values: {} diff --git a/kubernetes/apps/base/prometheus-operator-crds/kustomization.yaml b/kubernetes/apps/base/prometheus-operator-crds/kustomization.yaml new file mode 100644 index 0000000..2ccd5a8 --- /dev/null +++ b/kubernetes/apps/base/prometheus-operator-crds/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/base/prometheus-operator-crds/ocirepository.yaml b/kubernetes/apps/base/prometheus-operator-crds/ocirepository.yaml new file mode 100644 index 0000000..4ec7c26 --- /dev/null +++ b/kubernetes/apps/base/prometheus-operator-crds/ocirepository.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: source.toolkit.fluxcd.io/v1 +kind: OCIRepository +metadata: + name: prometheus-operator-crds +spec: + interval: 15m + layerSelector: + mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip + operation: copy + ref: + tag: 27.0.0 + url: oci://ghcr.io/prometheus-community/charts/prometheus-operator-crds diff --git a/kubernetes/apps/infra/victoria-metrics-operator-crds/app/helmrelease.yaml b/kubernetes/apps/base/victoria-metrics-operator-crds/helmrelease.yaml similarity index 50% rename from kubernetes/apps/infra/victoria-metrics-operator-crds/app/helmrelease.yaml rename to kubernetes/apps/base/victoria-metrics-operator-crds/helmrelease.yaml index 708c62c..86f5d7f 100644 --- a/kubernetes/apps/infra/victoria-metrics-operator-crds/app/helmrelease.yaml +++ b/kubernetes/apps/base/victoria-metrics-operator-crds/helmrelease.yaml @@ -8,5 +8,17 @@ spec: chartRef: kind: OCIRepository name: victoria-metrics-operator-crds + install: + crds: CreateReplace interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 values: {} diff --git a/kubernetes/apps/base/victoria-metrics-operator-crds/kustomization.yaml b/kubernetes/apps/base/victoria-metrics-operator-crds/kustomization.yaml new file mode 100644 index 0000000..2ccd5a8 --- /dev/null +++ b/kubernetes/apps/base/victoria-metrics-operator-crds/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/infra/victoria-metrics-operator-crds/app/ocirepository.yaml b/kubernetes/apps/base/victoria-metrics-operator-crds/ocirepository.yaml similarity index 93% rename from kubernetes/apps/infra/victoria-metrics-operator-crds/app/ocirepository.yaml rename to kubernetes/apps/base/victoria-metrics-operator-crds/ocirepository.yaml index 5b8f6b9..77ff7d4 100644 --- a/kubernetes/apps/infra/victoria-metrics-operator-crds/app/ocirepository.yaml +++ b/kubernetes/apps/base/victoria-metrics-operator-crds/ocirepository.yaml @@ -3,13 +3,11 @@ apiVersion: source.toolkit.fluxcd.io/v1 kind: OCIRepository metadata: name: victoria-metrics-operator-crds - namespace: operators spec: interval: 15m - url: oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator-crds - ref: - tag: 0.6.1 - layerSelector: mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip operation: copy + ref: + tag: 0.6.1 + url: oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator-crds diff --git a/kubernetes/apps/infra/victoria-metrics-operator/app/helmrelease.yaml b/kubernetes/apps/base/victoria-metrics-operator/helmrelease.yaml similarity index 66% rename from kubernetes/apps/infra/victoria-metrics-operator/app/helmrelease.yaml rename to kubernetes/apps/base/victoria-metrics-operator/helmrelease.yaml index 835fca1..d6be678 100644 --- a/kubernetes/apps/infra/victoria-metrics-operator/app/helmrelease.yaml +++ b/kubernetes/apps/base/victoria-metrics-operator/helmrelease.yaml @@ -3,12 +3,23 @@ apiVersion: helm.toolkit.fluxcd.io/v2 kind: HelmRelease metadata: name: victoria-metrics-operator - namespace: operators spec: chartRef: kind: OCIRepository name: victoria-metrics-operator + install: + crds: CreateReplace interval: 1h + rollback: + cleanupOnFail: true + upgrade: + cleanupOnFail: true + crds: CreateReplace + strategy: + name: RemediateOnFailure + remediation: + remediateLastFailure: true + retries: 2 values: admissionWebhooks: enabled: true diff --git a/kubernetes/apps/base/victoria-metrics-operator/kustomization.yaml b/kubernetes/apps/base/victoria-metrics-operator/kustomization.yaml new file mode 100644 index 0000000..2ccd5a8 --- /dev/null +++ b/kubernetes/apps/base/victoria-metrics-operator/kustomization.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./helmrelease.yaml + - ./ocirepository.yaml diff --git a/kubernetes/apps/infra/victoria-metrics-operator/app/ocirepository.yaml b/kubernetes/apps/base/victoria-metrics-operator/ocirepository.yaml similarity index 93% rename from kubernetes/apps/infra/victoria-metrics-operator/app/ocirepository.yaml rename to kubernetes/apps/base/victoria-metrics-operator/ocirepository.yaml index d868cbd..eea4adf 100644 --- a/kubernetes/apps/infra/victoria-metrics-operator/app/ocirepository.yaml +++ b/kubernetes/apps/base/victoria-metrics-operator/ocirepository.yaml @@ -3,13 +3,11 @@ apiVersion: source.toolkit.fluxcd.io/v1 kind: OCIRepository metadata: name: victoria-metrics-operator - namespace: operators spec: interval: 15m - url: oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator - ref: - tag: 0.58.1 - layerSelector: mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip operation: copy + ref: + tag: 0.58.1 + url: oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator diff --git a/kubernetes/apps/base/victoria-metrics/httproute.yaml b/kubernetes/apps/base/victoria-metrics/httproute.yaml new file mode 100644 index 0000000..4c815dc --- /dev/null +++ b/kubernetes/apps/base/victoria-metrics/httproute.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: vmauth-external +spec: + parentRefs: + - name: envoy + namespace: envoy-system + rules: + - backendRefs: + - name: vmauth-vmauth-external + port: 8427 diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/kustomization.yaml b/kubernetes/apps/base/victoria-metrics/kustomization.yaml similarity index 64% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/kustomization.yaml rename to kubernetes/apps/base/victoria-metrics/kustomization.yaml index 4a77e73..c83dbb2 100644 --- a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/kustomization.yaml +++ b/kubernetes/apps/base/victoria-metrics/kustomization.yaml @@ -2,7 +2,12 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./vmuser-external-write.yaml + - ./httproute.yaml + - ./vmagent.yaml + - ./vmauth-external.yaml + - ./vmauth-internal.yaml + - ./vmsingle.yaml - ./vmuser-external-read.yaml - - ./vmuser-internal-write.yaml + - ./vmuser-external-write.yaml - ./vmuser-internal-read.yaml + - ./vmuser-internal-write.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmagent.yaml b/kubernetes/apps/base/victoria-metrics/vmagent.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmagent.yaml rename to kubernetes/apps/base/victoria-metrics/vmagent.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmauth-external.yaml b/kubernetes/apps/base/victoria-metrics/vmauth-external.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmauth-external.yaml rename to kubernetes/apps/base/victoria-metrics/vmauth-external.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmauth-internal.yaml b/kubernetes/apps/base/victoria-metrics/vmauth-internal.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmauth-internal.yaml rename to kubernetes/apps/base/victoria-metrics/vmauth-internal.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmsingle.yaml b/kubernetes/apps/base/victoria-metrics/vmsingle.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmsingle.yaml rename to kubernetes/apps/base/victoria-metrics/vmsingle.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-external-read.yaml b/kubernetes/apps/base/victoria-metrics/vmuser-external-read.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-external-read.yaml rename to kubernetes/apps/base/victoria-metrics/vmuser-external-read.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-external-write.yaml b/kubernetes/apps/base/victoria-metrics/vmuser-external-write.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-external-write.yaml rename to kubernetes/apps/base/victoria-metrics/vmuser-external-write.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-internal-read.yaml b/kubernetes/apps/base/victoria-metrics/vmuser-internal-read.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-internal-read.yaml rename to kubernetes/apps/base/victoria-metrics/vmuser-internal-read.yaml diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-internal-write.yaml b/kubernetes/apps/base/victoria-metrics/vmuser-internal-write.yaml similarity index 100% rename from kubernetes/apps/o11y/victoria-metrics-stack/app/vmusers/vmuser-internal-write.yaml rename to kubernetes/apps/base/victoria-metrics/vmuser-internal-write.yaml diff --git a/kubernetes/apps/infra/cert-manager/app/helmrelease.yaml b/kubernetes/apps/infra/cert-manager/app/helmrelease.yaml deleted file mode 100644 index 9fcbcb2..0000000 --- a/kubernetes/apps/infra/cert-manager/app/helmrelease.yaml +++ /dev/null @@ -1,22 +0,0 @@ ---- -apiVersion: helm.toolkit.fluxcd.io/v2 -kind: HelmRelease -metadata: - name: cert-manager - namespace: network -spec: - chartRef: - kind: OCIRepository - name: cert-manager - interval: 1h - values: - crds: - enabled: true - - resources: - requests: - cpu: 10m - memory: 64Mi - limits: - cpu: 200m - memory: 256Mi diff --git a/kubernetes/apps/infra/kustomization.yaml b/kubernetes/apps/infra/kustomization.yaml deleted file mode 100644 index 583dec9..0000000 --- a/kubernetes/apps/infra/kustomization.yaml +++ /dev/null @@ -1,9 +0,0 @@ ---- -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - ./namespace.yaml - - ./cert-manager/ks.yaml - - ./victoria-metrics-operator-crds/ks.yaml - - ./victoria-metrics-operator/ks.yaml - - ./openebs/ks.yaml diff --git a/kubernetes/apps/infra/namespace.yaml b/kubernetes/apps/infra/namespace.yaml deleted file mode 100644 index c5ebbc3..0000000 --- a/kubernetes/apps/infra/namespace.yaml +++ /dev/null @@ -1,19 +0,0 @@ ---- -apiVersion: v1 -kind: Namespace -metadata: - name: storage - labels: - pod-security.kubernetes.io/enforce: privileged - pod-security.kubernetes.io/audit: privileged - pod-security.kubernetes.io/warn: privileged ---- -apiVersion: v1 -kind: Namespace -metadata: - name: network ---- -apiVersion: v1 -kind: Namespace -metadata: - name: operators diff --git a/kubernetes/apps/infra/victoria-metrics-operator-crds/ks.yaml b/kubernetes/apps/infra/victoria-metrics-operator-crds/ks.yaml deleted file mode 100644 index dfc00e8..0000000 --- a/kubernetes/apps/infra/victoria-metrics-operator-crds/ks.yaml +++ /dev/null @@ -1,15 +0,0 @@ ---- -apiVersion: kustomize.toolkit.fluxcd.io/v1 -kind: Kustomization -metadata: - name: victoria-metrics-operator-crds - namespace: flux-system -spec: - targetNamespace: operators - interval: 1h - path: ./kubernetes/apps/infra/victoria-metrics-operator-crds/app - prune: false - sourceRef: - kind: GitRepository - name: flux-system - wait: true diff --git a/kubernetes/apps/infra/victoria-metrics-operator/ks.yaml b/kubernetes/apps/infra/victoria-metrics-operator/ks.yaml deleted file mode 100644 index 9ff14ee..0000000 --- a/kubernetes/apps/infra/victoria-metrics-operator/ks.yaml +++ /dev/null @@ -1,18 +0,0 @@ ---- -apiVersion: kustomize.toolkit.fluxcd.io/v1 -kind: Kustomization -metadata: - name: victoria-metrics-operator - namespace: flux-system -spec: - targetNamespace: operators - dependsOn: - - name: cert-manager - - name: victoria-metrics-operator-crds - interval: 1h - path: ./kubernetes/apps/infra/victoria-metrics-operator/app - prune: true - sourceRef: - kind: GitRepository - name: flux-system - wait: true diff --git a/kubernetes/apps/o11y/namespace.yaml b/kubernetes/apps/o11y/namespace.yaml deleted file mode 100644 index c4046e9..0000000 --- a/kubernetes/apps/o11y/namespace.yaml +++ /dev/null @@ -1,4 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - name: o11y diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/app/kustomization.yaml b/kubernetes/apps/o11y/victoria-metrics-stack/app/kustomization.yaml deleted file mode 100644 index 2a94956..0000000 --- a/kubernetes/apps/o11y/victoria-metrics-stack/app/kustomization.yaml +++ /dev/null @@ -1,9 +0,0 @@ ---- -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - ./vmsingle.yaml - - ./vmagent.yaml - - ./vmauth-external.yaml - - ./vmauth-internal.yaml - - ./vmusers/ diff --git a/kubernetes/apps/o11y/victoria-metrics-stack/ks.yaml b/kubernetes/apps/o11y/victoria-metrics-stack/ks.yaml deleted file mode 100644 index 89a427c..0000000 --- a/kubernetes/apps/o11y/victoria-metrics-stack/ks.yaml +++ /dev/null @@ -1,23 +0,0 @@ ---- -apiVersion: kustomize.toolkit.fluxcd.io/v1 -kind: Kustomization -metadata: - name: victoria-metrics-stack - namespace: flux-system -spec: - targetNamespace: o11y - dependsOn: - - name: victoria-metrics-operator - - name: openebs - interval: 1h - path: ./kubernetes/apps/o11y/victoria-metrics-stack/app - prune: true - sourceRef: - kind: GitRepository - name: flux-system - wait: true - postBuild: - substituteFrom: - - kind: ConfigMap - name: vm-zone-endpoints - optional: false diff --git a/kubernetes/apps/staging/cert-manager/cert-manager-webhook-ovh.yaml b/kubernetes/apps/staging/cert-manager/cert-manager-webhook-ovh.yaml new file mode 100644 index 0000000..9daf7c5 --- /dev/null +++ b/kubernetes/apps/staging/cert-manager/cert-manager-webhook-ovh.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: cert-manager-webhook-ovh + namespace: flux-system +spec: + dependsOn: + - name: cert-manager + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: cert-manager-webhook-ovh + namespace: cert-manager + interval: 1h + path: ./kubernetes/apps/base/cert-manager-webhook-ovh + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: cert-manager + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: cert-manager-webhook-ovh + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/aureq/charts/cert-manager-webhook-ovh + tag: 0.9.4 + target: + kind: OCIRepository + name: cert-manager-webhook-ovh diff --git a/kubernetes/apps/staging/cert-manager/cert-manager.yaml b/kubernetes/apps/staging/cert-manager/cert-manager.yaml new file mode 100644 index 0000000..d500068 --- /dev/null +++ b/kubernetes/apps/staging/cert-manager/cert-manager.yaml @@ -0,0 +1,33 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: cert-manager + namespace: flux-system +spec: + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: cert-manager + namespace: cert-manager + interval: 1h + path: ./kubernetes/apps/base/cert-manager + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: cert-manager + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: cert-manager + spec: + ref: + # renovate: datasource=docker depName=quay.io/jetstack/charts/cert-manager + tag: v1.19.4 + target: + kind: OCIRepository + name: cert-manager diff --git a/kubernetes/apps/staging/cert-manager/kustomization.yaml b/kubernetes/apps/staging/cert-manager/kustomization.yaml new file mode 100644 index 0000000..6cfb0e9 --- /dev/null +++ b/kubernetes/apps/staging/cert-manager/kustomization.yaml @@ -0,0 +1,8 @@ +--- +# yaml-language-server: $schema=https://json.schemastore.org/kustomization +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./cert-manager.yaml + - ./cert-manager-webhook-ovh.yaml + - ./namespace.yaml diff --git a/kubernetes/apps/staging/cert-manager/namespace.yaml b/kubernetes/apps/staging/cert-manager/namespace.yaml new file mode 100644 index 0000000..6e34540 --- /dev/null +++ b/kubernetes/apps/staging/cert-manager/namespace.yaml @@ -0,0 +1,7 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: cert-manager + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled diff --git a/kubernetes/apps/staging/default/echo.yaml b/kubernetes/apps/staging/default/echo.yaml new file mode 100644 index 0000000..1c549a9 --- /dev/null +++ b/kubernetes/apps/staging/default/echo.yaml @@ -0,0 +1,50 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: echo + namespace: flux-system +spec: + dependsOn: + - name: envoy-proxy + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: echo + namespace: default + interval: 1h + path: ./kubernetes/apps/base/echo + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: default + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: echo + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/bjw-s-labs/helm/app-template + tag: 4.6.2 + target: + kind: OCIRepository + name: echo + - patch: |- + apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + metadata: + name: echo + spec: + values: + route: + app: + hostnames: + - echo.staging.futostat.us + - echo.staging.futostatus.com + target: + kind: HelmRelease + name: echo diff --git a/kubernetes/apps/o11y/kustomization.yaml b/kubernetes/apps/staging/default/kustomization.yaml similarity index 72% rename from kubernetes/apps/o11y/kustomization.yaml rename to kubernetes/apps/staging/default/kustomization.yaml index d9fdcaa..420537d 100644 --- a/kubernetes/apps/o11y/kustomization.yaml +++ b/kubernetes/apps/staging/default/kustomization.yaml @@ -2,5 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - - ./victoria-metrics-stack/ks.yaml + - ./echo.yaml - ./namespace.yaml diff --git a/kubernetes/apps/staging/default/namespace.yaml b/kubernetes/apps/staging/default/namespace.yaml new file mode 100644 index 0000000..e3f8a49 --- /dev/null +++ b/kubernetes/apps/staging/default/namespace.yaml @@ -0,0 +1,7 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: default + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled diff --git a/kubernetes/apps/staging/envoy-system/envoy-gateway.yaml b/kubernetes/apps/staging/envoy-system/envoy-gateway.yaml new file mode 100644 index 0000000..10ad8a8 --- /dev/null +++ b/kubernetes/apps/staging/envoy-system/envoy-gateway.yaml @@ -0,0 +1,61 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: envoy-gateway + namespace: flux-system +spec: + dependsOn: + - name: cert-manager-webhook-ovh + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: envoy-gateway + namespace: envoy-system + interval: 1h + path: ./kubernetes/apps/base/envoy-gateway + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: envoy-system + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: envoy-gateway + spec: + ref: + # renovate: datasource=docker depName=mirror.gcr.io/envoyproxy/gateway-helm + tag: 1.7.0 + target: + kind: OCIRepository + name: envoy-gateway + - patch: |- + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: futostat-us + spec: + commonName: staging.futostat.us + dnsNames: + - staging.futostat.us + - "*.staging.futostat.us" + target: + kind: Certificate + name: futostat-us + - patch: |- + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: futostatus-com + spec: + commonName: staging.futostatus.com + dnsNames: + - staging.futostatus.com + - "*.staging.futostatus.com" + target: + kind: Certificate + name: futostatus-com diff --git a/kubernetes/apps/infra/cert-manager/ks.yaml b/kubernetes/apps/staging/envoy-system/envoy-proxy.yaml similarity index 56% rename from kubernetes/apps/infra/cert-manager/ks.yaml rename to kubernetes/apps/staging/envoy-system/envoy-proxy.yaml index 2a2a7b8..9c1c7bd 100644 --- a/kubernetes/apps/infra/cert-manager/ks.yaml +++ b/kubernetes/apps/staging/envoy-system/envoy-proxy.yaml @@ -2,14 +2,17 @@ apiVersion: kustomize.toolkit.fluxcd.io/v1 kind: Kustomization metadata: - name: cert-manager + name: envoy-proxy namespace: flux-system spec: - targetNamespace: network + dependsOn: + - name: envoy-gateway interval: 1h - path: ./kubernetes/apps/infra/cert-manager/app + path: ./kubernetes/apps/base/envoy-proxy prune: true sourceRef: kind: GitRepository name: flux-system + namespace: flux-system + targetNamespace: envoy-system wait: true diff --git a/kubernetes/apps/staging/envoy-system/kustomization.yaml b/kubernetes/apps/staging/envoy-system/kustomization.yaml new file mode 100644 index 0000000..248445b --- /dev/null +++ b/kubernetes/apps/staging/envoy-system/kustomization.yaml @@ -0,0 +1,8 @@ +--- +# yaml-language-server: $schema=https://json.schemastore.org/kustomization +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./envoy-gateway.yaml + - ./envoy-proxy.yaml + - ./namespace.yaml diff --git a/kubernetes/apps/staging/envoy-system/namespace.yaml b/kubernetes/apps/staging/envoy-system/namespace.yaml new file mode 100644 index 0000000..81e56d9 --- /dev/null +++ b/kubernetes/apps/staging/envoy-system/namespace.yaml @@ -0,0 +1,7 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: envoy-system + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled diff --git a/kubernetes/apps/staging/external-secrets/cluster-secret-store.yaml b/kubernetes/apps/staging/external-secrets/cluster-secret-store.yaml new file mode 100644 index 0000000..7ccb216 --- /dev/null +++ b/kubernetes/apps/staging/external-secrets/cluster-secret-store.yaml @@ -0,0 +1,33 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: cluster-secret-store + namespace: flux-system +spec: + dependsOn: + - name: external-secrets + - name: onepassword-connect + interval: 1h + path: ./kubernetes/apps/base/cluster-secret-store + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: external-secrets + wait: true + patches: + - patch: |- + apiVersion: external-secrets.io/v1 + kind: ClusterSecretStore + metadata: + name: onepassword-environment + spec: + provider: + onepassword: + vaults: + o11y_tf_staging: 1 + target: + kind: ClusterSecretStore + name: onepassword-environment diff --git a/kubernetes/apps/staging/external-secrets/external-secrets.yaml b/kubernetes/apps/staging/external-secrets/external-secrets.yaml new file mode 100644 index 0000000..97912ee --- /dev/null +++ b/kubernetes/apps/staging/external-secrets/external-secrets.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: external-secrets + namespace: flux-system +spec: + dependsOn: + - name: prometheus-operator-crds + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: external-secrets + namespace: external-secrets + interval: 1h + path: ./kubernetes/apps/base/external-secrets + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: external-secrets + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: external-secrets + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/external-secrets/charts/external-secrets + tag: 2.2.0 + target: + kind: OCIRepository + name: external-secrets diff --git a/kubernetes/apps/staging/external-secrets/kustomization.yaml b/kubernetes/apps/staging/external-secrets/kustomization.yaml new file mode 100644 index 0000000..c1a1128 --- /dev/null +++ b/kubernetes/apps/staging/external-secrets/kustomization.yaml @@ -0,0 +1,9 @@ +--- +# yaml-language-server: $schema=https://json.schemastore.org/kustomization +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./namespace.yaml + - ./external-secrets.yaml + - ./onepassword-connect.yaml + - ./cluster-secret-store.yaml diff --git a/kubernetes/apps/staging/external-secrets/namespace.yaml b/kubernetes/apps/staging/external-secrets/namespace.yaml new file mode 100644 index 0000000..96f2c64 --- /dev/null +++ b/kubernetes/apps/staging/external-secrets/namespace.yaml @@ -0,0 +1,7 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: external-secrets + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled diff --git a/kubernetes/apps/staging/external-secrets/onepassword-connect.yaml b/kubernetes/apps/staging/external-secrets/onepassword-connect.yaml new file mode 100644 index 0000000..8fc5283 --- /dev/null +++ b/kubernetes/apps/staging/external-secrets/onepassword-connect.yaml @@ -0,0 +1,36 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: onepassword-connect + namespace: flux-system +spec: + dependsOn: + - name: prometheus-operator-crds + - name: external-secrets + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: onepassword-connect + namespace: external-secrets + interval: 1h + path: ./kubernetes/apps/base/onepassword-connect + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: external-secrets + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: onepassword-connect + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/1password/connect + tag: 2.4.1 + target: + kind: OCIRepository + name: onepassword-connect diff --git a/kubernetes/apps/staging/o11y/grafana-operator.yaml b/kubernetes/apps/staging/o11y/grafana-operator.yaml new file mode 100644 index 0000000..e7cd2a4 --- /dev/null +++ b/kubernetes/apps/staging/o11y/grafana-operator.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: grafana-operator + namespace: flux-system +spec: + dependsOn: + - name: prometheus-operator-crds + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: grafana-operator + namespace: o11y + interval: 1h + path: ./kubernetes/apps/base/grafana-operator + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: o11y + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: grafana-operator + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/grafana/helm-charts/grafana-operator + tag: 5.22.2 + target: + kind: OCIRepository + name: grafana-operator diff --git a/kubernetes/apps/staging/o11y/grafana.yaml b/kubernetes/apps/staging/o11y/grafana.yaml new file mode 100644 index 0000000..e7cd361 --- /dev/null +++ b/kubernetes/apps/staging/o11y/grafana.yaml @@ -0,0 +1,34 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: grafana + namespace: flux-system +spec: + dependsOn: + - name: prometheus-operator-crds + - name: grafana-operator + - name: external-secrets + interval: 1h + path: ./kubernetes/apps/base/grafana + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: o11y + patches: + - patch: |- + apiVersion: grafana.integreatly.org/v1beta1 + kind: Grafana + metadata: + name: grafana + spec: + httpRoute: + spec: + hostnames: + - grafana.staging.futostat.us + - grafana.staging.futostatus.com + target: + kind: Grafana + name: grafana diff --git a/kubernetes/apps/staging/o11y/kustomization.yaml b/kubernetes/apps/staging/o11y/kustomization.yaml new file mode 100644 index 0000000..d34cff4 --- /dev/null +++ b/kubernetes/apps/staging/o11y/kustomization.yaml @@ -0,0 +1,12 @@ +--- +# yaml-language-server: $schema=https://json.schemastore.org/kustomization +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./grafana-operator.yaml + - ./grafana.yaml + - ./namespace.yaml + - ./prometheus-operator-crds.yaml + - ./victoria-metrics-operator-crds.yaml + - ./victoria-metrics-operator.yaml + - ./victoria-metrics.yaml diff --git a/kubernetes/apps/staging/o11y/namespace.yaml b/kubernetes/apps/staging/o11y/namespace.yaml new file mode 100644 index 0000000..2c7ac62 --- /dev/null +++ b/kubernetes/apps/staging/o11y/namespace.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: o11y + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled + labels: + pod-security.kubernetes.io/enforce: privileged diff --git a/kubernetes/apps/staging/o11y/prometheus-operator-crds.yaml b/kubernetes/apps/staging/o11y/prometheus-operator-crds.yaml new file mode 100644 index 0000000..dc3cceb --- /dev/null +++ b/kubernetes/apps/staging/o11y/prometheus-operator-crds.yaml @@ -0,0 +1,33 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: prometheus-operator-crds + namespace: flux-system +spec: + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: prometheus-operator-crds + namespace: o11y + interval: 1h + path: ./kubernetes/apps/base/prometheus-operator-crds + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: o11y + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: prometheus-operator-crds + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/prometheus-community/charts/prometheus-operator-crds + tag: 27.0.0 + target: + kind: OCIRepository + name: prometheus-operator-crds diff --git a/kubernetes/apps/staging/o11y/victoria-metrics-operator-crds.yaml b/kubernetes/apps/staging/o11y/victoria-metrics-operator-crds.yaml new file mode 100644 index 0000000..da2f952 --- /dev/null +++ b/kubernetes/apps/staging/o11y/victoria-metrics-operator-crds.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: victoria-metrics-operator-crds + namespace: flux-system +spec: + dependsOn: + - name: prometheus-operator-crds + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: victoria-metrics-operator-crds + namespace: o11y + interval: 1h + path: ./kubernetes/apps/base/victoria-metrics-operator-crds + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: o11y + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: victoria-metrics-operator-crds + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator-crds + tag: 0.6.1 + target: + kind: OCIRepository + name: victoria-metrics-operator-crds diff --git a/kubernetes/apps/staging/o11y/victoria-metrics-operator.yaml b/kubernetes/apps/staging/o11y/victoria-metrics-operator.yaml new file mode 100644 index 0000000..861fb2e --- /dev/null +++ b/kubernetes/apps/staging/o11y/victoria-metrics-operator.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: victoria-metrics-operator + namespace: flux-system +spec: + dependsOn: + - name: victoria-metrics-operator-crds + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: victoria-metrics-operator + namespace: o11y + interval: 1h + path: ./kubernetes/apps/base/victoria-metrics-operator + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: o11y + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: victoria-metrics-operator + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator + tag: 0.58.1 + target: + kind: OCIRepository + name: victoria-metrics-operator diff --git a/kubernetes/apps/staging/o11y/victoria-metrics.yaml b/kubernetes/apps/staging/o11y/victoria-metrics.yaml new file mode 100644 index 0000000..a707bab --- /dev/null +++ b/kubernetes/apps/staging/o11y/victoria-metrics.yaml @@ -0,0 +1,37 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: victoria-metrics + namespace: flux-system +spec: + dependsOn: + - name: openebs + - name: victoria-metrics-operator + interval: 1h + path: ./kubernetes/apps/base/victoria-metrics + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: o11y + wait: true + patches: + - patch: |- + apiVersion: gateway.networking.k8s.io/v1 + kind: HTTPRoute + metadata: + name: vmauth-external + spec: + hostnames: + - vmauth.staging.futostat.us + - vmauth.staging.futostatus.com + target: + kind: HTTPRoute + name: vmauth-external + postBuild: + substituteFrom: + - kind: ConfigMap + name: vm-zone-endpoints + optional: false diff --git a/kubernetes/apps/staging/openebs-system/kustomization.yaml b/kubernetes/apps/staging/openebs-system/kustomization.yaml new file mode 100644 index 0000000..089b429 --- /dev/null +++ b/kubernetes/apps/staging/openebs-system/kustomization.yaml @@ -0,0 +1,7 @@ +--- +# yaml-language-server: $schema=https://json.schemastore.org/kustomization +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ./namespace.yaml + - ./openebs.yaml diff --git a/kubernetes/apps/staging/openebs-system/namespace.yaml b/kubernetes/apps/staging/openebs-system/namespace.yaml new file mode 100644 index 0000000..e58366c --- /dev/null +++ b/kubernetes/apps/staging/openebs-system/namespace.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: openebs-system + labels: + pod-security.kubernetes.io/enforce: privileged + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled diff --git a/kubernetes/apps/staging/openebs-system/openebs.yaml b/kubernetes/apps/staging/openebs-system/openebs.yaml new file mode 100644 index 0000000..deeb2dc --- /dev/null +++ b/kubernetes/apps/staging/openebs-system/openebs.yaml @@ -0,0 +1,33 @@ +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: openebs + namespace: flux-system +spec: + healthChecks: + - apiVersion: helm.toolkit.fluxcd.io/v2 + kind: HelmRelease + name: openebs + namespace: openebs-system + interval: 1h + path: ./kubernetes/apps/base/openebs + prune: true + sourceRef: + kind: GitRepository + name: flux-system + namespace: flux-system + targetNamespace: openebs-system + patches: + - patch: |- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: openebs + spec: + ref: + # renovate: datasource=docker depName=ghcr.io/openebs/charts/openebs + tag: 4.4.0 + target: + kind: OCIRepository + name: openebs diff --git a/kubernetes/apps/infra/openebs/ks.yaml b/kubernetes/clusters/staging/apps.yaml similarity index 64% rename from kubernetes/apps/infra/openebs/ks.yaml rename to kubernetes/clusters/staging/apps.yaml index 3924bb2..5ba86eb 100644 --- a/kubernetes/apps/infra/openebs/ks.yaml +++ b/kubernetes/clusters/staging/apps.yaml @@ -1,15 +1,14 @@ ---- apiVersion: kustomize.toolkit.fluxcd.io/v1 kind: Kustomization metadata: - name: openebs + name: cluster-apps namespace: flux-system spec: - targetNamespace: storage interval: 1h - path: ./kubernetes/apps/infra/openebs/app + path: ./kubernetes/apps/staging prune: true sourceRef: kind: GitRepository name: flux-system - wait: true + namespace: flux-system + wait: false diff --git a/kubernetes/flux/cluster/ks.yaml b/kubernetes/flux/cluster/ks.yaml deleted file mode 100644 index 2189f32..0000000 --- a/kubernetes/flux/cluster/ks.yaml +++ /dev/null @@ -1,41 +0,0 @@ -apiVersion: kustomize.toolkit.fluxcd.io/v1 -kind: Kustomization -metadata: - name: cluster-apps - namespace: flux-system -spec: - interval: 1h - path: ./kubernetes/apps - prune: true - sourceRef: - kind: GitRepository - name: flux-system - namespace: flux-system - wait: false - patches: - - patch: |- - apiVersion: kustomize.toolkit.fluxcd.io/v1 - kind: Kustomization - metadata: - name: _ - spec: - patches: - - patch: |- - apiVersion: helm.toolkit.fluxcd.io/v2 - kind: HelmRelease - metadata: - name: _ - spec: - install: - crds: CreateReplace - upgrade: - cleanupOnFail: true - crds: CreateReplace - remediation: - retries: 2 - target: - group: helm.toolkit.fluxcd.io - kind: HelmRelease - target: - group: kustomize.toolkit.fluxcd.io - kind: Kustomization