Generate Env Cutover

Generate Env Cutover

Decision

generate_env is scheduled for removal.

FractalOps will not keep a compatibility layer for:

• generated .env bundles
• overlay-driven env mutation
• fallback alias chains such as pick(...)
• duplicated FRACTALOPS_* vs legacy connector env names

The replacement strategy is a hard cutover to structured configuration.

Why

The current env rendering path mixes:

• topology intent
• runtime secret references
• host bootstrap details
• application settings
• IdP flow policy

into one flat string map.

That shape caused real drift in auth-plane behavior:

• broker flow requirements lived in topology, env overlays, script defaults, and live runtime
• fixing live Keycloak did not guarantee repo SSOT correctness
• fallback env lookups allowed stale values to survive silently

For FractalOps, this is the wrong abstraction. Portal, auth, runtime, and operations contracts are structured systems and should be expressed as structured systems.

Target Architecture

1. Kubernetes runtime

Use:

• Argo CD
• Helm values or Kustomize overlays
• External Secrets Operator

Do not pass large application contracts through a generated .env.

2. Secret delivery

Use:

• OpenBao as secret authority
• External Secrets Operator for cluster materialization
• secret refs in manifests, not rendered plaintext env blobs

3. Host and LXC operations

Use:

• Ansible inventory
• Ansible roles
• explicit systemd unit/drop-in templates

Do not route host bootstrap through a repo-generated shared env file.

4. Application config

Application code should consume:

• typed config files
• mounted secrets
• explicit manifest fields

and fail fast when required fields are missing.

Do not keep env alias fallback chains.

5. Schema and validation

Use a typed schema layer for runtime intent.

Recommended:

• CUE for topology and deployment contracts

Minimum acceptable alternative:

• typed YAML plus strict Pydantic validation without env fallback compatibility

Replacement Map

Remove

• backend/src/fractalops/foundation/runtime_contracts/render_env.py
• fractalops runtime-contract render-env in backend/src/fractalops/cli.py
• tests/test_generate_env.py
• generated env overlays under ops/infra/env/

Replace with Kubernetes-native config

• bootstrap scripts in platform/k8s/ that currently read FRACTALOPS_* env
• platform/k8s/bootstrap_fractalops_apps.sh
• platform/k8s/bootstrap_argocd_identity_gitops.sh
• platform/k8s/bootstrap_daytona_penpot_argocd.sh
• platform/k8s/bootstrap_datahub.sh
• platform/k8s/bootstrap_headlamp.sh
• platform/k8s/reconcile_k3s_oidc.sh

These should read from:

• Helm values
• Kustomize config
• mounted secret refs

Replace with host-automation config

• ops/lxc/deploy_to_ct.sh
• ops/lxc/remediate_from_topology.sh
• scripts under ops/lxc/ that read shared env files

These should move to:

• Ansible variables
• role templates
• explicit per-service manifests

Replace app env fallback readers

• backend/src/fractalops/foundation/idp_runtime.py
• backend/src/fractalops/contexts/identity/infrastructure/providers/factory.py
• backend/src/fractalops/contexts/identity/infrastructure/providers/keycloak.py
• backend/src/fractalops/contexts/identity/infrastructure/providers/scim.py
• backend/src/fractalops/contexts/access/application/integration/native_ops_runtime_support.py
• backend/src/fractalops/contexts/access/application/connector_executor_factory.py (removed; do not restore)
• backend/src/fractalops/foundation/presentation/security_baseline.py

These must stop using:

• fallback env aliases
• pick(...)
• runtime string defaults for policy

Current High-Risk Drift Zones

These were already proven to be operationally dangerous:

• IdP broker first-login flow requirements
• Keycloak admin/public/base URL fallbacks
• secret-ref promotion from one env name to another
• host bootstrap scripts that reinterpret app config
• bootstrap scripts that reconstruct issuer URLs from partial env state

Hard Cutover Rules

1. No compatibility env aliases.
2. No dual write between manifest and rendered env.
3. No overlay-based secret mutation for steady-state runtime.
4. No fallback from new keys to old keys.
5. Missing required config must fail startup.
6. Drift checks must compare structured sources, not derived env text.

Cutover Sequence

Phase 1. Freeze and isolate

• Stop adding new keys to render_env.py
• Stop adding new tests to test_generate_env.py
• Keep only safety fixes until removal

Phase 2. Move auth and secret contracts first

• Move Keycloak, Pomerium, and OpenBao contracts to structured manifests
• Remove broker flow policy from env rendering
• Make secret refs first-class in deployment manifests

Phase 3. Move Kubernetes consumers

• Port platform/k8s/* bootstrap scripts to Helm/Kustomize inputs
• Remove shared env generation from cluster deployment steps

Phase 4. Move LXC and host consumers

• Replace ops/lxc/* env-driven reconciliation with Ansible variables and roles
• Keep repo topology as source input, but render directly to host-specific config files

Phase 5. Remove app fallback layer

• Replace env alias reads in app/runtime code with typed config inputs only
• Delete pick(...)-style compatibility resolution in runtime support paths

Phase 6. Delete legacy generator

• delete render_env.py
• delete runtime_contract_render_env CLI entry
• delete test_generate_env.py
• delete env overlay docs and samples that only exist for generated runtime contracts

Breaking Changes to Expect

• ops/infra/apply_ssot_profile.py must stop invoking runtime env rendering
• Makefile targets that depend on ops/infra/env/lab.secrets.env must change
• ops/lxc/deploy_to_ct.sh and ops/lxc/remediate_from_topology.sh will no longer accept generated env overlays as the primary control path
• app startup will reject missing structured config instead of silently falling back

Definition of Done

The cutover is complete only when all of the following are true:

• no deployment path requires render_env.py
• no bootstrap path requires ops/infra/env/*.env
• no runtime path depends on env fallback aliases
• secrets are materialized through OpenBao refs and target-native delivery
• Kubernetes config is expressed in Helm/Kustomize inputs
• host/LXC config is expressed in Ansible vars/templates
• test_generate_env.py is deleted

Immediate Next Execution

1. Remove apply_ssot_profile -> runtime_contract_render_env from the deployment path.
2. Move IdP and OpenBao settings into a structured deployment contract.
3. Replace ops/lxc/remediate_from_topology.sh env reads with manifest or Ansible vars.
4. Delete fallback alias logic in native_ops_runtime_support.py after consumers switch.