Tilt Dev Loop

FractalOps now has a Tilt workflow for the real development path:

local workstation -> local image builder -> Helm dev release -> dev namespace -> browser smoke

Tilt is the local control-plane image build / dev-release loop for this repository’s own FractalOps API/worker/portal images. It is unrelated to project dev previews (those run as bare processes in the Daytona sandbox — see Dev Preview Plane).

Tilt is development mode only. Deployment mode stays on the GitOps, Helm, and Argo path.

The host running Tilt does not mutate the live k3s API directly. So the right shape here is not direct k8s_resource(...) or a remote daemon endpoint. Tilt drives local shell resources that:

build the runtime images locally with the container build toolchain
push dev images and cache through the Nexus build cache plane
upgrade dev-only Helm releases in the dev namespace
smoke the real public routes

Mode split

development mode: Tilt builds images and upgrades dev-only Helm releases
staging mode: builds staging-tagged images and cache scopes without mutating live workloads by default
deployment mode: Argo reconciles Helm-declared Git state

Never use Tilt to promote a release. Use it only for fast live-cluster iteration.

Requirements

tilt installed locally
- official install docs: https://docs.tilt.dev/install.html
a local container image build toolchain
helm
repo root at /home/devuser/work/fractalops

You do not need local kubectl for dev deploy mutation.

Start

make tilt-up

This will:

build the runtime image through the local Tilt image builder
- tag target defaults to ghcr.io/yamonco/...
- Tilt uses the GitOps machine GHCR credential automatically for dev pushes
- remote cache import/export are both off by default; this host cannot use the remote build cache efficiently enough to justify the extra complexity in the live dev loop
upgrade the runtime Helm release in the dev namespace
build and roll out the portal dev release the same way

Default resources:

api
portal

Optional resources:

make tilt-up TILT_ARGS=\"-- --worker --agent_server --e2e\"

Leave development mode

Stop the local Tilt session only:

make tilt-down

This stops the local dev loop. It does not mutate release/staging/prod apps.

Return fully to deployment mode and reassert Argo ownership:

make deploy-mode

make deploy-mode does three things:

stops any local Tilt session if Tilt is installed
asks Argo CD to sync the canonical FractalOps apps
waits for the canonical apps to return to Synced/Healthy

Resources

Tilt defines:

runtime
portal
optional portal-e2e-smoke
manual deploy-mode

Dev and Release Boundary

FractalOps runtime API/worker/portal apps use one deployment surface: Helm charts. Tilt uses the same charts for disposable dev namespaces, and Argo syncs the same chart paths for runtime environments.

Tilt must not call kubectl set image, patch Deployment env, or pause Argo for ordinary dev iteration.

Dev:
  Tilt -> local image build -> GHCR dev tag -> helm upgrade --install -> fractalops-dev

Release:
  Git/CI -> image build -> GHCR immutable tag -> Helm values image pin -> Argo sync

kubectl is reserved for diagnostics and bootstrap wrappers only. It is not the deployment mutation API for the dev or release loop.

Build plane and cache

All Tilt image builders load the same BuildPlane contract from:

python3 ops/infra/build_plane_env.py ops/infra/topology/lxc-pve-lab.yaml

The intended registry split is:

GHCR: final image registry only
Nexus docker-group :8083: docker.io + ghcr.io pull-through mirror
Nexus buildcache (docker-internal :8082): OCI build cache import/export only

Default cache scopes are stage and target scoped:

nexus.nexus.svc:8082/fractalops/buildcache/dev/runtime
nexus.nexus.svc:8082/fractalops/buildcache/dev/portal
nexus.nexus.svc:8082/fractalops/buildcache/dev/daytona-workspace
nexus.nexus.svc:8082/fractalops/buildcache/staging/runtime
nexus.nexus.svc:8082/fractalops/buildcache/staging/portal

This keeps GHCR from becoming the cache bucket and prevents runtime, portal, and workspace builds from sharing one invalidation-heavy cache tag.

Build telemetry

Build/cache/utilization telemetry belongs to the Build Plane, not the workflow test suite. Use the ubiquitous terms in Build Plane Observability before adding new event names or sinks.

Tilt/CI build runner -> Telemetry Marker -> OTLP HTTP logs -> OpenTelemetry Collector -> ClickHouse

Direct ClickHouse emission is optional for controlled runners with warehouse credentials:

FRACTALOPS_BUILD_EVENTS_ENABLED=true
FRACTALOPS_BUILD_EVENTS_TABLE=build_events
FRACTALOPS_CLICKHOUSE_HOST_PORT=clickhouse.clickhouse.svc.cluster.local:8123
FRACTALOPS_CLICKHOUSE_DATABASE=warehouse

The Tilt and GitHub build wrappers emit Build Plane Event records best-effort. The sink is non-blocking by default; set FRACTALOPS_BUILD_EVENTS_STRICT=true only when validating the telemetry path itself.

Autoscaling boundary

The live FractalOps control-plane workloads use Kubernetes HPA:

fractalops-api: min 2, max 6
fractalops-portal: min 1, max 4
fractalops-worker: min 1, max 4

Daytona project sandboxes are not HPA-managed deployments. They are runtime-owned workspace containers whose lifecycle follows the Daytona control plane and project lease policy; stopped sandboxes scale to zero or are reclaimed by the sandbox TTL.

Platform CI image builds use managed build workers with the Nexus registry/build cache. Parallel build demand should go through the shared build workers and the registry cache, not through new privileged daemon deployments.

Staging image channel

Use staging mode to publish staging-tagged runtime and portal images without touching the live cluster:

make staging-mode

Build one target:

make staging-mode TILT_ARGS="runtime --build-only"
make staging-mode TILT_ARGS="portal --build-only"

Apply staging-tagged images to the canonical live deployments only when you explicitly want a live-cluster check:

make staging-mode TILT_ARGS="runtime"
make staging-mode TILT_ARGS="portal"

Browser smoke

Tilt exposes a manual resource:

portal-e2e-smoke

It runs a real Playwright smoke against the port-forwarded portal and writes:

output/playwright/tilt-portal-smoke.png

You can trigger it from the Tilt UI or with:

make tilt-ci TILT_ARGS=\"-- --e2e\"

The smoke runs against the public route:

https://portal.yamon.io/projects

Why this shape

This workflow follows Tilt’s local-resource guidance, adapted to the real FractalOps boundary:

the host cannot directly talk to the live kube API
the host can build locally and reach the Nexus cache plane
Helm owns dev release mutation
Argo owns canonical release mutation
so development mode must be “local build + Helm dev release”, not “patch Argo-owned deployments”

Tilt models the build, rollout, smoke, and restore steps as local_resource pipelines.

Official references:

remote services and remote builds: https://docs.tilt.dev/local_vs_remote.html
local resources: https://docs.tilt.dev/local_resource.html
custom builds: https://docs.tilt.dev/custom_build.html

Notes

This loop is for fast developer iteration, not GitOps promotion.
Argo remains the deploy owner for the canonical live workloads on main.
make deploy-mode is the required way to hand ownership back to Argo.

Project Previews Are Not Tilt

This Tilt loop builds this repository’s own control-plane images. It is not the project preview path. Managed projects do not build or ship from Tilt or from inside the sandbox.

Project dev previews run as bare processes in the Daytona sandbox (vite / next / uvicorn bound to 0.0.0.0), exposed through the daytona-proxy signed preview URL via the dev-preview MCP. One public host per project (<slug>.monstore.io) serves the live dev preview when running, otherwise the project’s Dokploy delivery. See Dev Preview Plane.

Project dev preview:
  Daytona sandbox (no docker)
    -> bare dev server process
    -> daytona-proxy signed preview URL
    -> <slug>.monstore.io (live preview, else Dokploy delivery)

Rules:

Do not create a Daytona workspace per repo, branch, agent, or preview channel.
Do not deploy managed project previews into ad-hoc k3s namespaces from Tilt.
Dokploy is for persistent backing services only (databases, static-site / vercel-sim hosting, big-facility compose) — not dev servers.
PlaywrightGrid tests the project’s public preview host, not local Tilt routes.

This keeps the roles sharp:

Daytona: source-attached project workspace and bare dev preview process
daytona-proxy: signed preview URL surface
Dokploy: persistent backing services and static delivery
PlaywrightGrid: browser proof against the project’s public preview host
FractalOps Studio: control plane, lineage, and agent UX