Portal Per-Project Separation — Live Playwright Verification

Portal Per-Project Separation — Live Playwright Verification

This runbook drives a live browser traversal of the FractalOps Portal (and the other web UIs) to verify that per-project / per-tenant data separation actually works in the rendered UI, not just at the API layer.

Why it must run as a Studio/Armory persona (not a cron/loop session)

The browser plane is the PlaywrightGrid MCP at https://armory.yamon.io/servers/playwright/mcp (HTTP). It only connects when:

1. A PlaywrightGrid lease is issued — the MCP sends x-fractalops-lease-id + x-fractalops-browser-template-id headers, populated from FRACTALOPS_PLAYWRIGHTGRID_LEASE_ID / FRACTALOPS_PLAYWRIGHTGRID_BROWSER_TEMPLATE_ID. The grid keys a persistent browser session on the lease and applies the named browser template; the lease is a session-affinity key, not a secret.
2. The browser session is pre-authenticated to the Pomerium/CF-Access wall for project-scoped, signed-in traversal (Armory initializes named-agent browser auth before the agent turn — see the playwrightgrid-browser skill: do not do IdP/Keycloak login from the prompt). An unauthenticated browser still renders public pages; pre-auth is what makes the two-project separation checks meaningful.

For a full signed-in per-project separation run, launch a Studio persona (e.g. mercury for the Work Hub, compass for route/acceptance, cartographer for navigation/taxonomy) where Armory wires the named-agent browser auth. Escalation for a missing PlaywrightGrid registration: Atlas.

Bare (cron/loop/persona) sessions now get a browser too. The checked-in .mcp.json supplies non-empty defaults — x-fractalops-lease-id = agent-ai-swarm-system-yamon-io and x-fractalops-browser-template-id = fractalops-portal — and the Armory ContextForge registrar (platform/k8s/apps/armory/templates/contextforge-registrar-configmap.yaml) forwards those headers to the grid via passthrough_headers. So an agent that inherits the .mcp.json defaults binds one persisted browser without a per-session lease. It is not pre-authenticated (no CF-Access cookie), so it renders public pages only — use a Studio persona when a signed-in session is required. Under parallel personas a shared default lease shares one browser; set FRACTALOPS_PLAYWRIGHTGRID_LEASE_ID per launch to isolate.

Persona task spec (paste into the persona launch)

Traverse the live FractalOps Portal with your Armory-bound browser and verify per-PROJECT data separation. Your browser auth is initialized by Armory — do NOT perform any IdP/Keycloak login from this prompt. If mcp__playwright__* tools are absent, report playwright_unavailable and stop.

1. Snapshot the landing page to confirm the browser works + that you are authenticated.
2. Traverse the Portal sections the nav exposes: dashboard, projects list, a project detail/workspace, lineage (incl. feature-plane), launches, credentials, control-plane/studio, knowledge-curator, armory. Screenshot each.
3. Two-project separation check (the deliverable): pick TWO projects (A, B). For each project-scoped surface below, open it under project A, record the items shown, then switch to project B and confirm A’s items are ABSENT (and vice-versa). Any surface that shows the other project’s data, or does not change when you switch project, is a separation FAIL — capture the screenshot + the URL.
4. Report per surface: PASS (isolated) / FAIL (leak — describe) / N-A (global by design). Include the two project slugs, screenshots, and any broken/403/empty page.

Audit-derived separation checklist (focus the browser on these)

These came from the 2026-06-26 API-layer audit (see the portal-per-project-separation-leaks memory). The browser run should confirm in the rendered UI whether each isolates by project:

Surface (UI)	Route family	Expectation	Audit verdict to confirm/deny
Knowledge Curator (run/pin/restore)	/projects/{slug}/studio/knowledge-curator/*	only my project’s; mutate denied cross-project	S2 — was unguarded; fixed in PR #2015 (confirm 403 cross-project now)
People / Groups directory	/people, /groups	only my tenant’s people/groups	S3 LEAK — discards tenant (single Keycloak realm; pending decision)
Launch preview	/launches/{solution_id}/preview	only my project; not public	S4 — public, no tenant (confirm if intended)
Armory catalog / loadout preview	/armory/*	scoped or intentionally global	S1 — no auth/tenant (confirm if intended global)
Lineage project detail	/lineage/projects/{slug}	only members see it	LEAK-RISK — global can_observe_lineage, no membership check
Lineage proposals list	/lineage/proposals	only my accessible projects	LEAK-RISK — returns all tenant impacts
Control-plane studio / lineage-index / observability-feed	?project_slug=	project-bound	LEAK-RISK — query param, no project-bound check
Copilot	/copilot/* (page_path)	page_path validated vs my projects	LEAK-RISK — unvalidated
PASS (regression-confirm isolation holds)	packages, proposal review/approve, run/session control, jobs, project favorites, daytona org reconcile	isolated	already filter tenant+project / group_paths

Other web-UI surfaces to spot-check (same two-project lens where applicable)

fractalops (Portal), armory, grafana, daytona, devpod, cliproxy, supabase, langboard, phoenix, glitchtip, redpanda, npm, bentopdf, daytona-ssh, k3s — all under *.yamon.io behind Pomerium.

Pass criteria

• Every project-scoped surface shows ONLY the selected project’s data; switching projects changes the data; no surface leaks another project’s items.
• The S1–S4 / LEAK-RISK rows are explicitly resolved to FAIL (bug, file an issue) or N-A (intended global), with a screenshot.
• API-layer remediation tracked separately (PR #2015 = S2; S3/S1/S4/B5–B12 pending owner decision).

Verified execution findings (2026-06-26 — proven via in-cluster grid drive)

The live path was driven end-to-end from the in-cluster runner against the PlaywrightGrid MCP — these are confirmed, use them verbatim in the persona run:

• Portal URL = https://portal.yamon.io (NOT fractalops.yamon.io, which is NXDOMAIN and chrome-errors the browser). portal.yamon.io resolves via Cloudflare.
• Browser plane: PlaywrightGrid playwright-remote-browse MCP, in-cluster at http://playwrightgrid.browser.svc.cluster.local:3000/mcp (no CF/Pomerium wall internally; the external gateway armory.yamon.io/servers/playwright/mcp is CF-Access gated). Streamable-HTTP: thread the mcp-session-id response header on every call.
• Auth = the grid’s built-in flow, not agent-authored selectors: cli_auth_bootstrap({ url: "https://portal.yamon.io", username, password, flowToken: "fractalops_portal_microsoft_oidc", staySignedIn: "yes" }). Confirmed: it navigates portal.yamon.io → login.microsoftonline.com (Azure tenant 6b643feb-846c-4dc8-bffa-cc7b37be6137), fills username + password, handles the stay-signed-in prompt. The Portal IdP is Microsoft/Entra OIDC (auth.yamon.io callback), NOT Keycloak — use the Microsoft credential, not the grafana/Keycloak one.
• Credential requirement: a non-MFA Portal Microsoft account (Armory provisions this for a persona turn). A personal account with MFA cannot complete headlessly (auth_not_completed / microsoft_login_page after password_filled).
• Grid tools available: browser_navigate, browser_snapshot, browser_take_screenshot (saves server-side under /tmp/playwright-remote-browse/<sess>/screenshots/; retrieve via the grid’s MinIO playwrightgrid-minio.browser.svc.cluster.local:9000 or session artifacts), cli_goto/cli_click/cli_fill/cli_snapshot, session_card, session_orchestration_status, runbook_catalog.
• Reference harness: ops/ci/playwrightgrid_drive.mjs + .github/workflows/playwrightgrid-drive.yml drive the grid from the runner — usable as an alternative to a persona run IF given a non-MFA bot credential (set the username/password env; the auth + traversal is wired).

Net: the live verification is fully unblocked except for the Armory-provisioned non-MFA credential — which a Studio persona turn supplies. Launch the persona with the task spec above; the URL + auth flow + tool calls are now known-good.