Test Execution Policy

Test Execution Policy

FractalOps test policy is: run less first, then parallelize, then reuse the environment.

Execution tiers

Tier	Required checks	Forbidden by default
Pull request	lint, format check where available, type/codegen checks, unit tests, affected integration tests	full integration, full E2E, performance tests
Main merge	full integration, contract tests	full E2E unless explicitly promoted
Release	full E2E, performance tests, smoke tests	unrelated branch-wide reruns

PR checks must be changed-file focused. Use ops/ci/run_focused_pr_gate.py before broad test suites. A ci:full label can expand unit/contract proof, but must not turn a PR into the release E2E lane.

The Contract Driven Stack workflow maps the tiers this way:

• PR: codegen guardrail plus focused changed-file gate.
• Main push: full integration followed by contract proof.
• Manual release dispatch: full E2E and smoke after the contract job; live performance benchmark only when run_performance is enabled.

Ordering

Fail fast in this order:

1. Lint
2. Format
3. Type or generated-contract check
4. Unit
5. Affected integration
6. Full integration
7. Contract
8. E2E
9. Performance

If a step fails, stop. Do not keep spending runner minutes on lower-signal checks.

Affected tests

Prefer changed-file selection over whole-project execution:

• Python: sibling _test.py / test_*.py files selected by ops/ci/run_focused_pr_gate.py.
• Architecture: Tach affected checks where module dependency impact is needed.
• Frontend: Nx/Turbo/Vite/Playwright affected targets in repository-owned frontend repos.
• Contract changes: direct contract tests plus compatibility checks.

Whole-suite execution needs a main, release, manual dispatch, or explicit incident reason.

Cache and reuse

Use cache before extra compute:

• Python: uv cache under .cache/uv.
• Node: package-manager cache keyed by lockfile.
• Docker/OCI: BuildKit registry cache with cache_from and cache_to.
• Build images: dependency layer first, application source after dependency install.
• Kubernetes test workloads: use Jobs, not long-lived Deployments.

Registry cache failures must degrade to cache misses, not failed builds. Durable cache authority is Nexus or GHCR; scratch PVCs are disposable.

Test data lifecycle

Test-created data must carry ownership metadata:

created_by
test_run_id
created_at
expires_at

Prefer template database clone/drop over repeated migration and seed. Long-lived cleanup belongs to a TTL reaper, not ad hoc test teardown. The reaper scope includes databases, object storage, queues, caches, blobs, and temporary files.

Performance budget

Lane	Target
Unit	30 seconds
PR integration	3 minutes
Full integration	10 minutes
E2E	20 minutes
Docker/OCI cache hit	1-2 minutes
Docker/OCI cache miss	under 5 minutes

The default engineering response to slow CI is not more runners. First reduce scope, then shard, then reuse warm environments.