Release convention

  • One PR to `dev` per repo per workstream. Sequenced commits stack inside that one PR.
  • Wave-scoped sprints span multiple repos; each contributes one PR.
  • QA gate 1 — feature-branch E2E before merge.
  • QA gate 2 — wave dev E2E across all repos after all PRs merge.
  • Atomic promote — only after gate 2 do we promote `dev` → `prod` across the whole wave together.
  • Final production E2E after prod deploy.
  • Doc sweep before merge — legacy content removed, canonical docs updated against the code, tech-debt captured.

See Product Architecture for the layered composition; AGENTS.md in the repo carries the same rule as agent-facing text.

current sprint

Wave 1.9 — Verify-Stack SoTA & Quality-Gate Economics (+ peer workstreams)

feat/engine-optimization-resilience (PR #18)

Adopt the clear wins from the classical-toolchain survey into the verify half of the write/harness stack and fix the correctness/perf items that cap it; peer workstreams (Mac Local Cubicle governance, KYM Orchestrator publisher, ABI Chat SSE, MCP schema sanitizer, domain ozzydev.com) stack in the same single PR per the one-PR rule.

Shipped

  • Wave 1.9 correctness-first (PR #18): reward composite gains the diff-content signal (emptyDiffOnCodeTask → forceZero in bandit.ts — the empty-diff arm can no longer outscore a correct implementation; regression pinned); write-mcp verify commands resolved per-repo (config.verify.verify_commands[<repo>] with basename normalization, rung3/rung4 no longer hardcoded null); format rung (runFormatStep, prettier --write on changed files post-apply) fixes the morph→prettier friction; direct-mode queue lane + persistent hints cache already landed on the branch (feat(engine) 1f8c697e).
  • Wave 1.9 SoTA verify: mutation-score rung 5 (rungs_1_5) with extractSurvivors() structured survivor feedback; console Stryker wiring (@stryker-mutator/core 9.6.1, stryker.conf.json, test:mutation); verify-ordering pareto module + runner + 14 fixtures EXECUTED — n=30 synthetic-but-measured corpus, verdict current-on-frontier (lint-first arm +2.6% err/s, below the 15% displacement threshold → no reorder); continue_on_failure ladder mode added for corpus accrual (never default). Baseline mutation score roadmap-data.ts 63.21%. Toolchain-latency bench: ESLint ~3.00s vs Oxlint ~0.87s — DECISION keep ESLint (Oxlint lints 0 .svelte files).
  • Research + roadmap: /research/classical-toolchain-ai-era landed (11-category catalog, DAG economics, transfer matrix, 26 sources); roadmap Wave 1.9 entry; _WORKSPACE-DOCS/ROADMAP-20260802-classical-tooling.md full deliverable + measurements addenda.
  • Peer workstreams (same PR): Mac Local Cubicle per-workspace governance D1 schema + API + console panel; KYM Orchestrator Card publisher UI + /api/kym/publish-card; ABI Chat SSE stream bridge with cubicle policy enforcement; MCP schema sanitizer (@nexartis/ozzydev-mcp-schema-sanitizer, Vertex anyOf fix) integrated into all four native MCPs; GitHub Actions SHA-pinned (checkout/pnpm/action-setup/setup-node) per the trivy-action hardening finding.

Exit criteria

  1. MET — reward composite diff-content signal pinned; empty-diff arm cannot outscore a correct implementation.
  2. MET — rung3/rung4 verify commands populated per repo; experiment verdicts reach full-suite signal.
  3. MET — format rung landed (prettier --write post-apply); zero prettier-on-write friction incidents.
  4. MET — mutation-score rung + Stryker wiring + baseline (63.21%); pareto EXECUTED (current-on-frontier verdict, n=30); continue_on_failure mode added for live-corpus accrual.
  5. MET — toolchain-latency bench recorded (keep ESLint decision); Actions SHA-pinned.
  6. pnpm run validate green across all touched packages.

Waves ahead

Wave 1.8 — ABI Chat DevFlow Integration & Living Agentic UI Assets

Integrate the ABI Chat SDK into the Console DevFlow wizard and build living, LLM-powered interactive assets. Embed dynamic agentic overlays, structured code review panels, approval gates for implementation wave dispatches, file change diffs, and KYM agent card stats drawers with streaming Ed25519 receipt verification.

Participating repos

  • nexartis-ozzydev Console /chat view + /devflow wizard integration with /api/abi-chat SSE endpoint; living agentic UI assets embedded across wizard steps (WorkflowLauncher, ApprovalGate, AgenticOverlay, CodeReviewPanel, FileChangesDiff); KYM agent roster expansion.
  • nexartis-abi-chat-sdk Two new LLM-driven agentic UI annotations: agentic_overlay and code_review_panel; Svelte 5 component suite, annotation parser extensions, session timeline integration, and test suite.

Exit criteria

  1. ABI Chat Assistant embedded and operational in /chat and /devflow.
  2. Living agentic assets (ApprovalGate, CodeReviewPanel, FileChangesDiff, AgenticOverlay) rendered in DevFlow wizard.
  3. agent-roster.svelte expanded with KYM card stats and Ed25519 live receipt stream drawer.
  4. pnpm run validate green across all packages.

Wave 4 — write-path latency seams (classic-unix-write-optimization)

Status: SHIPPED — 5 write-side performance/correctness seams landed at write-router@0.0.10-verify-sota (249/249 tests green) and preregistered wave-4 write-path benchmark executed.

Participating repos

  • nexartis-ozzydev 5 write-side seams in write-mcp: (1) lazy_diff original auto-hydration, (2) multi-file batch lazy_diff files[], (3) xxHash64 CAS gate, (4) atomic-rename write_mode default, (5) code-map span anchors. Wave-4 benchmark preregistered and executed (mcp/servers/ozzydev-native-write-mcp/experiment/PREREG-wave4-write-benchmark.md + results-wave4-2026-08-03-*.json).

Exit criteria

  1. MET — 5 write-side seams landed at write-router@0.0.10-verify-sota (249/249 tests green).
  2. MET — Wave-4 benchmark executed with verdict: 34.4x p50 batched-vs-sequential speedup on 32 files (1825ms vs 53ms), 0% corruption target-state pending re-run, SDI <= 4.27 tail stability.
  3. MET — Preregistration locked and results recorded under mcp/servers/ozzydev-native-write-mcp/experiment/.

Wave 1.9 — Verify-Stack SoTA & Quality-Gate Economics

Implementation sprint adopting the clear wins from the classical-toolchain survey (research log: /research/classical-toolchain-ai-era, landed 2026-08-02): bring the verify half of the write/harness stack up to 2026 SoTA and fix the correctness/perf items that cap it. Three threads: (1) correctness-first — the reward composite gains a diff-content signal (an empty-diff arm must never outscore a correct implementation), rung3/rung4 verify commands populated per repo so experiment verdicts reach full-suite signal, and morph-routed writes pass prettier via a format rung; (2) SoTA verify — a mutation-score rung for changed files (line coverage is gameable by AI-generated tests; mutation score is the truth-teller) with surviving mutants routed back to the subagent as targeted test-fix input, plus the verify-ordering pareto experiment (cheap-first DAG derived from the receipt corpus, preregistered per experimental-framework v0.3.0); (3) performance/architecture — invokeQueue per-mode concurrency lanes, cross-process hint memo under .ozzydev/, and an honest toolchain-latency bench (Oxlint prefilter vs ESLint on the validate gate) with an adopt/keep decision recorded. Factor-out verdicts from the survey: no external review-agent adoption (weak empirical signal, arXiv 2607.03316), no full Biome migration (Svelte plugin gap), no Nix/hermetic lift — deferred with reasons in the research log.

Participating repos

  • nexartis-ozzydev write-mcp: format rung for morph-routed writes (prettier), rung3/rung4 verify commands populated per repo, mutation-score rung for changed files (Stryker; survivors → structured feedback payload), invokeQueue per-mode concurrency lanes, verify-ordering pareto run on the receipt corpus. harness-mcp: reward composite v1-linear-normalised gains a diff_content_signal term (or gates on artifact != null && diff_stat.files > 0) with a regression test pinning the empty-diff case; verify ladder consumes the new rungs. search-mcp: cross-process hint memo under .ozzydev/ (per-process memo is a known latency ceiling). console: toolchain-latency bench script + read-back wiring for mutation/pareto results into /research/classical-toolchain-ai-era.
  • knowyourmodel-sdk Consumer check-in: receipt payload accommodation for the new verify/mutation signal fields (rung outcomes, mutation score, survivor refs); gaps become a paired PR.

Exit criteria

  1. MET — reward composite has a diff-content signal (or gate); the empty-diff arm can no longer outscore a correct implementation (regression test pinned).
  2. MET — rung3/rung4 verify commands populated for workspace repos; mode=experiment verdicts reach full-suite signal (no skipped_by_scope ceiling).
  3. MET — morph-routed writes pass prettier via the format rung; zero prettier-on-write incidents across a dogfood session.
  4. MET — mutation-score rung live for changed files with survivors routed back to the subagent; baseline mutation score recorded for critical paths.
  5. MET — verify-ordering pareto executed preregistered on the receipt corpus; gate ordering updated from the measured error-catch-per-ms frontier.
  6. MET — invokeQueue lanes + cross-process hint memo landed with measured latency deltas (p50/p95 before/after).
  7. MET — toolchain-latency bench documented with an adopt/keep decision recorded (Oxlint vs ESLint on the validate gate).
  8. pnpm run validate green across all touched packages.

Wave 1.7 — Context Engineering, Local D1 Code-Map Engine & Read Precision

Take a systems view on context management during requirement analysis and code exploration. Implement a local D1-backed Code-Map Engine (SCIP symbol graph + PageRank file/repo dependency maps + call-graph impact analysis + contract boundary maps) to minimize token footprint and maximize precision when reading code before editing. Eliminate redundant full-file reads and unguided grep sweeps by providing structured, high-relevance code maps directly to subagent dispatches.

Participating repos

  • nexartis-ozzydev New Code-Map Expert in ozzydev-native-search-mcp; local D1 schema for symbol/dependency graphs (migration 0009_code_maps); PageRank file-importance indexer in refresh-index-meta.sh; pre-pinpoint context builder that formats structural call graphs and contract boundaries for subagent dispatches; local-first SCIP/tree-sitter symbol graph indexer.
  • knowyourmodel-sdk Type definitions for CodeMapSnippet and ContractBoundaryContext; receipt payload extension for code-map provenance.

Exit criteria

  1. Local D1 Code-Map tables migrated and indexed across all workspace repos.
  2. Code-Map expert added to search-router with sub-second call-graph impact resolution.
  3. Pre-pinpoint subagent context builder verified reducing token usage by >40% during multi-file requirement analysis.
  4. pnpm run validate green across all packages.

Wave 0.9 — Parallel Waves launch surface

Launch real work from Ozzy Dev: a new console page with the ABI chat interface wired into the Parallel Waves harnesses, so an operator can initialize a waves instance against a target repo, launch a wave through the claude-code harness (v1 parity) or the OpenCode engine (v2, primary), and watch progress and gate decisions live. Linear (Nexartis Portfolio Domains workspace, our default) provides the ticket context work is launched from.

Participating repos

  • nexartis-ozzydev Console /waves page: ABI Chat SDK surface + wave board. Local server endpoints wrapping the v2 harness CLI contract — /api/waves (polls waves:status --json + per-instance state.json), launch endpoint spawning init:* and run_batch.py with engine selection, live progress from manifest.json + results/*.md, gate decisions from gate.json. Harness targets resolved from the Current Workspace (memory-systems patterns reused: home-scoped, read-only status surfaces; spawns gated).
  • nexartis-parallel-waves-agent-harness Primary harness (v2). Stabilize the console-facing contract: waves:status --json, state.json schema, per-job session_id/diff. Follow-ups from its own roadmap as needed: structured gate verdicts (json_schema), prompt_async/SSE for live job streaming.
  • nexartis-parallel-waves-archived Frozen v1 tree kept for historical reference only; claude-code engine parity now runs through the single-library harness via --engine claude-code.
  • nexartis-abi-chat-sdk Chat surface for the /waves page; provider bridging for launch/monitor intents (chat → harness commands with confirmation).
  • nexartis-multi-linear-workspace-mcp Linear connector wired for the Nexartis Portfolio Domains workspace (our default; the only connected workspace). Launching a wave can reference the Linear ticket that motivated it.

Exit criteria

  1. From /waves in the console: describe work in ABI chat → initialize a v2 instance against a chosen workspace repo → launch wave 0.
  2. Same launch path runs through BOTH engines: OpenCode (default) and claude-code (v1 parity) via engine selection.
  3. Live progress in the console: instance status board, per-job results, gate decision — no terminal required to observe.
  4. Implementation waves stay gated: --tool-profile implementation requires explicit console confirmation (mirrors --implementation-approved).
  5. Linear tickets from Nexartis Portfolio Domains attachable to a launch; the wave record references the ticket.
  6. pnpm run validate green in ozzydev and both harness repos.

Wave 1 — Ozzy Dev MVP

Clone-and-install on macOS; voice-driven Ozzy agent on the local-mac host with the opencode harness; Ozzy pulled from KYM and Cubicle-compiled locally; every completed turn writes a signed receipt back to KYM.

Participating repos

  • nexartis-ozzydev Vendored opencode under vendor/opencode/. install.sh + install.ps1. Ozzy Dev OpenCode plugins (kym-loader, cubicle-runner, kym-receipts, voice, discovery). MCP catalog with our chosen code-context stack (see Research §Working with Large Codebases). agents/ozzy.md + subagents. pnpm run publish:ozzy. Console MVP with real Dashboard, Agents, Identity views.
  • nexartis-cubicle-sdk Add local-mac, local-windows, local-linux to CubicleRuntimeHost. Bump schema to 0.3.0. New @nexartis/cubicle-adapter-local-mac. Update assertAgentCardCubicleCompatible.
  • nexartis-abi-chat-sdk Local-single-user preset example. Verify CustomProvider bridging for a local stdio harness (OpenCode).
  • knowyourmodel-ai Formalise the x-nexartis-runtime Agent Card extension. Confirm well-known agent-card + receipts endpoints accept host = local-mac.
  • knowyourmodel-sdk Type additions for x-nexartis-runtime. Verify publish flow for host = local-mac.

Exit criteria

  1. Bare Mac → clone → ./install.sh → ≤10 min → Ozzy responds to voice.
  2. Ozzy pulled from KYM at install; Cubicle compiled via new local-Mac adapter with signatures verified.
  3. Every completed turn signs and submits a receipt to KYM (visible in the console Identity view).
  4. Console shows Dashboard, Chat, Agents (browse KYM), MCP tools, Discovery, Identity, Doctor, Research.
  5. Ozzy drives desktop (via Touchpoint MCP) and browser (via Playwright MCP) on a spoken instruction.
  6. pnpm run validate green across all participating repos.

Wave 1.1 — Local persistence baseline

Status: SHIPPED — migrations 0001–0004 applied local+dev (0004 pending prod at promote time); schema + repository layer + doctor probe live. Add a first database boundary for Ozzy Dev before session/history features grow ad hoc. Use Cloudflare D1 as the portable schema target for deployable environments, with a local SQLite-compatible dev database for the on-machine console. Secrets stay in the OS keychain; D1 stores only product state, audit metadata, receipts, and non-secret references.

Participating repos

  • nexartis-ozzydev Define console/db/schema.sql and migration flow for workspaces, sessions, turns, tool calls, receipts, agent installs, wave launches, and memory-system telemetry. Add a typed server-side repository layer and a doctor probe that reports database readiness without exposing row contents.
  • nexartis-abi-chat-sdk Confirm ABI Chat session/turn abstractions map cleanly onto the Ozzy Dev persistence schema. Keep ABI Chat storage adapter interfaces runtime-neutral; Ozzy owns the D1/SQLite implementation.
  • knowyourmodel-sdk Verify signed receipt payloads have stable IDs and timestamps suitable for durable local indexing before submission to KYM.

Exit criteria

  1. pnpm run validate green with schema tests and repository tests.
  2. First migration creates the D1-compatible schema and is replayable against local SQLite.
  3. No provider keys, OAuth tokens, API keys, private keys, or raw secrets are stored in the database.
  4. Wave 0.9 launch records and Wave 1 chat/session records have explicit tables instead of in-memory-only state.

Wave 1.1b — Sentinel cloud auth: landing page + admin-only console

Status: SHIPPED — Sentinel gate live on workers runtimes; remaining exit: non-admin refusal E2E awaits the operator-provisioned non-admin test account. Replace the blanket cloud_gate_locked shortcut with real authentication via @nexartis/sentinel-sdk (the documented acceptance gate for a cloud release). Workers runtimes get a public landing page at /, the full Sentinel auth surface (login, register, magic-link, verify), and admin-only access to every other page and API — an authenticated non-admin is refused loudly. The local runtime is completely unaffected: it is the machine trust boundary, and agent tooling (telemetry-emit, doctor, memory admin) depends on ungated 127.0.0.1:5173.

Participating repos

  • nexartis-ozzydev Install @nexartis/sentinel-sdk; hooks.server.ts sequences the Sentinel handle (workers runtimes only) with an admin-role authorization wrap per the nexartis-remote-control reference; public landing page at / on workers; auth routes + views styled with the console design system (tokens.css, ui primitives — brownfield rules); SENTINEL_API_URL / VITE_BASE_URL vars per env in wrangler.jsonc; cloud_gate_locked replaced by classified admin_auth_required; origin guard adapted so /api/auth works on workers without weakening the local loopback protections.
  • sentinel-sdk Consumer check-in only: ozzydev pins the latest published version; report any contract gaps (role-claim shape, Workers platform.env resolution) back for a paired PR if found.

Exit criteria

  1. Cloud dev deploy: / renders the landing page publicly; every other page 302s to /auth; every other API returns classified 401 admin_auth_required (cache-busted live verification).
  2. Operator can complete the full magic-link login on the dev deploy and, with the admin role, reach all console pages; a non-admin account is refused with a loud 403 surface.
  3. Local runtime behavior unchanged: no auth, telemetry-emit + doctor + all agent APIs work on 127.0.0.1:5173; validate green.
  4. No prerendering reintroduced (gate integrity); security headers verified on workers responses.

Wave 1.2 — Telemetry substrate v2 (migration 0003)

Close the second half of the measurement substrate on top of the shipped 0002 tables. Add signed KYM receipts joined to runs, the evaluator pool + dual (agentic + human) rubrics that turn runs into ranked outcomes, pre-aggregated daily rollups per (category, tool, day) so dashboards do not scan raw events, and the search-router hint store (moe-search-router persistence surface — per-question-class expert cache with hit/miss telemetry). Status: SHIPPED — migration 0003 (console/drizzle/0003_telemetry_v2.sql) landed run_receipts / run_receipt_payloads, rubricsrubric_itemsevaluationsevaluation_scores, event_rollups_daily, and search_router_hints; /api/receipts is live (ADR-009 write + read); the moe-search-router consumes the hint store in wave-1-5.

Participating repos

  • nexartis-ozzydev Migration 0003 schema: run_receipts (signed KYM receipt refs joined to experiment_runs), rubrics + evaluations (evaluator pool + dual scoring), event_rollups_daily (materialised (category, tool, day) → p50/p95/error_rate/tokens/dollars), search_router_hints (question-class → expert-set + hit/miss counters). New ingest / read routes; extend the doctor slice; wire read-backs into the affected research pages.
  • knowyourmodel-sdk Receipt payload shape check-in — stable IDs, canonical signing surface, per-run reference format that 0003 can index against without leaking secret material.

Exit criteria

  1. 0003 migration replays cleanly on all three environments; pnpm run validate green.
  2. GET /api/experiments?paper_slug= now returns receipts + evaluations alongside runs where present.
  3. Daily rollup job populates event_rollups_daily; dashboards read from rollups, never scan tool_benchmark_events directly.
  4. moe-search-router MCP reads / writes search_router_hints in a bounded round-trip; hit/miss counters visible on the router receipt.
  5. /research/telemetry-observability v0.2 documents the 0003 additions and marks the deferred list closed.

Wave 1.3 — ozzydev-native-search-mcp (research + reference design)

Research-first design for the MoE search-router as an Ozzy-Dev-native MCP: one question in, one fused answer out; classifies, fans out to the relevant search experts in parallel, fuses with RRF + provenance, emits a receipt of who fired. No implementation lands in this wave — the exit is a locked reference design plus a benchmarking plan against the shipped tools. Depends on D4 (chunkhound pause / compaction release), D5 (Serena HTTP transport hardening), D15 (moe-search-router paper), and D16 (search-router hint store shape from Wave 1.2). Status: implementation shipped 2026-07-13 (commits 5610805 / e7c1a86); hardening + benchmark moved to wave-1-5-search-hardening.

Participating repos

  • nexartis-ozzydev Reference-design doc + ADR: expert pool, classifier contract, fusion algorithm (RRF with per-expert provenance tiers), receipt schema, hint-store integration, bounded-latency budget per class. Benchmark plan wired to tool_benchmark_events (category="search") and read-back through /research/moe-search-router via paper_slug. No new MCP process ships yet.

Exit criteria

  1. ADR merged; reference-design doc lives under docs/DECISIONS/ and is linked from /research/moe-search-router.
  2. Benchmark plan approved: fixture query set, gold labels source, precision@k / latency-p95 / cost KPIs against the shipped tools.
  3. D4/D5/D15/D16 dependencies verified as met (or blocked with a written escape-hatch).
  4. No implementation code lands; wave exit is design + benchmark plan only.

Wave 1.4 — ozzydev-native-write-mcp (research + reference design)

Research-first design for a native write MCP that routes every edit through the a–h rubric (D8): single small edit, scattered edits in one large file, mechanical multi-file sweep, large partial rewrite, new file, interdependent multi-file refactor, docs/prose, generated data. Verify-column-first: every write emits a tool_benchmark_events row with the verify JSON populated (typecheck / lint / test result). Depends on D8 (write rubric), D15 (research substrate reads), and the Cloudflare Code Mode pattern for scripts-as-tools composition. Status: implementation started 2026-07-13 (W6, commit 2197a28) — shipped @nexartis/ozzydev-native-write-mcp v0.0.1 under mcp/servers/ozzydev-native-write-mcp/ with rubric router (a)–(h) + auto inference, direct mode for native_edit/whole_file/lazy_diff payloads, git-diff scope gate + rollback, verify ladder rungs 1–2 executable (rungs 3–4 plumbed skipped_by_scope), Ed25519 receipts (ADR-009), and Morph + Relace fast-apply adapters (deadline tests, distinct 401 / timeout / malformed classification). MILESTONE COMPLETE 2026-07-16: the r2→r5 bakeoff program served as the pre-enable gate and every criterion passed — five measured rounds (r2 20-task synthetic → r5 31 REAL production edits incl. a CommitPackFT external slice), deterministic splice.v1 engine shipped + wired into the router as a splice-first preflight ladder (74.2% of real edits resolve deterministically at $0/0ms; classified splice_* failures hand off loud to the vendor arm), fast_apply_engine=morph on reconciled artifact evidence (Morph won every decision round r3–r5, zero refusals; write-router@0.0.4), and registry enabled: true (kilo.json activates on next session restart). Combined system on the r5 corpus: vendor-solo accuracy at ~74% lower median latency and ~73% lower vendor cost. Deferred loudly (classified errors, not silent): multiplex/experiment MCP modes, unified_diff/ast_rule/codemod_transform/symbol_op payload kinds. Next rungs queued: tree-sitter scope-anchoring (ambiguous-anchor class = 16% of r5), Osmosis-Apply-1.7B local arm. Measured results on /research/write-strategies v0.8; decision history in ADR-008.

Participating repos

  • nexartis-ozzydev Reference-design doc + ADR: per-category primary / fallback tool bindings, verify ladder mapping (typecheck → lint/AST → targeted test → full suite), event emission contract with the verify JSON schema, Code-Mode-composed scripts as the escape hatch for category (h). Benchmark plan wired to tool_benchmark_events (category="write") and read-back through /research/write-strategies via paper_slug. No new MCP process ships yet.

Exit criteria

  1. ADR merged; reference-design doc lives under docs/DECISIONS/ and is linked from /research/write-strategies.
  2. Verify JSON schema locked; sample rows validated against the 0002 tool_benchmark_events shape.
  3. Benchmark plan approved: per-category fixture set, accuracy (edit applies + validate-green) and latency-p95 KPIs, false-positive rate on verify.
  4. D8 rubric + D15 research substrate + Code Mode composition dependencies confirmed; no implementation code lands.

Wave 1.4b — ozzydev-native-harness-mcp (ADR-013 Native Harness Router)

Third native MCP alongside search + write: harness_run accepts a HarnessRunRequest (intent + task_class + target repo/paths + verify ladder + budget + containment + mode + optional write_payload), routes it to an engine adapter, sets up containment, spawns the engine, verifies the output through the write-router-owned verify ladder + rollback, merges routed unified_diff writes back, and emits ONE ADR-009 signed receipt per production invocation. Phases 0–2 LANDED (this PR, feat/harness-router → #12): engine-adapter seam + offline bench (bench/run.mts fixture-replay + optional live claude); MCP serve subcommand + migration 0005 (harnesses, harness_runs, harness_routing_decisions, harness_run_rewards + run_receipts category widened to search|write|harness) + /api/harness-runs ingest/read; real ephemeral git worktree containment (src/containment.ts seam), routed unified_diff merge-back via git apply, per-repo verify commands (registry + config), optional write_payload short-circuit through the in-proc write-router, capability-mismatch = LOUD (harness_containment_setup_failed / _teardown_failed / harness_merge_back_failed / harness_verify_config_missing joining the Phase-1 taxonomy). Validate gate 944→947 tests green at wave close. Phase 3 = ACP adapter (generic engine-adapter seam proven when a SECOND transport family lands; auggie + gemini-cli are the two proving instances configured as data, never as type/seam identifiers). Phase 4 LANDED 2026-07-31 on feat/harness-phase4-bakeoff (PR #13): bakeoff harness (bench/bakeoff.mts + shared bench/lib.mts) publishing to tool_benchmark_events with paper_slug='harness-bandit-experiments' and experiment slug harness-bandit-phase4-bakeoff. Phases 5–7 queued: bandit picker inferring task_class='auto' and populating decision.alternates_considered; mode='multiplex' and mode='experiment' (parallel arms, arm-scoped worktrees, cross-arm receipts); worktree+sandbox and container containment classes (namespaces + network-scope + syscall filter). Companion research /research/harness-interop (durable prose) and /research/harness-bandit-experiments (measurement substrate).

Participating repos

  • nexartis-ozzydev New package mcp/servers/ozzydev-native-harness-mcp/ shipped in feat/harness-router (PR #12 open to dev). Engine-adapter seam under src/adapters/ (CliEngineAdapter + per-engine profile drops under src/adapters/cli/<slug>.ts; cli.claude-code live, cli.codex fixture-replay). Router (src/router.ts) implements the Phase-2 rubric deterministically: row 1 (edit + write_payload present → in-proc write-router with containment='inproc'); rows 2–5 (refactor|feature|debug|test-authoring|docs|research|orchestrate → CLI engine in a resolved-containment worktree); row 6 (mode='multiplex|experiment' → loud harness_request_invalid). Adds task_class='orchestrate' to TASK_CLASSES + seed capability lists in config.json + mcp/registry.json cli.claude-code row. Containment seam (src/containment.ts): ephemeral git worktree add --detach <path> HEAD inside the workspace parent; teardown = git worktree remove --force; routed unified_diff merged back via write-router unified_diff payload; per-repo verify commands live in harness_verify_commands (registry + config). Migration console/drizzle/0005_harness_router.sql adds four hand-authored tables + widens run_receipts.category. Bench harness bench/run.mts upserts an experiment for paper_slug='harness-bandit-experiments' (Phase 4 substrate) and consumes recorded fixtures under bench/fixtures/. Read/write endpoints in console/src/lib/server/harness-runs.ts + /api/harness-runs/+server.ts. ADR-013 documents the contract, engine-adapter seam, routing rubric, containment classes, receipts + D1 tables, phase plan, cost-source policy (EXCLUDE estimated in Phases 0–2; cost-silent engines emit explicit null with cost_source='engine_unreported'), and the workspace-§2 boundary (no vendor name in a type/seam identifier — adapter ids are 'acp' | 'cli.<slug>', vendor endpoint/auth/model/parser live in profile files + registry data).

Exit criteria

  1. MET — Phases 0–2 landed on feat/harness-router (PR #12 open to dev); engine-adapter seam + offline bench + MCP serve + migration 0005 + /api/harness-runs + real worktree containment + routed unified_diff merge-back + per-repo verify commands all shipped.
  2. MET — pnpm run validate green at wave close: 947 passed / 2 skipped across the workspace (test:bin 12, receipts 28, index-meta 25, console 387, search-mcp 294+2 skipped, write-mcp 159, harness-mcp 42; may grow slightly with the routing tests added by ADR-013 pre-review corrections).
  3. MET — ADR-013 merged under docs/DECISIONS/ADR-013-native-harness-router.md; cross-linked from /research/harness-interop (durable prose companion) and /research/harness-bandit-experiments (measurement substrate).
  4. PLANNED — Phase 3: ACP adapter class alongside CliEngineAdapter implementing the same EngineAdapter interface; seam declared _proven generic_ when a second transport family runs through it. Two proving vendor instances configured as data (registry rows + profile files), never as new type/enum members: acp.auggie and acp.gemini-cli.
  5. MET — Phase 4 bakeoff harness LANDED 2026-07-31 on feat/harness-phase4-bakeoff (PR #13): new bench/bakeoff.mts + shared bench/lib.mts (extracted spawn/scratch/diff/experiment-upsert helpers plus pure rotateArms); registry-driven --arms <slug,…> required (no default-all); corpus-declared per-task verify executed via write-router runLadder with {bench}/{ws} placeholders; null-decision semantics (decision:null always; round null when N=1; --round required only for N≥2 EMIT runs; wave_slug via --wave, never hardcoded — the r5b defect class does not recur); bench_fixture added to mcp/registry.json harnesses[] rows (data field); publishes to tool_benchmark_events with paper_slug='harness-bandit-experiments', experiment slug harness-bandit-phase4-bakeoff (category 'harness-bandit'); read-back via GET /api/experiments?paper_slug=harness-bandit-experiments. bench/run.mts stays frozen Phase-0 evidence (imports lib helpers; hardcoded round/wave preserved). Selftests: test/bakeoff.test.ts covers happy path + N=1 + refusals + Phase-0 regression + rotateArms unit; hermetic (no network, no keys, EMIT off).
  6. MET — Phase 5 bandit picker LANDED 2026-07-31 on feat/harness-phase4-bakeoff (PR #13, wave-3a): Thompson Sampling with per-(engine × task_class) Beta–Bernoulli posteriors (Marsaglia–Tsang gamma + Beta-via-gamma-ratio in src/bandit.ts, node stdlib only, no new deps); deterministic named-rule classifier for task_class='auto' in config.classifier_rules[] (no LLM, first-match wins, no-match = LOUD harness_request_invalid naming rules_considered); reward composite v1-linear-normalised (weights {verify:1.0, cost:0.5, latency:0.25}) with cohort anchors (cohort / not_observable per OPERATOR DECISION B / degenerate_cohort); cold-start uniform Beta(1,1) — exploration emerges, no rubric-to-bandit mode switch. Bandit picks ENGINE only; containment stays the deterministic 'auto'-resolver. decision.alternates_considered populated per schema (every non-chosen live arm {score:<sample>, skipped:'lower_sample'}; every excluded row {score:0, skipped:'fixture_replay_provenance'|'not_live'|'class_unsupported'}). Declared-class runs also compute + ingest rewards from Phase 5 on — posteriors need signal from ALL production runs. POST /api/harness-runs accepts optional reward object; atomic 3-row batch (run + decision + reward); partial reward = LOUD 400. GET /api/harness-runs?include=rewards left-joins the reward row. config_version bumps to harness-router@0.0.4-phase5. harness-mcp tests 82 → 115; workspace validate 947 → 1127 (2 skipped) green.
  7. MET — Phase 6 mode='multiplex' | 'experiment' LANDED 2026-08-01 s20 on feat/harness-phase4-bakeoff (PR #13): console 53fa6fe (migration 0008 harness_multiplexmultiplex_group_id TEXT NULL on harness_runs + harness_routing_decisions, error_class, outcome CHECK widened + 'receipt_emission_failed', v3 chain-rebuild, applied to shared remote D1) + harness-mcp bca7071 (config_version=harness-router@0.0.6-phase6) + write-mcp 4a970d8 (co-graduated, config_version=write-router@0.0.9-phase6, arm shape {strategy:{primary,fallback},slug?}, scratch worktrees under <repo>/.worktrees-write-mcp/<group>-<slug>/). Wire shape (both routers): receipt_id:null + multiplex_group_id + arms[] (min 2, cap 8 default + hard max 8) + winner/selected null (compare-never-merge). Bandit multi-play SHIPS (arms:string[] OR {bandit:{k, task_class?}} D-union). Experiment = multiplex + paper_slug (caller-provided; default harness-bandit-experiments harness / write-strategies write) + forced rungs_1_4 + optional seed. Per-arm receipts with variant_id=<arm-slug> + detail.multiplex_group_id; per-arm loud failure with request-level partial success (≥1 receipt = success); pre-flight failure = LOUD harness_multiplex_group_failed / write_multiplex_group_failed. LIVE evidence: harness group 043f7f75-e812-4c12-ac7c-e974bce4c374 (arms session.opencode rc-06FVY1VNPKRN1988J6VFM2FXSG + cli.claude-code rc-06FVY1VNQK46V9FM44R00XRAPR); write group 65ffb30b-42b4-43b1-98e4-4112d77fa350 (arms splice rc-06FVY1F0DNHHK4KH8SYDR3CK94 + udiff rc-06FVY1F0D9ZMPAPN7T8SMHBVTW).
  8. PLANNED — Phase 7 worktree+sandbox + container containment classes (namespaces + network-scope enforcement + syscall filter; declared in CONTAINMENT_CLASSES).

Wave 1.6 — custom subagent runtime (own dispatch, KYM-defined roles, console model UX)

COMPLETE (s19, 2026-08-01 — evidence rc-06FVSZ20AW1Y0H87YGJ2S4Z900): 1.6a session engine-adapter family landed (third transport family behind the EngineAdapter seam, generic SessionEngineAdapter + session.opencode profile, PERMISSION_PROFILES verbatim, finalize() teardown wired to router shutdown; ozzydev 0b78f6e). 1.6b dispatch-time KYM role→card resolution landed (kym-resolver.ts — no cache, LOUD classified harness_registry_invalid, precedence request-over-card, tuple pinned into receipt detail + harness_routing_decisions.inputs; ozzydev 177f8b2). KYM side landed: harnessDefaults joins the agent-card runtime-extension allow-list (ADDITIVE-ONLY) + admin-only /api/sync/seed-harness-roles seeding implementer/verifier/validator/researcher cards (knowyourmodel-ai 1ae883f; operator applies the seed per env after deploy). 1.6c console /harness model-picker landed with EDIT capability wired (KYM-admin secret provisioned; per-role model + tool_profile + permissions overrides POST through the admin surface; loud classified refusal on missing secret). 1.6d ADR-013 amended (session family + role resolution). Live dogfood COMPLETE — orchestrator dispatched a real implementation wave through the session adapter end-to-end (receipt rc-06FVSZ20AW1Y0H87YGJ2S4Z900). DIRECTION RULED (operator, 2026-08-01): the orchestrator dispatches OUR OWN subagents instead of Kilo built-ins. Root-cause evidence (s17 probes): explore subagents are MCP-blind, general subagents carry the full toolset — but only OUR runtime gives per-dispatch model selection, mechanical permission profiles, per-role worktree isolation, and end-to-end signed receipts under our control. Substrate: port the proven long-lived-session engine pattern from nexartis-parallel-waves-agent-harness (scripts/engines/opencode.py — serve-per-worktree + session-per-job, ~200-line TS port) into ozzydev-native-harness-mcp as a new GENERIC engine adapter behind the existing EngineAdapter seam (§2 protocol-not-vendor: the adapter id is a slug, never a type). Request schema gains agent_role + model + tool_profile. Role→model→tool-profile config lives on KYM Agent Cards (x-nexartis-runtime.harnessDefaults) — KYM is the source of truth for who our agents are (implements the outstanding wave-harness-kym-split boundary ADR); the harness resolves role→card at dispatch; per-dispatch overrides ride in the request. Console gains a model-picker UX (role cards + effective model per dispatch, auditable via cost.model on every run receipt). INTERIM (already landed): dispatches needing routed tools use Kilo general subagents; explore is MCP-free recon only.

Participating repos

  • nexartis-ozzydev Port the session engine adapter into mcp/servers/ozzydev-native-harness-mcp/src/adapters/ (generic slug id; mechanical permission profiles ported from engines/__init__.py PERMISSION_PROFILES — git push*/gh pr create*/pnpm publish*/wrangler deploy* deny-listed mechanically, replacing prose enforcement). Schema: agent_role? + model? + tool_profile? on harnessRunRequestSchema. Role→card resolver (KYM SDK getAgentCard + x-nexartis-runtime.harnessDefaults sanction + loud harness_registry_invalid on missing/malformed). Console model-picker UX surface. Dogfood: orchestrator waves dispatched end-to-end through the new adapter with receipts.
  • knowyourmodel-ai Agent Cards for the first harness roles (orchestrator-subagent roles: implementer, verifier, validator, researcher) carrying x-nexartis-runtime.harnessDefaults{model, tool_profile, permissions} — ADDITIVE-ONLY per the KYM compatibility policy. Cards validated against the A2A agent-card surface already live.
  • knowyourmodel-sdk Consumer surface check: getAgentCard(id) + receipt submission keyed by (role, model, wave) — gaps become a paired PR.
  • nexartis-parallel-waves-agent-harness Reference implementation for the port (opencode serve + session HTTP dance, model-alias resolution, permission profiles). No code changes — it remains the waves runner; the adapter pattern is what migrates.

Exit criteria

  1. MET — Orchestrator dispatched a real implementation wave through the new session adapter end-to-end (fixture E2E green + LIVE dogfood green; evidence rc-06FVSZ20AW1Y0H87YGJ2S4Z900).
  2. MET — Role→model config viewable AND editable from console /harness (KYM card view + effective-model resolution from D1 + kym-admin edit wiring shipped: per-role model + tool_profile + permissions overrides POST through the admin surface, loud classified refusal on missing secret). Per-dispatch model override rides the request and lands on the run receipt (1.6b tuple pinned).
  3. MET — Mechanical permission profiles enforced by the runtime: OPENCODE_PERMISSION env serialization of PERMISSION_PROFILES (verbatim substrate port; card overrides merge after base, last-match-wins).
  4. MET (code) — KYM Agent Cards for the first roles landed via seed endpoint + allow-list; missing/malformed harnessDefaults = loud classified refusal at both KYM write-time and harness dispatch-time. Resolvable-after-apply per env.
  5. MET — pnpm run validate EXIT 0 in every touched repo (ozzydev 1230 tests; knowyourmodel-ai 664).

Wave 1.5 — search-MCP hardening & benchmark

Turn the shipped ozzydev-native-search-mcp (commits 5610805 / e7c1a86, 2026-07-13) into a measured artefact. Status: MEASURED (commits 1c2a9ca / 9fe6c03 / 448fc53 / b9547ba, 2026-07-13). Fixture set v1.1.0 (36 scored across 6 intent classes + 12 adversarial across 4 classes, 68 qrels, three-tier provenance incl. behavior-scored docs class); IR baseline overall nDCG@10 0.821 / MRR@10 0.857 / Recall@10 0.830; router-selection micro-F1 0.824 / macro 0.861 with two honest under-performers (identifier P=0.333, conceptual P=0.5 — both R=1.0, over-firing not missing); W3 latency optimization landed (single-query p50 4.4s → ~2.5s); W5 soak/chaos/adversarial harness live — built an mcpbench-compatible driver (evidence: mcpbench is Go-only, wrong default thresholds, no receipt awareness) with ladder / steady / chaos / adversarial modes + benchmark/compare.mts regression gate (p95>10% / p99>20% / receipt<100% / router_errors>0 / F1 drop>0.05); chaos slice verifies loud-classified per-expert failures end-to-end; adversarial 12/12 pass; receipt emission 100% with D1 cross-checks; healthy-stack baseline benchmark/baselines/wave-1-5-entry.json (n=81, 0 errors). Receipt-tail transient root-caused (vite-dev sqlite serialized writes → machine.ts last_seen_at cache + 60s throttle + evidence-based 60s deadline). Semantic-slot arm (Zilliz @zilliz/claude-context-core@0.1.15) LIVE 2026-07-15: dedicated cluster aws-us-west-2, semantic index 84 files / 1179 chunks (voyage-code-3, hybrid dense+sparse), fixture v1.2.0, config_version search-router@1.2.0. Data-driven re-admission: cross-repo-impact accepted semantic (AS-IS run +0.076 nDCG@10; a clean healthy-stack rerun measured −0.021 — within noise on n=6, kept as design-consistent with no measurable harm); conceptual NOT re-admitted (nDCG@10 collapsed 0.656 → 0.231; re-tested on the corpus-fixed index 2026-07-15 evening — still collapses 0.656 → 0.218, decision double-verified). Semantic corpus fixed same evening: 84→280 files (.svelte additive-extension fix + .prettierignore slurp root cause). Final selection micro-F1 1.000. Detailed measured results on /research/moe-search-router §Measured results v0.6.

Participating repos

  • nexartis-ozzydev Fixture set under version-controlled test-data (BEIR-shape corpus/queries/qrels with gold expert-set annotations, real-agent-transcript slice mined from KYM receipts); ranx-based IR-metrics harness running as a CI-adjacent nightly script, emitting rows into tool_benchmark_events with category="search" plus an experiments row keyed paper_slug="moe-search-router"; router-selection accuracy scorer (precision/recall/F1 on gold expert-set, expert-order rotation for positional-bias delta); mcpbench soak + concurrency runs (1/10/50/100 concurrent JSON-RPC clients + 60-min steady-state at 50% throughput ceiling + chaos slice) with regression-gate compare; three-tier gold-label bootstrapping (typed-graph weak supervision from serena/SCIP, LLM-as-labeler with 10% audit sampling, human-in-the-loop for adversarial tail); ADR-009 benchmark-plan template instantiated under docs/DECISIONS/; results read-back via GET /api/experiments?paper_slug=moe-search-router.

Exit criteria

  1. MET — Fixture set v1.1.0 committed under mcp/servers/ozzydev-native-search-mcp/benchmark/fixtures/ (six intent + four adversarial classes, 68 qrels, three-tier provenance incl. behavior-scored docs-* class).
  2. MET — ranx IR-metrics harness (benchmark/metrics.py via uv run --with ranx) shipped; per-class nDCG@10 / MRR@10 / Recall@10 emitted to tool_benchmark_events (category="search") via benchmark/emit-telemetry.mts.
  3. MET — Router-selection accuracy scored per class (benchmark/lib.mts scoreSelection); baseline micro-F1 0.824 / macro 0.861; oracle-gap under-performers (identifier P=0.333, conceptual P=0.5) tracked honestly for follow-up prior tuning.
  4. PARTIALLY MET — Per-class p50/p95/p99 measured and reported; budgets still exceeded against the ADR-007 §9 ceilings (overall p95 3457ms vs a ~3000ms docs ceiling; identifier / typed-symbol p95 ~3.4s vs 800ms). Stated loud in the results, not silently relaxed; optimization is the follow-up sprint surface.
  5. MET — ADR-009 receipt-verified emission rate = 100% across the fixture run (36 / 36) and across the soak baseline (81 / 81, D1 cross-checked 10 / 10).
  6. MET — Results emitted as tool_benchmark_events + experiments row keyed paper_slug="moe-search-router"; read-back via GET /api/experiments?paper_slug=moe-search-router verified.
  7. MET — Shared benchmark plan instantiated at docs/DECISIONS/BENCHMARK-PLAN-ozzydev-native-search-mcp.md (ADR-009 §12 template); v2 change-log entry captures the measured baseline.
  8. MET (parameterized) — Built an mcpbench-compatible soak driver (benchmark/soak.mts) because upstream mcpbench is Go-only, ships wrong default thresholds, and lacks receipt / envelope awareness. Ladder (c ∈ {1,10,50,100}) documents a vite-dev console receipt-ingest ceiling at c ≥ 50; recommended production concurrency c=10. Shipped 2-min live steady runs (n=81 healthy-stack baseline at benchmark/baselines/wave-1-5-entry.json, 0 errors, 100% receipts) + parameterized SOAK_MINUTES; 60-min operator invocation documented as a single command on-demand. Chaos slice verifies loud classified per-expert failures end-to-end (dead-port serena → 70× expert_unavailable, broken codesearch → expert_internal / timeout, receipts 100%). Adversarial 12/12. Regression gate benchmark/compare.mts (p95>10% / p99>20% / receipt<100% / router_errors>0 / F1 drop>0.05) live against the wave-entry baseline.
  9. MET — pnpm run validate green (295 console + 193 package tests at wave close).
  10. MET (LIVE 2026-07-15) — Semantic-slot arm activated per ADR-007 §2.1: dedicated Zilliz Cloud cluster provisioned in aws-us-west-2; applyZilliz resolved MILVUS_ADDRESS into the ozzydev-search MCP env; semantic index built (84 files / 1179 chunks, collection hybrid_code_chunks_b5fa75c8, voyage-code-3, hybrid dense+sparse). Live QUERY smoke returned 2 hits in 1234 ms (0 expert errors). Fixture set bumped to v1.2.0 and config_version to search-router@1.2.0. Data-driven re-admission per class: (a) cross-repo-impact gold_experts extended to [codesearch, semantic, serena] because the AS-IS baseline showed nDCG@10 +0.076 / MRR@10 +0.125 with semantic in the fan-out; a clean healthy-stack rerun (results/2026-07-16T00-33-35-848Z) measured −0.021 nDCG@10 — the IR effect is within run-to-run noise on n=6, so the re-admission is kept as design-consistent with no measurable harm rather than as a measured win; (b) conceptual NOT re-admitted — the ADR-007 §4.1 design priors {semantic: 1.0, codesearch: 0.8} collapsed conceptual nDCG@10 from 0.656 → 0.231 on this fixture because @zilliz/claude-context-core@0.1.15 does not chunk .svelte files (console UI sources absent from the semantic index) so RRF-fusion pushed hand-verified codesearch hits out of top-10; priors kept at {codesearch: 1.0}. CORPUS-FIX ADDENDUM (2026-07-15 evening): the .svelte gap was root-caused — (i) supportedExtensions is additive (fixed: .svelte now indexed via LangChain splitter fallback) and (ii) claude-context slurps root .prettierignore whose console entry excluded the ENTIRE console tree; fixed by renaming to prettier.ignore with --ignore-path rewiring. Semantic corpus 84→280 files / 1179→3098 chunks. Conceptual re-admission RE-TESTED on the full corpus: STILL collapses (0.656→0.218, results/2026-07-16T00-52-22-329Z) — decision double-verified; open hypothesis moved to the research wave (unmapped semantic hit paths: 27/36 queries; codesearch-centric qrels provenance). Final v1.2.0 selection micro-F1 1.000 / macro-F1 1.000 across all 6 classes; conceptual p95 2408 ms within §9 budget; cross-repo-impact p95 2991 ms over §9 2500 ms budget (semantic tail — declared loud, follow-up sprint surface). Full analysis on /research/moe-search-router §Measured results v0.6.

Wave harness/KYM split — harness is operational code, agent definitions live in KYM

Encode the operator architecture directive (2026-07-15): the parallel-waves harness stays OPERATIONAL CODE ONLY — engine (worktree fan-out, wave launching, gate wiring, permission enforcement, telemetry/receipt plumbing, merge-ladder automation hooks). Orchestrator and subagent DEFINITIONS — including every dispatch-contract clause as versioned prompt/skill content — are KYM artifacts (Agent Cards + versioned skills), where they are optimized, tested, and compared through KYM experiments (A/B, MAB) without touching the harness. Swapping an agent definition never requires a harness commit; comparisons flow as receipts back to KYM. Status: DESIGN LANDED (2026-07-15) — architecture rule + migration ledger live in /research/harness-bandit-experiments §architecture-rule (v0.4), cross-linked from /research/parallel-agent-orchestration §AGENTS-3; the boundary ADR under docs/DECISIONS/ is still OUTSTANDING; implementation (kym:// agent-definition resolution at dispatch) is a follow-up wave gated on operator approval.

Participating repos

  • nexartis-parallel-waves-agent-harness Honest current-shape assessment against the new boundary. Agent-definition-shaped content living in the harness today that MUST migrate to KYM: templates/*/claude-agent-prompt.md (subagent prompt bodies), templates/*/augment-orchestrator.md (orchestrator prompt bodies), templates/shared/spawned-AGENTS.md (spawned-run behavior contract), and any dispatch-contract clause text baked into scripts/prompt_rendering.py. Operational-and-staying: scripts/run_batch.py, scripts/provision_worktrees.py, scripts/waves_status.py, scripts/report_parsing.py, permission enforcement via OPENCODE_PERMISSION (edit/bash/task/external_directory deny), --tool-profile implementation --implementation-approved gate, engine adapters under scripts/engines/, per-job session-id + diff capture, wave-config schema, tracker.csv contract. Migration is design-first: reference agent-definition IDs (kym://agents/<slug>@<version>) in wave-config instead of inlining prompt text; harness resolves via KYM SDK at dispatch time. No code changes land in this wave; the migration ADR is the deliverable.
  • nexartis-ozzydev Document the split as the ARCHITECTURE RULE in /research/harness-bandit-experiments (primary home) with a short cross-link amendment in /research/parallel-agent-orchestration §AGENTS-3 (harness bakes OPERATIONAL clause enforcement; the dispatch-contract TEXT is KYM-versioned). Experiments key on the tuple (agent-definition version, skill-set + versions, wave spec, engine adapter) with reward flowing back to KYM as signed receipts — the same substrate as the harness-bandit study, now with agent-definition as a first-class variant axis. experiment_variants.config already accommodates this (harness-bandit v0.3.3 §phase-2-schema). No new schema; new ADR under docs/DECISIONS/ capturing the boundary.
  • knowyourmodel-ai Confirm Agent Card + versioned-skill publication surface (versioned manifest, Ed25519 receipt loop) hosts orchestrator + subagent definitions as first-class agents. Verify well-known agent-card endpoint returns skill versions and that receipts can be keyed by (agent-card-id, agent-card-version, skill-id, skill-version) so a KYM experiment can vary those axes without harness changes. Open questions to close before implementation: (a) exact Agent Card field for "role" (orchestrator vs subagent) — x-nexartis-runtime extension is the current hook; (b) whether dispatch-contract clauses ride as a single skill or as N versioned skills; (c) experiment-key shape for MAB read-back.
  • knowyourmodel-sdk Consumer check-in: getAgentCard(id) + signAndSubmitReceipt(payload) (already shipped) are the two surfaces the harness will call at dispatch and turn-completion. Verify receipt payload accommodates the experiment-key tuple above. No new SDK surface required for the split itself; any gaps become a paired PR.

Exit criteria

  1. Architecture rule DOCUMENTED as the primary section of /research/harness-bandit-experiments and cross-linked from /research/parallel-agent-orchestration §AGENTS-3 (no duplication — one canonical home, others cross-link).
  2. ADR under docs/DECISIONS/ captures the boundary: what is operational (stays in harness), what is agent-definition (moves to KYM), and the migration list from the harness assessment above.
  3. Migration list agreed with the operator: every file under nexartis-parallel-waves-agent-harness/templates/ that carries prompt or contract TEXT (not schema) is tagged for KYM migration; every file under scripts/ is tagged operational unless it embeds prompt strings.
  4. KYM surface facts confirmed: (a) Agent Card publishes with a version; (b) receipts carry (agent_card_id, agent_card_version, skill_id, skill_version) refs; (c) experiments can key on that tuple via experiments.paper_slug + experiment_variants.config. Any gap is an open question tracked back to knowyourmodel-ai / knowyourmodel-sdk. [CORRECTED 2026-07-16 survey: (a) and (b) are ASPIRATIONAL — entities carry one opaque version string, ReceiptPayload carries entity_id only; the gap list lives in the wave-kym-mcp-resident-agents entry (G1–G15).]
  5. PROOF-OF-EXIT (deferred to the implementation wave, not this one): the harness launches two DIFFERENT KYM-defined orchestrator variants against the same wave spec with ZERO harness diffs; both variants emit receipts to KYM keyed by the experiment tuple; changing an agent-definition version and re-running the same wave requires no harness commit and no harness restart.
  6. No harness code changes in this wave — this wave is direction + migration plan. Implementation lands in a follow-up wave gated on operator approval per the workspace convention.
  7. ROLE-SOURCE RESOLVED (operator, 2026-08-01) — subagent role→model→tool-profile config lives on KYM Agent Cards (x-nexartis-runtime.harnessDefaults); the harness resolves role→card at dispatch. Implementation lands in wave-1-6-custom-subagent-runtime.

Wave KYM MCP-resident agents — optimization-mode dogfooding

Status: DIRECTION SET (operator, 2026-07-16) — design outline at /research/mcp-resident-agents. Put a small, rigorously-defined agent (pinned LLM + versioned tool inventory + MCP-residency declaration) INSIDE each native MCP (ozzydev-search, ozzydev-write) to help orchestrators/subagents use the tool more effectively; register it as a KYM agent; run dogfooding hypotheses as optimization-mode experiments (A/B → MAB) over agent-definition versions with rewards flowing back as the ADR-009 signed receipts every operation already emits. Phase A needs ZERO KYM changes (resident agents as versioned MCP config, experiments in the console D1 substrate); Phase B lands the rigorous KYM registration + kym:// session-start pull-in; Phase C lands KYM optimization mode. Survey-verified gap list (2026-07-16): G1 immutable card versioning (entities.version is one opaque string), G2 structured tool inventory w/ versions (today names-only JSON), G3 llmPin (primaryModelId FK is not a version pin), G4 mcpResidency declaration, G5 per-skill versions (entitySkills has none), G6 SDK type surfacing, G7 ReceiptPayload pinning tuple (today entity_id only), G8 getAgentAtVersion + kym:// resolver (today prose-only), G9 roles enum on the runtime extension, G10 @nexartis/cubicle-adapter-local-mac + pullAgent session-start primitive (local-mac is not yet in CubicleRuntimeHost), G11 KYM write-experiments API + allocator (ab_tests is display-only today), G12 reward wiring on receipt verify, G13 console-D1 → KYM receipt forwarder (run_receipts.kym_status already models the lifecycle) [SHIPPED 2026-08-01: kym-forward cron drain live, loop closed — accepted rows on both ozzydev router entities, errors:0], G14 register the two resident helpers as KYM agents, G15 attribution indexes. COMPATIBILITY POLICY (operator, 2026-07-16): KYM is a GENERAL developer toolbox with beta clients — every extension is ADDITIVE-ONLY (new nullable columns, new optional fields, new routes/params; never rename fields/MCP tools, never change enum values in place, never make optional fields required). Receipt pinning-tuple constraint from the SDK audit: the verify endpoint signs JSON.stringify(payload) so SDK + platform must ship new optional fields in the SAME wave; persistence requires metadata-spread or new columns (top-level extras pass verify but are dropped at storage). The canonical KYM-side plan lives in knowyourmodel-ai/docs/ROADMAP.md (created 2026-07-16, aligned to this wave). Note: "KnowYourModel Link" is the product label for the x402/USDC payment surface in KYM docs, not a separate repo.

Participating repos

  • nexartis-ozzydev Phase A: agent.json (versioned resident-agent definition: llmPin, skills, version) alongside each native MCP config.json; ship the first skill — write ambiguity-resolver for the splice_ambiguous_anchor residual (4/31 on r5b) as a bounded, classified rung between splice and the vendor apply; run the first hypothesis through the existing console D1 experiments substrate keyed on definition version; every resident-agent action emits the standard signed receipt with the definition version in payload.common. Later phases: G13 forwarder, G15 indexes.
  • knowyourmodel-ai Additive-only per the compatibility policy; sequenced in knowyourmodel-ai/docs/ROADMAP.md §4. Phase B: schema deltas G1–G5 (versionSemver/versionId, tool_inventory, llmPin, mcpResidency, entitySkills.version) + pinned agent-card route (G8); Phase C: write-experiments API + allocator (G11) and reward upsert on receipt verify (G12).
  • knowyourmodel-sdk Phase B: surface the new card fields (G6), extend ReceiptPayload with the optional pinning tuple (G7), add getAgentAtVersion + resolveKymUri (G8).
  • nexartis-cubicle-sdk Phase B: add local-mac to CubicleRuntimeHost, publish @nexartis/cubicle-adapter-local-mac with the session-start pullAgent(cardRef) primitive (G10); formalize the roles enum + per-role tool/LLM overrides on the runtime extension (G9).

Exit criteria

  1. Phase A: first resident-agent skill live behind a loud classified gate; first experiment readout (variant vs control) persisted in the console D1 with signed receipts carrying the definition version.
  2. Phase B: both resident helpers registered as KYM agents with pinned LLM + versioned tool inventory; MCP boot hydrates the definition via kym://agents/<slug>@<version> through the local-mac adapter.
  3. Phase C: one dogfooding hypothesis runs end-to-end AS a KYM optimization-mode experiment — variant allocation by KYM, rewards from receipts, readout on the KYM dashboard.
  4. No hidden behavior: resident-agent calls that exceed their latency budget are loud classified skips, never silent degradation (workspace §2).

Wave ABI Chat primary surface — the operator loop moves off Kilo

Status: PLANNED. ABI Chat becomes the primary in-console agent surface for operator tasks — the /chat route ships on the console, driving the same cubicle runtime + native MCPs that Kilo drives today. Kilo remains a supported harness (the MCP-residency architecture keeps all intelligence below the harness boundary), but the default operator loop is the console. Exit: an operator completes a full session (search, write, validate, commit-request) through ABI Chat without opening Kilo.

Participating repos

  • nexartis-ozzydev Console /chat route on the ABI Chat SDK; session wiring to the cubicle runtime; the same kilo.json-equivalent MCP wiring generated from mcp/registry.json for the ABI Chat harness context.
  • nexartis-abi-chat-sdk Consumer check-in: session/turn abstractions against the ozzydev persistence schema (wave-1-1); any gaps become a paired PR.

Wave Ozzy-as-sidecar — clean runtime boundary for any harness

Status: PLANNED (architecture direction, operator 2026-07-16). Ozzy Dev must always be runnable as a SIDECAR: the memory/search/write systems (native MCPs + console + receipts flywheel) attach to any harness — Kilo today, ABI Chat next, the native console UI after — with zero re-porting. The MCP-residency principle (see /research/mcp-resident-agents) is the load-bearing rule: intelligence lives inside the MCP processes behind stable stdio/HTTP contracts; harnesses only wire spawn entries. Deliverable: a target-architecture doc naming the sidecar contract (what a host harness must provide: spawn context at workspace root, console reachability, session lifecycle) and a conformance checklist; then PRODUCT_ARCHITECTURE.md absorbs it as current-state once shipped.

Participating repos

  • nexartis-ozzydev Target-architecture doc for the sidecar contract; audit every native-MCP assumption that is Kilo-specific (spawn CWD, session-death behavior, no-respawn) and classify each as contract requirement vs Kilo quirk to abstract.

Wave console dogfood admin — editing-tools surface + receipts viewer

Status: PLANNED (from the 2026-07-16 console audit). Two admin gaps for the production dogfooding run: (1) a first-class editing-tools surface (native write router, Morph, weave-patch, ast-grep — category "editing" in mcp/registry.json) parallel to /memory, with the same live doctor/KPI pattern; (2) a receipts viewer — GET /api/receipts is healthy but has zero UI callers, so the flywheel is invisible to operators (per-workspace/per-run drilldown, kym_status chip).

Participating repos

  • nexartis-ozzydev Console /editing (or /write-tools) route iterating category=editing registry entries; receipts viewer route consuming GET /api/receipts with drilldown into run_receipt_payloads.

Wave 1.5 — Code-context benchmark

Status: SUPERSEDED by wave-1-5-search-hardening (2026-07-13; measured fixtures/IR-harness artifacts + F1 1.000 / nDCG 0.814 baseline live there). Kept for the historical scope description. Land a benchmark harness in the console so we pick the code-context MCP stack from measured data, not gut feel. Ship a score-card page comparing ChunkHound / Serena / codesearch across a fixture query set drawn from the Nexartis SDK monorepos.

Participating repos

  • nexartis-ozzydev Console /research/large-codebases/benchmark page: fixture query set, side-by-side scoring, precision@k + latency + tokens + freshness. mcp/registry.json is already seeded and admin-configurable at /memory (pulled forward into Foundation); the benchmark validates those picks. The Secrets panel shipped ahead of this wave (ADR-006): Voyage voyage-code-3 is the opinionated ChunkHound default with its key saved encrypted via /system → API Keys; Ollama is an opt-in local profile for machines that can hold an embedding model.

Wave 2 — Ozzy Dev Payments

Browse KYM for a second agent, pay in NANDA Points (x402-NP) or Base USDC, install locally. First working reference for NANDA payments in a local runtime. MVP "fork Ozzy" persona editor.

Wave 3 — Ozzy Dev Cross-Platform

install.ps1 produces the same working state on Windows 11 as install.sh on macOS; install.sh on Ubuntu produces the same on Linux. Same Agent Card, same Cubicle spec, different adapter.

Wave 4 — Ozzy Dev + Remote Control

Pair Ozzy Dev to Nexartis Remote Control from the console. Explicit, revocable, gated. Delegation via KYMOperatorDelegationCredential. Mobile Remote Control instructs Ozzy the same way the local console does.

Tech debt

Live log. New entries land in the sprint they are discovered; entries close when the sprint that fixes them ends.

  • Migrations are hand-authored from now on: drizzle-kit generate emits a whole-schema recreate against our 0002+ hand-authored tables (discovered writing 0006; convention pinned in ADR-014 D-B). Never commit an auto-generated recreate; follow the 0005/0006 hand-authored pattern.
  • Hint memo is per-process: hintCacheTtlMs amortizes D1 round-trips within one search-MCP session only. A cross-process file-backed memo under .ozzydev/ (shared across MCP spawns) is the follow-up design if first-call-per-session cost still matters in probe timings.
  • Hero-scene 3D materials use fixed dark-tuned colors (0x58a6ff, grid 0x22272e) — low contrast against the light theme. Cosmetic; revisit when the hero gets its next design pass.
  • Mermaid palettes in doc-view.svelte used to mirror tokens.css §3 as duplicated literals, and the theme flip re-called mermaid.initialize on every effect run — re-emitting global style rules into the document head. Fixed: palette is now sampled live from tokens.css via getComputedStyle (CSSOM), and initialize is guarded by a lastAppliedTheme sentinel so it only re-runs on an actual theme flip.
  • Light mode shipped broken: mode-watcher defaults add only a dark class while the palette lives under :root.light. Fixed by passing lightClassNames; a theme smoke test should cover both modes when component testing lands.
  • codesearch serve silently died under launchd: legacy launchctl load never bootstrapped the GUI-domain service, and the default serve mode expects a TTY/TUI. Fixed with daemon flags (--quiet true --no-tui) + modern launchctl bootstrap/enable/kickstart in the plist and mcp/install.sh. The Wave 1 doctor should alert on daemon-down, not just report it.
  • Serena edit-system live test (2026-07-09): outside tsconfig coverage the TS reference graph is blind — cross-file rename half-applied and safe_delete removed a referenced symbol; Serena modes do NOT gate editing tools (hard gate = read_only in project.yml). With a tsconfig: rename atomic across files, safe_delete correctly refused with reference coordinates. Policy: edits enabled for symbol-level refactors in tsconfig-covered code; findings recorded in registry notes + AGENTS.md; Svelte language backend + workspace folders added to the Serena project.
  • prettier-plugin-tailwindcss incompatible with prettier-plugin-svelte on Svelte 5 markup — removed from .prettierrc. Revisit when compatible.
  • KYM x-nexartis-runtime extension — HB3 uses the shape in practice today but never declared it. Wave 1 formalises across KYM, KYM SDK, and Cubicle Agent Card compatibility.
  • Fleet-wide lucide-svelte@lucide/svelte migration — Ozzy Dev migrated. Sibling PRs pending for knowyourmodel-ai (0.563 → 1.x) and nexartis-nanda-node (0.575 → 1.x).
  • ssr.noExternal explicitly listed for every Svelte-shipping package. Add new packages to this list when we consume them (@nexartis/* regex covers the SDK family).
  • Dashboard hero killed client-side navigation. Root cause: <T.OrbitControls /> referenced a class not in three.js core, and the Canvas container captured pointer events. Fixed by removing OrbitControls, gating on browser, adding pointer-events: none to the container, and rewriting the RAF loop as an $effect with explicit running flag.
  • Page-level typography lived in inline style="..." attributes on 4 pages. Consolidated into .page-header / .page-eyebrow / .page-title / .page-title-lg / .page-lede / .section-title / .section-label in app.css. Fonts, weights, sizes now change in one place.
  • morph→prettier friction: every morph-routed write to prettier-governed files fails prettier on comment formatting (4× in one session) — fix in ozzydev-write (format rung or post-format)
  • console vite dev reload churn drops in-flight POSTs under pnpm run validate (receipt_workspace_unresolved) — supervised now via KeepAlive LaunchAgent; durable fix is dev→preview or excluding .svelte-kit regen churn (console-repo scope)
  • checkpoint script derives nohead session id when OZZYDEV_SESSION_ID is unset in a fresh shell — discipline is --session always; consider a workspace session-file default
  • Peer-owned drizzle 0006 snapshot incomplete — the auto-generated meta snapshot missed columns on the hand-authored repos/workspace_repos tables (ADR-014 D-B). Regenerate against the hand-authored 0006 tables when the peer PR lands; never commit an auto-generated whole-schema recreate.
  • Session-adapter hardening follow-ons (receipt audit s19, 2026-08-01): session.opencode still emitted a 400 model-shape refusal at 09:41 UTC, two UnknownError 500s at 10:47/10:55 (refs err_7ad415a9 / err_99626768), and a .ozzydev-harness-servers/opencode-4310.log scope-violation at 10:57 — verify the running harness MCP dist actually carries d1f8c39 (model-shape) / 29af919 (agent-field 500) / 3e08dca (serve-log scope), rebuild+reload, and read the opencode server log by ref for the two unclassified 500s.
  • KYM submitter was stuck (RESOLVED 2026-08-01): every receipt sat kym_status='pending' with nothing posted upstream. Resolution: the nexartis-ozzydev-kym-forward cron worker drains every 5 min — rows without common.entity_id classify truthfully local-only (workspace §2), entity-bearing rows POST to KYM /api/receipts. Emitters shipped wave-15b (harness role cards + search/write router_entity_id); loop closed with accepted rows on both router entities, errors:0.
  • Write-mcp experiments-row upsert intent (Phase 6 s20 co-graduation follow-up): today per-arm receipts thread paper_slug and the join to experiments (keyed by paper_slug) works via the shared-receipt substrate. ADR-008 §7's distinct pre-spend experiments-row upsert on mode='experiment' is deferred pending operator confirmation — confirm intent, then wire the write-side pre-spend upsert to match the harness-side pattern.
  • Write-mcp rung3/rung4 commands null in config.verify_commands (Phase 6 s20): forced-rungs_1_4 on mode='experiment' reports skipped_by_scope honestly — the ladder plumbing is correct, but the ceiling for experiment-mode verify signal is capped until rung3/rung4 commands are declared per repo. Populate the write-side verify command config to unlock full-suite experiment verdicts.
  • Write-mcp invokeQueue FIFO serializes direct writes behind a long-running multiplex on the same router instance. Not a correctness issue (all writes still land in submission order), but a latency ceiling for concurrent direct-mode callers while a large multiplex arm-set is in flight. Consider a separate queue lane for direct-mode short-tasks, or per-mode concurrency budgets.
  • Reward composite v1-linear-normalised has no diff-content signal (Phase 6 s20 live-smoke observation): the empty-diff session.opencode arm scored 0.999 vs the working cli.claude-code arm at 0.54 in group 043f7f75-e812-4c12-ac7c-e974bce4c374 — the verify/cost/latency composite rewards an engine that returns no change and passes verify by construction over one that produces a correct implementation. Candidate fixes: add a diff_content_signal term to the composite, or gate composite on artifact != null && diff_stat.files > 0. Logged for the experiment framework; not a Phase 6 defect.
  • Write-mcp .worktrees-write-mcp/ parent dir shared across concurrent multiplex requests on one repo (Phase 6 s20): each request scopes its arms under <group>-<slug>/ inside the per-repo parent, so no path collision today, but two concurrent multiplex requests on the same repo share the parent directory — note for Phase 7 sandbox/container containment, where per-request isolation guarantees may want per-request parent dirs (or namespace-level scope-guards).
  • Wave 1.9 follow-up — behavioral verification rung (Shiplight pattern) LANDED on PR #18 (2026-08-03): rung 6 behavioral in the write/harness verify ladder (rungs_1_6), rung6_behavioral_cmd per-repo config, Playwright smoke surface for the console. Remaining: decide the 3 operator questions (port isolation, task-class default scope, env-vs-impl failure classification) before wiring it as a default harness rung.