KeiSeiKit-1.0/docs/ARCHITECTURE.md

Architecture

How agents, blocks, manifests, the assembler, and cross-tool bridges fit together.

---

The build pipeline

  Manifest (_manifests/<name>.toml)          <-- source of truth
         |
         |   [assembler/src/*.rs]              <-- Rust binary
         v
  Generated agent (.claude/agents/<name>.md)  <-- regenerated, never hand-edited
         ^
         |                                     [hook: assemble-agents]
  Block edit (_blocks/<block>.md)             <-- triggers rebuild of ALL agents

12 hooks enforce the pipeline (7 pipeline + 3 session-audit + 2 capability — agent-capability-check / agent-capability-verify wire the v0.24 substrate capability layer; see hooks/ directory for details):

• assemble-agents (PostToolUse, Write/Edit) — rebuilds the affected agent(s) whenever a manifest or a block changes. No manual rebuild needed.
• assemble-validate (PreToolUse, Bash) — blocks git commit inside ~/.claude if any manifest fails validation. Keeps the repo in a buildable state at all times.
• no-hand-edit-agents (PreToolUse, Edit/Write) — refuses edits to any .md under ~/.claude/agents/ that starts with the <!-- GENERATED --> marker, pointing you at the manifest instead. Override with AGENT_MIGRATION=1 for emergencies only.
• tomd-preread (PreToolUse, Read) — auto-converts opaque binary formats (.docx, .doc, .xlsx, .pptx, .csv) to markdown via the tomd primitive and redirects Claude to read the cached .md instead.
• agent-fork-logger (PreToolUse, Agent) — RULE 0.12 advisory: logs every Agent subagent invocation to the kei-ledger SQLite DB so the orchestrator can validate the fork bundle. Never blocks; silent no-op if kei-ledger is absent.
• orchestrator-dirty-check (PreToolUse, Agent) — RULE 0.13 advisory: stderr-warns when git status --porcelain of the current repo is non-empty before spawning a sub-agent, so orchestrators don't compound uncommitted output across parallel agents. Never blocks; bypass with ORCHESTRATOR_DIRTY_OK=1 (per-call) or ORCHESTRATOR_META=1 (meta-orchestrator).
• site-wysiwyd-check (PostToolUse, Edit/Write) — on frontend-source edits (.tsx, .vue, .svelte, .astro, .css, .html, .jsx, .ts) in a project with a live dev server (.keisei/dev-server.pid), takes a Playwright screenshot via mock-render and diffs against .keisei/target.png via visual-diff. Advisory-only — drift is reported to stderr, never blocks.
• session-end-dump (Stop event) — RULE 0.14 self-audit: archives the session JSONL trace and ingests it into kei-memory.
• milestone-commit-hook (PostToolUse, Bash) — RULE 0.14 self-audit: appends a one-line session summary to ~/.claude/memory/audit-backlog.md on every feat:/refactor:/merge commit.
• error-spike-detector (PostToolUse, any tool) — RULE 0.14 self-audit: tags + logs the pattern when 3+ errors occur within the last 20 tool calls.

Creating a new agent

Run the wizard in Claude Code:

/new-agent

You'll be asked (via multiple option-picker batches, not free-text) — each batch groups several click-only questions into a single AskUserQuestion call:

1. Project stack (Rust CLI / axum / SwiftUI / Flutter / FastAPI / Next.js / React-Vite / Vue-Nuxt / SvelteKit / Astro / Go / Embedded / Python ML)
2. Deploy target (local-only / EC2 / Cloudflare / Modal / Docker / none)
3. Uses paid APIs? (Yes / No)
4. Contains ML? (Yes / No)
5. Has credentials? (Yes / No)
6. Uses scrapers? (None / Free-tier / Paid tier)

Then one free-text prompt for slug + description + path + gotchas. The wizard composes the manifest, validates it, assembles the .md, and prints a two-step git-commit command you can run or edit first.

Adding custom blocks

Blocks are plain markdown in ~/.claude/agents/_blocks/. To add one:

1. touch ~/.claude/agents/_blocks/stack-mystack.md and write the block.
2. Reference it in a manifest's blocks = [...] list.
3. The PostToolUse hook rebuilds the affected agent(s) automatically.

Blocks should be 10-50 lines, single-concern, and readable in isolation. If a block exceeds ~60 lines, split it into two.

Adding custom manifests

Copy _templates/specialist.toml.template and fill the placeholders, OR run /new-agent and answer the wizard. Either way, the assembler validates the manifest and generates the .md on write.

Agents overview

All kit agents are namespaced under kei-* so they won't collide with your own agents (e.g. your personal validator or critic) living in ~/.claude/agents/.

Agent	Role
kei-code-implementer	Write production code, Constructor Pattern enforced, Test-First discipline
kei-infra-implementer	Deploy scripts, CI/CD, secrets management, cost-aware paid infra
kei-ml-implementer	Training scripts, inference code, Modal jobs, exact param counts
kei-critic	Read-only anti-pattern / bug / security / perf / debt finder
kei-validator	Fact-checker; verifies API existence, version compat, citations, doc claims
kei-security-auditor	Risk-classified security audit with variant analysis + supply chain check
kei-architect	Read-only structural analysis; dep graph, patterns, coupling
kei-researcher	Generic web + codebase research, evidence-graded findings
kei-ml-researcher	ML literature, benchmarks, reproducibility, tooling-reuse search
kei-cost-guardian	Pre-launch GO/NO-GO for paid compute (Modal, AWS, fal.ai, Apify, etc.)
kei-modal-runner	Modal compute orchestrator with anti-stop guard (never stops running jobs)
kei-fal-ai-runner	fal.ai image/video/3D generation expert

Cross-tool bridges

KeiSeiKit ships 11 verified tool-bridge templates under _bridges/. Render them into any project and the same Constructor-Pattern ruleset is visible to every AI coding tool you use — no drift, one source of truth.

Tools covered:

Tool	Output file
Cursor (legacy)	.cursorrules
Cursor (modern MDC)	.cursor/rules/main.mdc
Codex CLI / Warp / Zed / Antigravity fallback	AGENTS.md
GitHub Copilot	.github/copilot-instructions.md
Windsurf	.windsurf/rules/main.md
JetBrains Junie	.junie/guidelines.md
Continue.dev	.continue/rules/main.md
Google Antigravity / Gemini CLI	GEMINI.md
Aider	CONVENTIONS.md + .aider.conf.yml
Replit Agent	replit.md

Three ways to generate:

1. At install time — ./install.sh --with-bridges renders all 11 into $PWD after the normal install completes. Skipped if $PWD is the KeiSeiKit repo itself.
2. From the /new-agent wizard — Phase 8 asks click-only whether to generate all 11, just AGENTS.md, or skip.
3. Manually, any time — ~/.claude/agents/_bridges/emit.sh <project-dir> (the install copies _bridges/ into your agent fleet dir). Add --only <output-path> to restrict to a single file.

All paths are idempotent: existing bridge files in the project are skipped, never overwritten. See _bridges/README.md for the full template→output-path table.

Meta-composer

/compose-solution is the meta-creator: tell it what you want to solve in one free-text paragraph, it decomposes the task, greps existing blocks / skills / manifests / primitives / bridges for prior art, proposes a minimal math-first architecture, and assembles the right artefact — agent, skill, hook, rule, block, or pipeline invocation. Every decision except the intake is a click (option-picker), never free-text.

Example: "I want a hook that blocks rm -rf ~/ in any Bash call" → Phase 2 decomposes into (pattern-match, severity, event, wiki entry) → Phase 3 greps hooks/, _blocks/, _primitives/ for prior art → Phase 5 proposes hook = PreToolUse:Bash + pattern + exit 2 → Phase 7 hands off to /escalate-recurrence with severity and event pre-filled.

Phase 6 is the feedback loop: when a component has no prior art, the skill drafts a new _blocks/<slug>.md and — on your click — persists it. Next time /compose-solution (or /new-agent) runs, that block is discoverable. Every session leaves the kit a little smarter; the report prints _blocks/ count before → after so the growth is visible.

See skills/compose-solution/SKILL.md and its phase files (phase-1-intake.md through phase-7-assemble.md) for the full 7-phase pipeline.

Regenerating counts

Every number in the README / INSTALL.md (crates / skills / hooks / blocks / primitives / profile sizes) is wrapped in an HTML-comment marker — <!-- count:NAME -->24<!-- /count:NAME --> — and regenerated from sources of truth (_primitives/MANIFEST.toml, _primitives/_rust/Cargo.toml, filesystem walks). No more drift when a primitive or skill is added.

./scripts/regen-counts.sh            # rewrite README.md in place
./scripts/regen-counts.sh --check    # exit 1 if drift detected (no writes)

Pre-commit gate: scripts/precommit-counts-check.sh — wire it into your hook manager (or symlink into .git/hooks/pre-commit) to block commits when README counts drift from the sources.

Workflow-file editing protocol

Every .github/workflows/*.yml edit is defended by three gates. The v0.20.1 incident (a real-but-wrong-semantic SHA pin on dtolnay/rust-toolchain broke CI for 30 minutes before discovery) motivated formalising them.

• scripts/lint-workflows.sh — runs actionlint over every workflow file. Catches syntax errors, expression typos, dead if: branches, and shell-injection risks. If the binary isn't on PATH, the script prints an install hint and exits 0 (advisory). Install with bash scripts/install-actionlint.sh or brew install actionlint.
• scripts/validate-workflow-shas.sh — extracts every uses: <repo>@<sha40> pin from .github/workflows/*.yml + .github/dependabot.yml and runs git ls-remote https://github.com/<repo>.git <sha>. A fabricated or force-pushed-out-of-existence SHA exits 1 with SHA MISSING:. Network errors are soft ([UNVERIFIED]). Tag refs like @v4 or @stable are skipped (policy decision). Add trailing comment # validate-workflow-shas: skip=<reason> on a line to intentionally skip it.
• CI job workflow-lint — runs both scripts on every push and PR. Finishes in well under 30 s.
• Optional pre-commit hook: ln -sf ../../scripts/pre-commit-workflow-lint.sh .git/hooks/pre-commit — runs the two scripts only when a workflow file is staged.

SHA-pinning third-party actions defeats tag re-point attacks (CVE-2025-30066 class), but only if the SHA you wrote is real AND means what you think it means. actionlint catches the first class of mistake; validate-workflow-shas.sh catches the second. Together they close the window between local edit and CI-fail.

---

Layer model (atoms, recipes, skills, frontends)

The substrate is layered. Each layer has a single contract; layers below are stable foundations for layers above.

┌──────────────────────────────────────────────────────────────────────────┐
│ Layer 4 — Frontends (where the user types)                               │
│   Claude Code plugin · cortex-ui (Svelte) · kei-tty (ratatui) ·          │
│   @keisei/vscode-cortex · MCP clients (Cline, OpenClaw, Cursor MCP)      │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 3 — Skills (markdown wizards)                                      │
│   68 `/commands` under skills/<name>/SKILL.md                            │
│   Each is an AskUserQuestion-driven phase-pipeline (5-9 phases typical)  │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 2 — Recipes (TOML DAGs)                                            │
│   Topo-sorted atom graphs with explicit JSON I/O between steps           │
│   Runtime: kei-pipe                                                      │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 1 — Primitives (cubes)                                             │
│   105 Rust workspace crates (47 declared in MANIFEST.toml `full` profile)│
│   + 14 shell primitives. Each ≤ 200 LOC / file, ≤ 30 LOC / function.     │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 0 — Atoms (locked vocabulary)                                      │
│   13 verbs: INGEST PARSE TRANSFORM ENRICH VALIDATE DECIDE DISPATCH       │
│             PERSIST RETRIEVE SEQUENCE EMIT OBSERVE EXECUTE               │
│   Discovery: kei-atom-discovery walks _primitives/_rust/*/atoms/*.md     │
└──────────────────────────────────────────────────────────────────────────┘

The atom set is the substrate's locked vocabulary. New verbs require an explicit schema change (see SUBSTRATE-SCHEMA.md and AGENT-SUBSTRATE-SCHEMA.md). New primitives, recipes, skills, and frontends compose without touching the lower layers.

Atom → MCP tool

kei-mcp walks _primitives/_rust/<crate>/atoms/<verb>.md files and emits one MCP tool per atom — <crate>::<verb> is the tool name; the atom's first paragraph is the description; the atom's frontmatter input_schema is the JSON-Schema. The same atoms become callable from any MCP client (Claude Code, Cline, OpenClaw, Cursor with MCP) without rewiring. See _primitives/_rust/kei-mcp/README.md for the wire format and the three client-config examples.

Skill → MCP resource

The same kei-mcp server walks skills/<name>/SKILL.md and emits one MCP resource per skill at skill://<name>. MCP-aware clients can read the skill text without invoking it — useful for "explain this command before running it" UX flows.

Frontend → daemon

The cortex stack inverts the flow for browser / terminal / editor usage. Instead of an LLM client calling MCP tools, a Rust daemon (kei-cortex) hosts the LLM loop and exposes a REST + WS surface to thin frontends. The token in ~/.keisei/cortex.token is the only auth boundary.

  Browser (cortex-ui)        Terminal (kei-tty)      VSCode (vscode-cortex)
       │                            │                          │
       │      Authorization: Bearer <cortex.token>             │
       └────────────┬───────────────┴──────────────────────────┘
                    │
              http://127.0.0.1:9797
                    │
                kei-cortex (axum)
                    │
       ┌────────────┼────────────┬─────────────┐
       │            │            │             │
   Anthropic    ElevenLabs    fal.ai     local Python
    (chat)        (TTS)      (portrait)    (faster-whisper STT)

Every endpoint streams when streaming makes sense (/chat SSE, /tts audio chunks, WS /term PTY frames). The 8 built-in tools that the /chat agentic loop wires — read, write, edit, bash, glob, grep, webfetch, agent (see _primitives/_rust/kei-cortex/src/tool/registry.rs) — all execute locally; only the LLM call leaves the machine.

---

Memory architecture (3-layer)

Three markdown layers — ~/.claude/CLAUDE.md (algorithms, depends on nothing) → ~/.claude/memory/<project>.md (per-project data, self-contained) → ~/.claude/memory/MEMORY.md (≤ 200-line index, links only). Three SQLite files underlie them: agents/ledger.sqlite (RULE 0.12), memory/sessions.sqlite (JSONL + embeddings), memory/kei-memory.sqlite (RULE 0.14 pattern detector). All inspectable with sqlite3 and kei-brain-view summary. Nothing in RAM-only.

---

Sleep architecture (Phase A / B / C)

Three nightly phases, each addressing a different consolidation problem. Full spec in SLEEP-LAYER.md.

03:00 local
  │
  ├─ Phase A — Incubation (the daytime task queue)
  │    Up to 5 user-submitted tasks from /sleep-on-it, ≤ 480 min total.
  │    Marathon mode: 1 task gets the whole night.
  │    Per-task budgets: quick 15 / standard 60 / deep 240 / marathon 480.
  │    Checkpointing every N min so a cut-short run leaves an artefact.
  │    Output: sleep-results/<uuid>.md, sleep-results/<uuid>.partial.md
  │
  ├─ Phase B — REM consolidation (cross-session pattern report)
  │    Agent reads the day's traces, finds recurrences, writes
  │    sleep-reports/YYYY-MM-DD.md. User reads in the morning.
  │    Skipped on marathon nights.
  │
  └─ Phase C — NREM deep-sleep (system consolidation, every 7 days)
       kei-conflict-scan + kei-refactor-engine + kei-graph-check.
       Output: deep-sleep/YYYY-MM-DD branch (plan + optional patch).
       Zero-conflict guarantee: any conflict requiring human decision
       is excluded from the patch and listed explicitly in the plan.

The user always reads. Nothing the cloud agent writes is auto-injected into the next session — codification goes through /escalate-recurrence.

---

Substrate dogfood (kei-fork → kei-ledger → kei-spawn)

Every non-trivial agent invocation is itself a substrate transaction. The lifecycle:

kei-fork create <agent-slug>
       │
       │  - branch: agent/<slug>-<timestamp>
       │  - worktree: _forks/<slug>-<ts>/
       │  - ledger row: (agent_id, branch, parent, status=running)
       │  - DNA: <role>::<caps>::<scope8>::<body8>-<nonce8>
       ▼
agent writes 6 artefacts inside the worktree:
  spec.md plan.md progress.json chatlog.md handoffs.md review.md
       │
       │  - touches .DONE marker on completion
       │  - kei-watch hook auto-collects
       ▼
kei-fork collect
       │
       │  - merge or squash decision via AskUserQuestion (RULE 0.12)
       │  - ledger row → status=merged|rejected|deferred
       │  - worktree archived: _forks/_archive/...
       ▼
kei-ledger validate         ← optional gate before merge
kei-replay <agent_id>       ← reconstruct the spawn from DNA + ledger

Per RULE 0.13, the orchestrator (parent agent or main session) creates the branch, commits, and pushes. The child agent only writes files inside the worktree — it cannot invoke git from there because the sandbox denies Bash inside .claude/worktrees/<agent>/ by design. The 6-file artefact bundle is the agent's full audit trail.

---

Model router — current state (2026-05-03)

The kei-model-router crate implements a Bayesian-posterior tier selector (Haiku / Sonnet / Opus) keyed on task-class DNA + a Beta posterior over per-(task-class, model) success rates. The companion kei-token-tracker crate logs TokenEvent rows per LLM call to SQLite.

What this is and is not, today:

• It is a long-running learning loop. The active-learning consumer reads outcome rows from kei-token-tracker and updates Beta-posterior parameters per (task-class, model) pair. As more outcomes accumulate, the ranking deviates from the manifest-declared default.
• It is not "smart routing on day one". A fresh install has 0 outcome rows. The router uses a Wilson-style lower confidence bound (δ=0.10, q*=0.70) — it falls back to the manifest-declared default UNTIL the per-(task-class, model) lower-bound clears the quality bar. This is a continuous metric, NOT a discrete 100-row threshold (see _primitives/_rust/kei-model-router/src/select.rs:74-124); typically tens of successful observations per pair are sufficient. With 37 agent manifests currently declaring model: opus, the practical effect on a fresh install is "always Opus" — the router's posterior is dominated by the uniform Beta(1,1) prior and has no data to override the default.
• Outcome-row count for a fresh install: 0. Plan to run for some weeks under realistic load before the router meaningfully reorders tier selection. Until then, route by orchestrator discipline + advisor-hook stderr nudges, exactly as the README's "Honest limits" section calls out.

The Beta-posterior + cost-minimisation math is in _primitives/_rust/kei-model-router/src/. The aggregation surface (per-model cost / day, sleep-report markdown emitter) is in kei-token-tracker.

---

Git model summary

Cross-references to the rules that govern git state:

• RULE 0.12 — agent-fork lifecycle (6-file bundle + ledger row + merge ceremony).
• RULE 0.13 — orchestrator owns branch + commits + pushes; agents write files only.

The checkpoint:, feat:, refactor:, audit:, and fix: commit prefixes are SSoT'd in ~/.claude/rules/git-conventions.md (a per-user rule file installed outside this public repo; see _blocks/ for the in-repo block-format SSoT). kei-changelog walks Conventional Commits to regenerate CHANGELOG.md blocks; kei-fork validate checks the 6-file bundle on every merge.