From 2bb2f10af046833a99e8463d3b8ed267578160f6 Mon Sep 17 00:00:00 2001 From: Parfii-bot Date: Sun, 3 May 2026 17:00:28 +0800 Subject: [PATCH] docs: add 6-file substrate encyclopedia (1739 LOC) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generated by parallel Haiku writer agents during 4-wave audit; covers the substrate at the top-down explanatory level the reviewer asked for. - substrate-overview.md (425 LOC) — top-down: what runs at install, daily, nightly; data-flow ASCII diagrams; how the 4 layers fit - hooks-and-blocks.md (394 LOC) — every hook + every assembler block, with trigger event + severity + rule reference - rust-crates-A-G.md (507 LOC) — first third of the 106 crates, one paragraph per crate - rust-crates-H-N.md (194 LOC) — middle third - rust-crates-O-Z.md (59 LOC) — last third (smaller because alphabet) - skills-and-agents.md (160 LOC) — 67 skills + 43 agent manifests, one row each Encyclopedia complements the auto-generated DNA-INDEX.md: that file is mechanical (count + DNA prefix + sha8), this is narrative (what does this thing do, when does it fire, how to use it). Username-path leaks scrubbed via sed pre-commit: - /Users//Projects/KeiSeiKit-public/ → / - /Users// → / Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/encyclopedia/hooks-and-blocks.md | 394 ++++++++++++++++++ docs/encyclopedia/rust-crates-A-G.md | 507 ++++++++++++++++++++++++ docs/encyclopedia/rust-crates-H-N.md | 194 +++++++++ docs/encyclopedia/rust-crates-O-Z.md | 59 +++ docs/encyclopedia/skills-and-agents.md | 160 ++++++++ docs/encyclopedia/substrate-overview.md | 425 ++++++++++++++++++++ 6 files changed, 1739 insertions(+) create mode 100644 docs/encyclopedia/hooks-and-blocks.md create mode 100644 docs/encyclopedia/rust-crates-A-G.md create mode 100644 docs/encyclopedia/rust-crates-H-N.md create mode 100644 docs/encyclopedia/rust-crates-O-Z.md create mode 100644 docs/encyclopedia/skills-and-agents.md create mode 100644 docs/encyclopedia/substrate-overview.md diff --git a/docs/encyclopedia/hooks-and-blocks.md b/docs/encyclopedia/hooks-and-blocks.md new file mode 100644 index 0000000..9521f55 --- /dev/null +++ b/docs/encyclopedia/hooks-and-blocks.md @@ -0,0 +1,394 @@ +# KeiSeiKit Hooks & Blocks Catalogue + +Comprehensive index of all 55 hooks and 85 blocks in KeiSeiKit-public as of 2026-05-02. + +**Structure:** +- **Hooks (55):** Claude Code safety + observability rules, wired into `.claude/settings.json` hook events (PreToolUse, PostToolUse, UserPromptSubmit, Stop). +- **Blocks (85):** Reusable architectural patterns covering API design, auth, databases, deployment, security, stacks, testing, rules, and domain-specific practices. + +--- + +## Part 1: Hooks Reference + +All hooks live under `hooks/` directory. Format: `| Hook Name | Event | Severity | Purpose | Bypass Env |` + +**Event types:** +- **PreToolUse:Bash** — called before any shell command +- **PreToolUse:Edit|Write** — called before modifying files +- **PreToolUse:Agent** — called before spawning sub-agent +- **PostToolUse:Agent** — called after agent returns +- **PostToolUse:Bash** — called after shell command completes +- **UserPromptSubmit** — called when user submits a prompt +- **Stop** — called when session ends + +**Severity levels:** +- **block (exit 2)** — Claude Code aborts the tool call +- **enforce (exit 1)** — error; must fix before retrying +- **warn (exit 0 + stderr)** — advisory message, tool call proceeds +- **remind (exit 0 + stderr on trigger)** — passive reminder +- **advisory** — informational, never blocks + +### Core Safety Hooks + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| no-github-push.sh | PreToolUse:Bash | block | Prevent pushing KeiTech patent IP to github.com — destroys priority date | KEI_NO_GITHUB_PUSH_BYPASS | +| no-python-without-approval.sh | PreToolUse:Bash | block | Enforce RULE 0.2 (Rust first) — Python requires exception justification | none | +| rust-first.sh | UserPromptSubmit | remind | Remind about Rust-first default for new work | none | +| secrets-pre-guard.sh | PreToolUse:Edit\|Write | block | Detect hardcoded API keys, tokens, private keys before commit | KEI_SECRETS_GUARD_BYPASS | +| destructive-guard.sh | PreToolUse:Bash | block | Block dangerous commands (rm -rf /, git reset --hard main, truncate) | none | +| tomd-preread.sh | PreToolUse:Read | warn | Auto-convert .docx, .xlsx, .pptx to .md before reading | none | + +### Disk & Resource Management + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| disk-headroom-check.sh | PreToolUse:Bash | block/warn (tiered) | RULE 0.17: 4-tier ladder (≥20G silent, 10-20 warn, 5-10 warn+suggest, 2-5 block-heavy, <2 hard-block) | DISK_GUARD_BYPASS | +| disk-reclaim.sh | launchd 03:30 | advisory | Nightly cleanup: orphan worktrees + stale target/ dirs (168h+ old, clean, unpushed, no live PID) | none | + +### Numeric Claims & Evidence (RULE 0.18) + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| numeric-claims-guard.sh | PreToolUse:Edit\|Write | enforce | Block time/cost/effort claims without `[REAL:]` / `[FROM-JOURNAL:]` / `[ESTIMATE-HTC:]` marker | RULE_017_BYPASS | +| numeric-claims-record.sh | Post-Write | block | Auto-log numeric claims to `memory/time-metrics/*.jsonl` | NUMERIC_CLAIMS_RECORD_BYPASS | +| chat-numeric-prewarn.sh | UserPromptSubmit | remind | Inject RULE 0.18 reminder when user prompt contains time/cost keywords | RULE_018_CHAT_BYPASS | +| chat-numeric-postflag.sh | Stop | warn | Flag any bare numerics in last assistant message without evidence marker | RULE_018_CHAT_BYPASS | + +### Agent Lifecycle & Status Truth (RULE 0.12, 0.16) + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| agent-fork-logger.sh | PreToolUse:Agent | advisory | Log agent fork to ledger (RULE 0.12) — non-blocking, silent on missing kei-ledger | none | +| agent-event-spawn.sh | PreToolUse:Agent | block | Record parent-child agent relationship in /tmp/kei-active-children.tsv | KEI_EVENTS_BYPASS | +| agent-event-done.sh | PostToolUse:Agent | block | Mark agent as done in active-children ledger | KEI_EVENTS_BYPASS | +| agent-fork-done.sh | PostToolUse:Agent | block | Transition ledger row from 'running' → 'done' or 'fail' (pairs with agent-fork-logger) | none | +| agent-outcome-backfill.sh | PostToolUse:Agent | block | Parse STATUS-TRUTH MARKER from agent transcript + backfill shipped/stubs/cargo-check/behaviour-verified (RULE 0.16) | OUTCOME_BACKFILL_BYPASS | +| agent-stub-scan.sh | PostToolUse:Agent | enforce | RULE 0.16: validate shipped=functional ⟹ stubs=0 consistency; 7-day WARN → then ENFORCE | STATUS_TRUTH_BYPASS | +| agent-heartbeat-tick.sh | PostToolUse:\* | advisory | Heartbeat: auto-update `progress.json` in agent's `.claude/agents//` dir every 30s | KEI_PING_BYPASS | +| tool-use-event.sh | PostToolUse:\* | block | Flatten tool response + attribute to parent agent via active-children ledger | KEI_EVENTS_BYPASS | +| extract-task-durations.sh | Pre/Post Agent | block | Record agent start/end times in `memory/time-metrics/tasks.jsonl` | none | +| task-timer.sh | Pre/Post Agent + Stop | block | Master timer: compute wall-clock duration per task + session | none | + +### Code Quality & Architecture + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| citation-verify.sh | PreToolUse:Edit\|Write | block | RULE 0.4: block academic citations without `[VERIFIED: url]` or `[UNVERIFIED]` marker | none | +| auto-dev-guard.sh | PreToolUse:Edit | warn | Warn on frontend (.tsx/.ts/.svelte/.vue/.dart) or DB-layer changes without corresponding tests | none | +| auto-register-on-edit.sh | PostToolUse:Edit | block | When substrate file (skill/hook/block/capability/role) edited → auto-register in manifest registry | AUTO_REGISTER_BYPASS | +| auto-encyclopedia-refresh.sh | PostToolUse:Edit | warn | Refresh docs/encyclopedia/ after substrate file changes (pairs with auto-register-on-edit) | AUTO_ENCYCLOPEDIA_BYPASS | +| no-hand-edit-agents.sh | PreToolUse:Edit | block | Prevent hand-editing `.claude/agents//` — edit manifest instead (RULE 0.12) | none | +| post-write-check.sh | PostToolUse:Write | warn | Async check: warn on large files (>5MB) + hardcoded secrets | none | +| post-commit-audit.sh | PostToolUse:Bash (git commit) | remind | Remind about double audit after commit | none | +| decompose-rules-on-edit.sh | PreToolUse:Edit (~/.claude/rules/\*.md) | warn | When rule file edited → re-run kei-decompose to update RULES.md registry | DECOMPOSE_RULES_BYPASS | + +### Observability & Session State + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| session-end-dump.sh | Stop | block | Call `kei-memory ingest` (populate traces/) + `kei-sleep-sync.sh` (push to memory-repo) | none | +| skill-record.sh | PostToolUse:Agent | block | Record skill invocation for analytics (never blocks — exit 0 always) | SKILL_RECORD_BYPASS | +| tool-use-event.sh | PostToolUse:\* | block | Event logging for agent attribution + wall-clock tracking | KEI_EVENTS_BYPASS | +| check-error-patterns.sh | PostToolUse:\* | block | Detect error spikes (≥3 in last 20 calls) → append to audit-backlog.md | none | +| error-spike-detector.sh | PostToolUse:\* | block | Rolling 20-call error window; on spike → log pattern + suggest escalation | none | +| milestone-commit-hook.sh | PostToolUse:Bash (git commit) | block | On feat:/refactor:/merge → run `kei-memory analyze` + append to audit-backlog.md | none | +| alignment-check.sh | (internal) | block | Track 3-time recurrence: exp6, exp24-28, basecaller forgot alignment | none | +| graph-export-watcher.sh | PreToolUse:Edit | warn | When keimd graph file edited → watch for drift | GRAPH_EXPORT_BYPASS | +| site-wysiwyd-check.sh | PostToolUse:Edit | advisory | After frontend edit → WYSIWYG drift check against dev-server | none | + +### Sleep Layer & Cloud Agent Triggers (RULE 0.15) + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| sleep-report-tg.sh | (nightly Phase B remote) | block | Cloud agent reads `reports/sleep-*.md` + sends Telegram summary | SLEEP_REPORT_TG_BYPASS | +| phase-b-rem.sh | (nightly Phase B REM consolidation) | block | Run `kei-conflict-scan` + `kei-refactor-engine` for NREM deep-sleep (v0.13) | none | +| affect-live-scan.sh | Stop | remind | Pairs with session-end-dump — writes affect file for pattern analysis | AFFECT_LIVE_BYPASS | + +### Assembly & Manifest Management + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| assemble-agents.sh | PostToolUse:Edit (_manifests/\*.toml) | block | When agent manifest edited → rebuild that agent crate | none | +| assemble-validate.sh | PostToolUse:Edit (_manifests/) | block | Validate all manifests on edit; failure → block commit | none | +| no-hand-edit-agents.sh | PreToolUse:Edit (.claude/agents/) | block | Prevent hand-editing agent directories | none | + +### Safety & Enforcement + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| stop-verify.sh | Stop | advisory | Check for uncommitted changes + running Modal compute before session ends | none | +| orchestrator-dirty-check.sh | UserPromptSubmit | advisory | Warn if orchestrator has uncommitted agent output from prior waves | none | +| orchestrator-branch-check.sh | PreToolUse:Agent | remind | Remind that orchestrator should be on feat/\* branch (not main) before spawning agent | ORCHESTRATOR_META | +| block-dangerous.sh | (internal) | block | Block dangerous patterns (defined inline) | none | +| safety-guard.sh | (internal) | block | Generic safety pattern matcher | none | + +### Specialized Domain Hooks + +| Hook | Event | Severity | Purpose | Bypass Env | +|------|-------|----------|---------|-----------| +| no-downgrade.sh | PreToolUse:Edit\|Write | enforce | RULE -1: when problem found in Edit/Write, require 3+ solution paths (not defeatism) | none | +| recurrence-suggest.sh | UserPromptSubmit | remind | RULE 0.10: detect same mistake ≥2× in session → suggest `/escalate-recurrence` skill | none | +| agent-capability-check.sh | (internal) | block | Check agent capability before spawn | none | +| agent-capability-verify.sh | (internal) | block | Verify agent claimed capability matches manifest | none | + +--- + +## Part 2: Blocks Reference + +All blocks live under `_blocks/` directory. Format: `| Block Name | Category | Purpose |` + +**Categories:** +- **api** — HTTP/API design, versioning, conventions +- **auth** — authentication, authorization, sessions +- **ci** — continuous integration + release +- **db** — databases, migrations, SQL +- **deploy** — cloud/VPS deployment targets +- **docs** — documentation conventions +- **domain** — domain-specific rules (ML, secrets, paid APIs) +- **evidence** — grading framework +- **memory** — session memory protocol +- **mode** — reasoning modes (first-principles, maximalist, skeptic, etc.) +- **obs** — observability (metrics, logs, traces) +- **path** — file system paths +- **pipeline** — multi-phase project templates +- **rule** — development discipline rules +- **scraper** — web scraping tiers + patterns +- **security** — hardening, patching, audit logging +- **stack** — tech stack reference patterns +- **test** — testing strategies + +### API & Integration (8 blocks) + +| Block | Purpose | +|-------|---------| +| api-anthropic.md | Anthropic API integration (models, tokens, rate limits) | +| api-apify.md | Apify web scraping API (actors, storage, datastores) | +| api-elevenlabs.md | ElevenLabs TTS API (voices, models, billing) | +| api-fal-ai.md | fal.ai image generation API (models, async webhooks) | +| api-graphql.md | GraphQL design conventions (schema, queries, subscriptions) | +| api-openapi-first.md | OpenAPI/Swagger as SSoT for API contracts | +| api-rest-conventions.md | REST verbs, status codes, resources, idempotency, ETags (RFC 9110, 9457) | +| api-versioning-pagination-ratelimit.md | API versioning strategy, cursor/offset pagination, rate-limit headers | + +### Authentication & Authorization (4 blocks) + +| Block | Purpose | +|-------|---------| +| auth-authorization.md | RBAC, ABAC, scopes, permission delegation | +| auth-oauth2-oidc.md | OAuth 2.0 / OpenID Connect flows (authz code, PKCE, ID tokens) | +| auth-passkeys.md | WebAuthn / FIDO2 passkey registration + assertion | +| auth-sessions.md | Session management (secure cookies, CSRF tokens, refresh tokens) | + +### CI/CD & Release (3 blocks) + +| Block | Purpose | +|-------|---------| +| ci-forgejo-actions.md | Forgejo Actions (internal runner, workflow.yml, secrets) | +| ci-github-actions.md | GitHub Actions workflows (matrix, artifacts, deployment) | +| ci-release-automation.md | Semantic versioning, changelog, auto-release tags | +| ci-security-gate.md | Pre-commit scanning (secrets, CVE, licenses) | + +### Database (5 blocks) + +| Block | Purpose | +|-------|---------| +| db-drizzle.md | Drizzle ORM (schema, relations, migrations in TypeScript) | +| db-migration-hygiene.md | Reversible migrations, zero-downtime deployments, test safety | +| db-postgres.md | PostgreSQL (setup, performance tuning, connection pooling) | +| db-sqlite.md | SQLite (embedded, dev fixtures, backup strategy) | +| db-sqlx.md | sqlx Rust macro (compile-time checked SQL, migrations) | + +### Deployment (7 blocks) + +| Block | Purpose | +|-------|---------| +| deploy-aws-ec2.md | AWS EC2 instances, security groups, Elastic IPs | +| deploy-cloudflare.md | Cloudflare Workers, KV, R2, Pages (edge compute + CDN) | +| deploy-docker.md | Docker image best practices, multi-stage builds, scanning | +| deploy-hetzner-cloud.md | Hetzner Cloud VPS (volume, networking, backup) | +| deploy-local-only.md | Local/private deployment (Tailscale, no public IP) | +| deploy-modal.md | Modal.com GPU cloud (Functions, Volumes, checkpointing, KILL GUARD) | +| deploy-vps-generic.md | Generic VPS pattern (Systemd, Caddy, SSH hardening) | + +### Documentation (5 blocks) + +| Block | Purpose | +|-------|---------| +| docs-architecture-diagrams.md | Mermaid, C4, ADR diagrams for architecture | +| docs-claude-md.md | CLAUDE.md project-specific instructions + umbrella rules | +| docs-decisions-adr.md | Architecture Decision Records (format, storage, reversals) | +| docs-readme-template.md | Minimal README structure (goal, quickstart, docs link) | +| docs-runbook.md | Operational runbooks for prod incidents, backup/restore, scaling | + +### Domain-Specific Rules (3 blocks) + +| Block | Purpose | +|-------|---------| +| domain-has-secrets.md | Projects with credentials / keys / auth (setup, RULE 0.8) | +| domain-ml-training.md | ML experiment discipline (pre-reg, math-first, observability) | +| domain-paid-apis.md | Cost guard for Modal, AWS, fal.ai, Apify billing | + +### Evidence & Grading (1 block) + +| Block | Purpose | +|-------|---------| +| evidence-grading.md | E1-E6 evidence grades (E1=fact, E6=speculation) | + +### Memory & Session (1 block) + +| Block | Purpose | +|-------|---------| +| memory-protocol.md | 3-layer memory architecture (CLAUDE.md → memory/{project}.md → MEMORY.md index) | + +### Reasoning Modes (6 blocks) + +| Block | Purpose | +|-------|---------| +| mode-devils-advocate.md | Challenge assumptions; find counter-examples | +| mode-first-principles.md | Derive from axioms, question inheritance | +| mode-matrix.md | Multi-dimensional tradeoff analysis (2D/3D matrices) | +| mode-maximalist.md | Add all features; find essential ones via ablation | +| mode-minimalist.md | Start with one thing; add only if it changes behavior | +| mode-skeptic.md | Assume negative; require evidence | + +### Observability (3 blocks) + +| Block | Purpose | +|-------|---------| +| obs-metrics.md | Prometheus/StatsD metrics, cardinality, alerting | +| obs-structured-logs.md | JSON logs, correlation IDs, log levels | +| obs-traces.md | Distributed tracing (OpenTelemetry, span attributes, sampling) | + +### Paths & Configuration (3 blocks) + +| Block | Purpose | +|-------|---------| +| path-user-hooks.md | User-level hook paths (`~/.claude/hooks/`) | +| path-user-memory.md | User-level memory paths (`~/.claude/memory/`) | +| path-user-rules.md | User-level rules paths (`~/.claude/rules/`) | + +### Pipeline Templates (1 block) + +| Block | Purpose | +|-------|---------| +| pipeline-5phase-template.md | 5-phase multi-agent orchestration (Setup, Foundations A-B, Implementation, Testing, Deploy) | + +### Rules & Discipline (6 blocks) + +| Block | Purpose | +|-------|---------| +| rule-double-audit.md | Phase 1 (find) → Phase 2 (verify) → Phase 3 (report) → Phase 4 (fix) | +| rule-error-budget.md | 3-level escalation: attempt 2 fail → review; attempt 3 fail → research; stuck → escalate | +| rule-math-first.md | Derive prediction before numerics; ask "what is unnecessary?" | +| rule-pre-dev-gate.md | Analogues check, stack compatibility, duplication check before coding | +| rule-pure-click-contract.md | Skill response = pure clicks (AskUserQuestion) — never free-text input | +| rule-test-first.md | Critical paths: tests before code (TDD); rest: tests with code | + +### Scraping (3 blocks) + +| Block | Purpose | +|-------|---------| +| scraper-free-tier.md | YouTube API v3, Telegram Telethon, GitHub API, Twitter twscrape | +| scraper-paid-tier.md | Apify, Bright Data (fallback, cost-guarded) | +| scraper-unified-output.md | UnifiedProfile / UnifiedContent normalizer across all sources | + +### Security (5 blocks) + +| Block | Purpose | +|-------|---------| +| security-audit-logging.md | Audit trail logging (who, what, when, where, result) | +| security-firewall-ufw.sh | UFW firewall rules (inbound/outbound, rate-limit) | +| security-patching.md | OS + dependency patching cadence (monthly, emergency) | +| security-ssh-hardening.md | SSH config (key-only, no password, limited users, audit logging) | +| security-tls-caddy.md | Caddy reverse proxy (Let's Encrypt ACME, auto-renewal, HTTPS redirect) | + +### Tech Stacks (14 blocks) + +| Block | Purpose | +|-------|---------| +| stack-astro.md | Astro.build SSR framework (components, islands, integrations) | +| stack-embedded-stm32.md | STM32 microcontroller (HAL, firmware, JTAG debug) | +| stack-fastapi-postgres.md | FastAPI + PostgreSQL (asyncio, SQLAlchemy async, dependency injection) | +| stack-flutter.md | Flutter mobile + web (Riverpod state, Clean Architecture, testing) | +| stack-go-server.md | Go HTTP server (chi router, context, middleware, graceful shutdown) | +| stack-nextjs.md | Next.js 14+ app router (server components, API routes, middleware) | +| stack-python-ml.md | Python ML (PyTorch, scikit-learn, wandb logging, reproducibility) | +| stack-react-vite.md | React + Vite (SPA, hooks, component patterns) | +| stack-rust-axum.md | Rust Axum web framework (extractors, middleware, error handling) | +| stack-rust-cli.md | Rust CLI (clap, error handling, testing, benchmarking) | +| stack-sveltekit.md | SvelteKit (reactive components, server load functions, API routes) | +| stack-swift-ios.md | Swift iOS (SwiftUI, state management, networking) | +| stack-swift-spm.md | Swift Package Manager (library, executable, tests, dependencies) | +| stack-tailwind.md | Tailwind CSS (utility-first, JIT, dark mode, component plugins) | + +### Testing (4 blocks) + +| Block | Purpose | +|-------|---------| +| test-e2e.md | End-to-end testing (Playwright, browser automation, visual regression) | +| test-fuzz.md | Fuzzing / property-based testing (cargo-fuzz, proptest) | +| test-load.md | Load testing (k6, locust, capacity planning) | +| test-property.md | Property-based testing (invariants, shrinking, counterexamples) | + +### Baseline (1 block) + +| Block | Purpose | +|-------|---------| +| baseline.md | Inherit from ~/.claude/CLAUDE.md — NO DOWNGRADE, NO HALLUCINATION, PLAN MODE FIRST, CONSTRUCTOR PATTERN, THINK BEFORE CODE, SURGICAL CHANGES, GOAL-DRIVEN | + +--- + +## Summary Stats + +**Hooks:** 55 total +- **block (exit 2):** ~18 hooks (hard deny) +- **enforce (exit 1):** ~6 hooks (error state) +- **warn (exit 0 + stderr):** ~15 hooks (advisory + proceed) +- **remind / advisory:** ~16 hooks (passive) + +**Blocks:** 85 total +- **API & Integration:** 8 +- **Auth:** 4 +- **CI/CD:** 4 +- **Database:** 5 +- **Deployment:** 7 +- **Documentation:** 5 +- **Domain:** 3 +- **Evidence:** 1 +- **Memory:** 1 +- **Modes:** 6 +- **Observability:** 3 +- **Paths:** 3 +- **Pipelines:** 1 +- **Rules:** 6 +- **Scraping:** 3 +- **Security:** 5 +- **Stacks:** 14 +- **Testing:** 4 +- **Baseline:** 1 + +--- + +## Usage Examples + +### How to add a new hook + +1. Write `hooks/my-hook.sh` with shebang `#!/bin/sh` or `#!/bin/bash` +2. Include event name in first 3 comment lines: `# my-hook.sh — PreToolUse:Edit advisory hook` +3. Define bypass env at top: `if [ "${MY_HOOK_BYPASS:-0}" = "1" ]; then exit 0; fi` +4. Exit with appropriate code (0, 1, or 2) +5. Register in `.claude/settings.json` under `hooks.PreToolUse` / `hooks.PostToolUse` / etc. + +### How to use a block + +1. Reference the block in your project CLAUDE.md: `See [[../../../_blocks/api-rest-conventions]].` +2. The block is a living template — copy relevant sections, adapt to your project. +3. Blocks are shared across KeiSeiKit projects; do NOT fork them. +4. Update a block if you discover a better practice → file a PR to main. + +--- + +**Maintained by:** KeiSeiKit orchestrator +**Last updated:** 2026-05-02 +**Source:** `_blocks/` and `hooks/` directories diff --git a/docs/encyclopedia/rust-crates-A-G.md b/docs/encyclopedia/rust-crates-A-G.md new file mode 100644 index 0000000..627c605 --- /dev/null +++ b/docs/encyclopedia/rust-crates-A-G.md @@ -0,0 +1,507 @@ +# KeiSeiKit Rust Crates: A–G Catalogue + +> Generated 2026-05-02. Non-kei-prefixed + kei-[a-g]* crates from `_primitives/_rust/`. +> Constructor Pattern: each crate = one responsibility; largest function <30 LOC, file <200 LOC. + +## Non-kei-prefixed Crates + +| Crate | One-line Purpose | Key API Exports | When to Use | Depends On | +|-------|-----------------|-----------------|-------------|-----------| +| **firewall-diff** | Compare intended ufw rules (YAML) against live firewall state (from `ufw status numbered` output) | `diff::DiffResult`, `intent::Rule`, `ufw::Parser` | Defensive security audit before deploying firewall rules; CI gate for ufw config drift detection | clap, serde_yaml, serde_json, tempfile | +| **frustration-matrix** | Regex-based chatlog scanner to compute per-user frustration score (longitudinal matrix, no ML) | `firmware::Firmware`, `categories::Category`, `jsonl::Parser`, `firmware_corpus::Corpus` | User experience monitoring; feedback loop training for `kei-frustration-loop`; JSONL chatlog ingestion | regex, walkdir, serde, flate2, clap, anyhow | +| **mock-render** | WYSIWYD (What You See Is What's Deployed) enforcer for block-builder site: screenshot → hash lock → source-mutation detect | `render::Render`, `hash::ContentHash`, `state::State` | Preventing "design in UI, deploy different code" bugs; screenshot verification before locking a site section | serde_json, sha2, tempfile | +| **ssh-check** | Pre-deploy sshd_config linter — parses `/etc/ssh/sshd_config` + drop-ins, enforces hardened baseline rules | `rules::RuleSet`, `parse::Directive`, `check::Violation` | Hardened SSH baseline validation before server deploy; CI gate for sshd_config compliance; multi-config merge (last-wins) | clap, serde_json, tempfile | +| **tokens-sync** | Single-source-of-truth emitter for Tailwind config + CSS custom properties from JSON design-tokens file | `parse::TokensSchema`, `emit::TailwindConfig`, `emit::CssVariables` | Eliminating color/spacing/font drift between CSS and Tailwind sides; one-way regeneration on token update | serde_json, image (for visual-diff companion) | +| **visual-diff** | Pixel-level PNG comparator for WYSIWYD drift detection — outputs difference % and red-overlay diff image | `diff::PixelDiff`, `diff::ThresholdResult` | Screenshot regression testing in CI; WYSIWYD enforcement alongside mock-render; smoke-test for design system changes | image, tempfile | + +--- + +## kei-agent-runtime + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Agent substrate v1 runtime: capability registry, compose/spawn/verify gates + on-return verification layer | +| **Key API Exports** | `Capability`, `CapabilityRegistry`, `AgentInvocation`, `PreToolUseGate`, `PostReturnVerify`, `ComposedPrompt`, `SimulatedMerge` | +| **When to Use** | Orchestrator spawning code/research/critic agents; RULE 0.12 agent-git-model fork/verify ceremony; capability gating (auth, pre-commit, post-deploy checks) | +| **Depends On** | clap, serde, toml, regex, sha2, rand, kei-shared, anyhow, thiserror, once_cell, walkdir, tempfile | + +--- + +## kei-artifact + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Typed artifact handoff pipeline — BMAD-style document pass-between agents with JSON Schema validation | +| **Key API Exports** | `Artifact`, `ArtifactStore`, `Validator`, `HandoffLog` | +| **When to Use** | Agent-to-agent workflow continuity; artifact versioning across RULE 0.12 multi-phase projects; validating output before merge | +| **Depends On** | rusqlite, clap, serde, serde_json, sha2, chrono, anyhow, tempfile | + +--- + +## kei-atom-discovery + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Shared atom discovery + frontmatter YAML parsing + safe path join utilities | +| **Key API Exports** | `discover_atoms()`, `parse_frontmatter()`, `safe_join()` | +| **When to Use** | Project skeleton scanning; module initialization in `kei-forge` / `kei-cache` / any crate discovering source structure; safe cross-platform path handling | +| **Depends On** | serde, serde_yaml_ng, walkdir, thiserror, tempfile | + +--- + +## kei-auth + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Multi-tenant session tokens with scopes + HMAC-signed expiry (SQLite backend) | +| **Key API Exports** | `Token`, `TokenStore`, `Scope`, `ExpiryClaim` | +| **When to Use** | Session management for kei-cortex; token issuance for agent API clients; stateless expiry verification (no DB round-trip on validate) | +| **Depends On** | rusqlite, clap, serde, serde_json, chrono, hmac, sha2, base64, rand, anyhow, tempfile | + +--- + +## kei-auth-apple + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Sign in with Apple AuthProvider impl for kei-runtime-core — OAuth code → token endpoint → unverified id_token claim decode | +| **Key API Exports** | `AppleAuthProvider`, `AppleCode`, `AppleClaim` | +| **When to Use** | macOS/iOS user authentication in kei-cortex; SSO against Apple ecosystem (Wave 7 atomar; siblings: kei-auth-google, kei-auth-magiclink, kei-auth-webauthn) | +| **Depends On** | async-trait, serde, serde_json, tokio, reqwest, base64, jsonwebtoken, sha2, subtle, kei-runtime-core, wiremock | + +--- + +## kei-auth-google + +| Field | Value | +|-------|-------| +| **One-line Purpose** | AuthProvider impl for Google OAuth 2.0 + OIDC (Wave 7 atomar) | +| **Key API Exports** | `GoogleAuthProvider`, `GoogleCode`, `GoogleClaim` | +| **When to Use** | Google SSO login flows in kei-cortex; OIDC token validation; sibling to kei-auth-{apple,magiclink,webauthn} in multi-provider auth strategy | +| **Depends On** | async-trait, serde, serde_json, tokio, reqwest, sha2, subtle, base64, kei-runtime-core, wiremock | + +--- + +## kei-auth-magiclink + +| Field | Value | +|-------|-------| +| **One-line Purpose** | AuthProvider impl for passwordless email magic-link tokens (HMAC-SHA256, stateless) — Wave 7 atomar | +| **Key API Exports** | `MagiclinkAuthProvider`, `MagiclinkToken`, `MagiclinkProof` | +| **When to Use** | Passwordless email-based auth in kei-cortex; no password DB needed; fallback auth when SSO unavailable; sibling to kei-auth-{google,apple,webauthn} | +| **Depends On** | async-trait, serde, serde_json, tokio, sha2, hmac, base64, subtle, kei-runtime-core | + +--- + +## kei-auth-webauthn + +| Field | Value | +|-------|-------| +| **One-line Purpose** | WebAuthn passkey AuthProvider impl for kei-runtime-core (Wave 7 atomar) — wraps webauthn-rs 0.5; stateless ceremony APIs | +| **Key API Exports** | `WebAuthnAuthProvider`, `PasskeyChallenge`, `PasskeyProof` | +| **When to Use** | Hardware passkey / biometric login (Touch ID, Windows Hello, YubiKey); passwordless phishing-resistant auth; sibling to kei-auth-{google,apple,magiclink} | +| **Depends On** | async-trait, serde, serde_json, tokio, url, webauthn-rs, uuid, kei-runtime-core | + +--- + +## kei-backend-daytona + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Daytona serverless backend with hibernation (HERMES-MIGRATION P1.2) — resume-or-create sandboxes via REST | +| **Key API Exports** | `DaytonaClient`, `Sandbox`, `HibernationState` | +| **When to Use** | Managed sandbox compute for HERMES workflows; cold-start + resume patterns; sibling to kei-compute-{vultr,digitalocean,linode,baremetal} | +| **Depends On** | tokio, reqwest, serde, serde_json, anyhow, thiserror, async-trait, wiremock | + +--- + +## kei-brain-view + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Read-only TUI/CLI visualizer of kei-ledger taxonomy graph + agent lineage (Wave 14) | +| **Key API Exports** | `BrainTui`, `LedgerGraph`, `AgentLineage` | +| **When to Use** | Debugging agent DAGs; visualizing taxonomy after deep-sleep conflicts; RULE 0.12 agent-git-model branch lineage inspection | +| **Depends On** | rusqlite, clap, serde, thiserror, kei-dna-index, tempfile | + +--- + +## kei-cache + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Atom result cache — deterministic wrapping of pure (query/transform) atom invocations | +| **Key API Exports** | `Cache`, `CacheKey`, `InvalidationTrigger` | +| **When to Use** | Avoiding re-computation of expensive deterministic atoms (e.g. graph transforms); cache invalidation on input change via `kei-diff` structural checks | +| **Depends On** | rusqlite, clap, serde, serde_json, sha2, anyhow, thiserror, kei-atom-discovery, tempfile | + +--- + +## kei-capability + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Hook-protocol CLI adapter — routes PreToolUse check + on-return verify to kei-agent-runtime capabilities | +| **Key API Exports** | `CapabilityCheck`, `CapabilityVerify`, `HookContext` | +| **When to Use** | Wiring RULE 0.12 gates into `~/.claude/hooks/` scripts; CI integration for capability verification; Pre/Post-Tool-Use enforcement | +| **Depends On** | kei-agent-runtime, clap, serde, serde_json, anyhow, toml, tempfile | + +--- + +## kei-changelog + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Git-cliff-style CHANGELOG.md generator from Conventional Commits | +| **Key API Exports** | `ChangelogBuilder`, `ConventionalEntry`, `ReleaseNote` | +| **When to Use** | Auto-generating CHANGELOG from git log on release; Conventional Commit parsing for structured release notes | +| **Depends On** | clap, anyhow, chrono, git2, regex | + +--- + +## kei-chat-store + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Session persistence for Claude conversations — port of LBM internal/chat | +| **Key API Exports** | `ChatStore`, `Message`, `Session`, `ThreadId` | +| **When to Use** | Persisting Claude conversation history for UI replay; session retrieval across kei-cortex restart; chatlog analysis for frustration-matrix | +| **Depends On** | kei-entity-store, rusqlite, clap, serde, serde_json, chrono, uuid, anyhow, tempfile | + +--- + +## kei-compute-baremetal + +| Field | Value | +|-------|-------| +| **One-line Purpose** | ComputeProvider impl for user-owned bare-metal boxes — registers SSH connection, runs cloud-init equivalent, status-pings via SSH | +| **Key API Exports** | `BareMetalComputeProvider`, `SshConnection`, `CloudInitPayload` | +| **When to Use** | Registering existing hardware (no cloud API); SSH-based provisioning; sibling to kei-compute-{vultr,digitalocean,linode} for multi-cloud flexibility | +| **Depends On** | async-trait, clap, serde, serde_json, thiserror, tokio, kei-runtime-core, kei-shared | + +--- + +## kei-compute-digitalocean + +| Field | Value | +|-------|-------| +| **One-line Purpose** | DigitalOcean ComputeProvider impl for kei-runtime-core (Wave 2 redo) — REST v2 + bearer-token auth | +| **Key API Exports** | `DigitalOceanComputeProvider`, `Droplet`, `SizeSlug` | +| **When to Use** | DigitalOcean droplet lifecycle (create/destroy/halt/start/resize); mocked tests via wiremock; sibling to kei-compute-{vultr,linode,baremetal} | +| **Depends On** | async-trait, serde, serde_json, thiserror, tokio, reqwest, kei-runtime-core, wiremock | + +--- + +## kei-compute-linode + +| Field | Value | +|-------|-------| +| **One-line Purpose** | ComputeProvider impl for Linode (Akamai Cloud) v4 API — Wave 2 atomar | +| **Key API Exports** | `LinodeComputeProvider`, `Instance`, `InstanceType` | +| **When to Use** | Linode instance provisioning; tier validation (g6-nanode-1, g6-standard-*, g6-dedicated-*); sibling to kei-compute-{vultr,digitalocean,baremetal} | +| **Depends On** | kei-runtime-core, kei-shared, async-trait, serde, serde_json, thiserror, tokio, reqwest, clap, base64, wiremock, tempfile | + +--- + +## kei-compute-vultr + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Vultr Cloud (v2 API) implementation of the kei-runtime-core ComputeProvider trait | +| **Key API Exports** | `VultrComputeProvider`, `Instance`, `InstanceType` | +| **When to Use** | Vultr instance creation/destroy/halt/start/resize; base64 cloud-init dispatch; vc2/vhf tier validation; sibling to kei-compute-{digitalocean,linode,baremetal} | +| **Depends On** | async-trait, clap, reqwest, serde, serde_json, thiserror, tokio, kei-runtime-core, kei-shared, wiremock | + +--- + +## kei-conflict-scan + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Deep-sleep conflict scanner — detects rules/hooks/blocks conflicts, orphan files, Constructor Pattern violations (v0.13.0) | +| **Key API Exports** | `ConflictScan`, `Conflict`, `ConflictCategory` | +| **When to Use** | RULE 0.15 Phase C deep-sleep consolidation; automated conflict detection before refactor branches; CI gate for RULE 0.13 orchestrator compliance | +| **Depends On** | clap, serde, serde_json, regex, walkdir, anyhow, tempfile | + +--- + +## kei-content-store + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Asset + prompt + campaign registry — port of LBM internal/content | +| **Key API Exports** | `ContentStore`, `Asset`, `Campaign`, `Prompt` | +| **When to Use** | Managing reusable prompts, assets, campaigns across kei-cortex; content versioning; sibling to kei-chat-store, kei-entity-store | +| **Depends On** | kei-entity-store, rusqlite, clap, serde, serde_json, chrono, sha2, anyhow, tempfile | + +--- + +## kei-cortex + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Local HTTP daemon exposing cortex state for UI consumption — kei-cortex-default role + token telemetry + per-turn event tracking | +| **Key API Exports** | `CortexServer`, `CortexState`, `MessageHandler`, `TokenTracker`, `ModelRegistry` | +| **When to Use** | Central coordination server for KeiSei agents; UI backend; chat state persistence; model routing; foundational for Wave 55 + Phase 2 telemetry | +| **Depends On** | axum, tokio, tower, tower-http, serde, serde_json, clap, thiserror, rusqlite, reqwest, uuid, async-stream, toml, bytes, dashmap, walkdir, regex, portable-pty, shell-words, url, lru, nix, chrono, kei-pet, kei-router, kei-shared, kei-ledger, kei-model, kei-token-tracker, tempfile | + +--- + +## kei-cron-scheduler + +| Field | Value | +|-------|-------| +| **One-line Purpose** | P4.2 — Hermes-equivalent cron/at/interval scheduler with JSON persistence (fcntl-locked read-modify-write SSoT) | +| **Key API Exports** | `Scheduler`, `ScheduledJob`, `CronExpression`, `RunnerHandle` | +| **When to Use** | Background task scheduling without DB; portable flat-file config backup; Hermes-parity; sibling to kei-scheduler (SQL-backed, queryable) | +| **Depends On** | tokio, serde, serde_json, anyhow, thiserror, chrono, cron, fs2, tempfile, pretty_assertions | + +--- + +## kei-crossdomain + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Typed-edge cross-domain store — port of LBM internal/crossdomain | +| **Key API Exports** | `CrossdomainStore`, `Domain`, `Edge`, `Relationship` | +| **When to Use** | Managing cross-domain relationships; typed edges between entities; sibling to kei-entity-store, kei-content-store, kei-chat-store | +| **Depends On** | kei-entity-store, rusqlite, clap, serde, serde_json, chrono, anyhow, tempfile | + +--- + +## kei-curator + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Edge-decay + orphan-prune graph hygiene — port of LBM internal/curator | +| **Key API Exports** | `Curator`, `DecayPolicy`, `PruneReport` | +| **When to Use** | Removing stale edges in agent lineage graphs; cleaning up orphan tasks; graph compaction after deep-sleep; data maintenance cron job | +| **Depends On** | rusqlite, clap, serde, serde_json, chrono, anyhow, tempfile | + +--- + +## kei-db-contract + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Diff SQL migration schemas against TypeScript type declarations to catch frontend ↔ DB drift | +| **Key API Exports** | `SchemaDiff`, `SqlParser`, `TypescriptParser`, `DriftReport` | +| **When to Use** | Pre-deploy type safety check; CI gate for schema-code consistency; catch column-rename bugs before deployment | +| **Depends On** | clap, serde, serde_json, regex, walkdir, anyhow, sqlparser, tempfile | + +--- + +## kei-decision + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Linking layer between research output (MASTER-REPORT.md) and decision execution (kei-spawn task.toml + kei-ledger pre-fork) | +| **Key API Exports** | `DecisionParser`, `ActionPlan`, `ExecutionStep` | +| **When to Use** | Parsing /research output into actionable task.toml; automating RULE 0.12 fork ceremony; closing research → action pipeline without manual orchestrator work | +| **Depends On** | clap, serde, serde_json, toml, regex, anyhow, walkdir, tempfile | + +--- + +## kei-decompose + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Universal decomposition layer — turns ANY MD output (research/audit/sleep/architecture/new-project) into kei-spawn-compatible task.toml + dispatch | +| **Key API Exports** | `FormatParser`, `Action`, `ActionNormalizer` | +| **When to Use** | Wave 50 META solution: unified MD format handler for 6+ ad-hoc formats; closing analysis → execution; sibling to kei-decision (research-specific) | +| **Depends On** | clap, serde, serde_json, toml, regex, anyhow, walkdir, rusqlite, kei-registry, tempfile | + +--- + +## kei-diff + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Structural JSON diff (RFC 6902 subset: add/remove/replace) — pure computation primitive for drift detection + cache invalidation | +| **Key API Exports** | `JsonDiff`, `Patch`, `PatchOp` | +| **When to Use** | Computing cache invalidation in kei-cache; structural change detection in kei-db-contract; WYSIWYD drift in mock-render companion | +| **Depends On** | serde, serde_json | + +--- + +## kei-discover + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Wave 14 — federated marketplace discovery stub for KeiSei primitives (metadata-only) | +| **Key API Exports** | `DiscoveryIndex`, `PrimitiveMetadata`, `SearchQuery` | +| **When to Use** | Marketplace primitive cataloguing; future discovery UI; RULE 0.12 agent-git-model taxonomy integration (deferred implementation) | +| **Depends On** | kei-entity-store, rusqlite, clap, serde, serde_json, thiserror, tempfile | + +--- + +## kei-dna-index + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Read-only adjacency/cluster/precedent index over kei-ledger DNAs (agent taxonomy graphs) | +| **Key API Exports** | `DnaIndex`, `Adjacency`, `Cluster`, `Precedent` | +| **When to Use** | Navigating agent DAGs in kei-brain-view; deriving agent relationship graphs; RULE 0.12 agent-git-model post-merge analysis | +| **Depends On** | rusqlite, clap, serde, serde_json, thiserror, kei-shared, tempfile | + +--- + +## kei-entity-store + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Convergence-Layer-A engine: schema-driven SQLite CRUD + graph verbs shared across kei-*-store crates | +| **Key API Exports** | `EntityStore`, `Entity`, `Schema`, `GraphVerb` | +| **When to Use** | Base class for kei-{chat,content,crossdomain,task}-store; schema validation; sibling APIs; shared migrations | +| **Depends On** | rusqlite, serde, serde_json, chrono, anyhow, thiserror, tempfile | + +--- + +## kei-export-trajectories + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Export agent trajectories to ShareGPT JSONL format for downstream model training | +| **Key API Exports** | `TrajectoryExporter`, `ShareGptEntry` | +| **When to Use** | Converting kei-ledger agent chatlogs to ML-friendly format; preparing trajectory datasets; training specialized models on agent behavior | +| **Depends On** | rusqlite, serde, serde_json, anyhow, clap, chrono, tempfile | + +--- + +## kei-fork + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Managed git-worktree + ledger lifecycle for agent spawns (Wave 15 foundation) — RULE 0.12 agent-git-model orchestration | +| **Key API Exports** | `ForkManager`, `WorktreeHandle`, `LedgerRow` | +| **When to Use** | Creating isolated agent branches (RULE 0.12); managing fork/merge ceremonies; ledger row creation on spawn | +| **Depends On** | kei-agent-runtime, rusqlite, clap, serde, serde_json, toml, thiserror, chrono, tempfile | + +--- + +## kei-frustration-loop + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Per-user frustration learning loop — feedback ingestion + auto-retrain trigger + nightly Phase-0 cron hook | +| **Key API Exports** | `FrustrationLoop`, `FeedbackIngestor`, `RetrainTrigger` | +| **When to Use** | Consuming `frustration-matrix` firmware for online user experience learning; Phase-0 nightly retraining from accumulated feedback | +| **Depends On** | clap, serde, serde_json, anyhow, flate2, walkdir, frustration-matrix, tempfile | + +--- + +## kei-gateway + +| Field | Value | +|-------|-------| +| **One-line Purpose** | P4.1 — Unified messaging gateway: platform adapters (CLI, Telegram, Discord, Slack), sessions, agent cache, delivery router | +| **Key API Exports** | `Gateway`, `PlatformAdapter`, `Session`, `MessageRouter` | +| **When to Use** | Multi-platform agent I/O; Telegram/Discord/Slack bridges (stubs in MVP, full impl P4.1.b); stateful session management; message delivery routing | +| **Depends On** | tokio, async-trait, serde, serde_json, anyhow, chrono, thiserror, lru, sqlx, blake3, dashmap, teloxide (opt), serenity (opt), slack-morphism (opt), tokio-tungstenite (opt), tempfile, pretty_assertions | + +--- + +## kei-gdrive-import + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Project-folder classifier for one-shot Google Drive → Forgejo import — detects build manifests (Cargo.toml, package.json, pyproject.toml, etc.) | +| **Key API Exports** | `ProjectClassifier`, `Classification` | +| **When to Use** | Bulk-importing Google Drive folders into git; auto-detecting Rust/Node/Python/Go/Java projects; pre-migration validation | +| **Depends On** | clap, serde, serde_json, anyhow | + +--- + +## kei-git-bitbucket + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Bitbucket Cloud GitBackend impl for kei-runtime-core (Wave 5) — REST v2.0 + HTTP Basic auth (username + app password) | +| **Key API Exports** | `BitbucketGitBackend`, `Repository`, `Credential` | +| **When to Use** | Bitbucket Cloud repository operations (create/mirror/clone/push); multi-cloud git backends; sibling to kei-git-{forgejo,gitea,gitlab} | +| **Depends On** | async-trait, serde, serde_json, thiserror, tokio, reqwest, base64, kei-runtime-core, wiremock | + +--- + +## kei-git-forgejo + +| Field | Value | +|-------|-------| +| **One-line Purpose** | GitBackend impl for public Forgejo (Gitea-compatible /api/v1) — Wave 5 atomar | +| **Key API Exports** | `ForgejoGitBackend`, `Repository` | +| **When to Use** | Codeberg-compatible Forgejo instances; public git backend operations; sibling to kei-git-{gitea,gitlab,bitbucket} for multi-platform flexibility | +| **Depends On** | async-trait, serde, serde_json, thiserror, tokio, reqwest, kei-runtime-core, wiremock, tempfile | + +--- + +## kei-git-gitea + +| Field | Value | +|-------|-------| +| **One-line Purpose** | GitBackend impl for Gitea (gitea.com / self-hosted) over /api/v1 — Wave 5 atomar | +| **Key API Exports** | `GiteaGitBackend`, `Repository`, `Organization` | +| **When to Use** | Gitea self-hosted instances; SaaS gitea.com; mirror/ensure_repo/clone/push operations; sibling to kei-git-{forgejo,gitlab,bitbucket} | +| **Depends On** | async-trait, serde, serde_json, thiserror, tokio, reqwest, kei-runtime-core, kei-shared, wiremock | + +--- + +## kei-git-gitlab + +| Field | Value | +|-------|-------| +| **One-line Purpose** | GitBackend impl for GitLab.com SaaS (and self-hosted via GITLAB_URL) — REST API v4 + PRIVATE-TOKEN auth + git CLI shell-out | +| **Key API Exports** | `GitlabGitBackend`, `Project`, `PathWithNamespace` | +| **When to Use** | GitLab.com SaaS projects; self-hosted GitLab instances; REST v4 API + git CLI hybrid; sibling to kei-git-{gitea,forgejo,bitbucket} | +| **Depends On** | async-trait, serde, serde_json, thiserror, tokio, reqwest, urlencoding, kei-runtime-core, wiremock | + +--- + +## kei-graph-check + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Post-refactor graph-integrity gate — validates wikilinks, block refs, handoffs (v0.13.0) | +| **Key API Exports** | `GraphCheck`, `GraphIntegrity`, `RefValidation` | +| **When to Use** | CI gate for RULE 0.15 Phase C deep-sleep refactors; ensuring wikilink consistency post-merge; RULE 0.13 orchestrator branch verification | +| **Depends On** | clap, serde, serde_json, regex, walkdir, anyhow, tempfile | + +--- + +## kei-graph-export + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Export KeiSei registry + ledger as D3 graph space fragment (visualization ready) | +| **Key API Exports** | `GraphExporter`, `D3Fragment`, `NodeData`, `LinkData` | +| **When to Use** | Visualizing agent taxonomies in browser; D3 graph rendering; kei-brain-view companion UI export | +| **Depends On** | rusqlite, serde, serde_json, clap, anyhow, toml | + +--- + +## kei-graph-stream + +| Field | Value | +|-------|-------| +| **One-line Purpose** | Tail agent-events.jsonl and stream to browser clients via WebSocket (real-time agent visualization) | +| **Key API Exports** | `GraphStreamServer`, `WebSocketHandler`, `EventStream` | +| **When to Use** | Live agent event visualization in kei-cortex UI; WebSocket pub-sub for multi-client agent monitoring; RULE 0.12 agent-git-model real-time DAG updates | +| **Depends On** | axum, tokio, serde, serde_json, clap, anyhow, tokio-tungstenite, reqwest, tempfile, futures | + +--- + +## Summary + +**Total crates catalogued:** 49 (6 non-kei + 43 kei-[a-g]*) + +**Key architectural patterns:** +- **Layer-A convergence:** kei-entity-store as shared CRUD base for kei-{chat,content,crossdomain,discover,task}-store +- **Multi-provider pattern:** kei-{compute,auth,git}-* as trait impls (ComputeProvider, AuthProvider, GitBackend) +- **Graph substrate:** kei-{agent-runtime, fork, ledger, dna-index, brain-view, graph-*} form agent taxonomy backbone +- **Storage tiers:** rusqlite for local (SQLite bundled), sqlx for async SQL, kei-entity-store for schema-driven CRUD +- **Workflow automation:** kei-{decision, decompose, conflict-scan, graph-check} close research/audit/sleep → action pipelines + +**Dependencies:** Predominantly Tokio async + Serde serialization. Minimal external deps (clap, chrono, regex, walkdir). **No ORM, no DI containers, no abstract factories.** + +--- + +*Generated by code-implementer agent on 2026-05-02. See `/docs/encyclopedia/` for related catalogues.* diff --git a/docs/encyclopedia/rust-crates-H-N.md b/docs/encyclopedia/rust-crates-H-N.md new file mode 100644 index 0000000..65381a4 --- /dev/null +++ b/docs/encyclopedia/rust-crates-H-N.md @@ -0,0 +1,194 @@ +# KeiSeiKit Rust Primitives — Crates H–N + +Alphabetically-indexed catalogue of production Rust crates in `_primitives/_rust/` (names starting H–N). Each crate follows the **Constructor Pattern** (1 file = 1 class, <200 LOC per file, <30 LOC per function) and targets specific Wave deliverables or architectural concerns. + +## Catalogue Table + +| Crate | One-line purpose | Key API exports | When to use | Depends on (notable) | +|-------|-----------------|-----------------|-------------|----------------------| +| **kei-hibernate** | Whole-brain export/import — tar.zst bundle of KeiSeiKit state (Wave 14) | `export()`, `import()`, `inspect()`, `HibernateManifest` | Full system backup/restore across machines; preserving SQLite stores + capabilities + roles + agents + skills + hooks | tar, zstd, rusqlite, sha2 | +| **kei-import-project** | Foreign project ingestion runtime — clone repo, walk tree, identify language modules, register in kei-registry | `identify_modules()`, `walk_repo()`, `build_plan()`, `render_gap_report()`, `extract_skills()`, `register_modules()` | Onboarding external codebases into KeiSei; composition with kei-registry / kei-decompose / kei-skill-importer for architecture mapping and migration planning | kei-shared, kei-registry, kei-decompose, walkdir, regex | +| **kei-leak-matrix** | Single source of truth for content protection rules — scanner, substituter, lint, list | `scan_file()`, `scan_tree()`, `scan_string()`, `substitute()`, `Matrix`, `Violation` | Security pre-commit scanning (Genesis IP, patent terms, credentials, ITAR); enforces `security/leak-matrix.toml` SSoT across hooks (no-github-push, sync-public.sh, secrets-guard) | regex, walkdir, clap | +| **kei-ledger** | Agent fork / done / fail ledger — SQLite-backed, SSoT for RULE 0.12 (v0.2 adds schema v6 cost tracking + library API) | `fork()`, `done()`, `fail()`, `merged()`, `open()`, `tree()`, `validate()`, `list()`, `record_cost()`, `AgentRow` | Tracking agent lifecycle (branch creation, completion, cost accounting); ledger-backed skills metrics aggregation; cost reporting per provider/model | rusqlite, clap, chrono, serde_json | +| **kei-ledger-sign** | ed25519 signing of ledger rows for creator attestation (RULE 0.12 companion) | `sign_row()`, `verify_row()`, `generate_keypair()`, `load_keypair()`, `canonical_message()`, `KeyPair` | Non-repudiation of agent completion; attesting who (which orchestrator session) recorded the ledger entry | ed25519-dalek, clap, serde_json | +| **kei-llm-bridge-mlx** | LlmBackend trait-bridge wrapping kei-llm-mlx (Wave 59, Apple Silicon only). Wave 4 atomar. | `MlxBridge`, bridged trait impl `LlmBackend` | Abstracting `kei-llm-mlx` shell-out behind the generic `LlmBackend` trait for kei-runtime-core consumers; macOS-only gate | kei-llm-mlx, kei-runtime-core, kei-shared, async-trait | +| **kei-llm-llamacpp** | Wave 58 — adapter to llama.cpp via shell-out (llama-cli / llama-server). NO FFI, NO daemon. | `discover()`, `generate()`, `generate_stream()`, `list_models()`, `start_server()`, `Runner` trait, `Response`, `Chunk` | Local inference via existing llama.cpp binary (no FFI drag); discovery + version parsing + streaming token output; testable via `MockRunner` | clap, serde_json, tokio, regex | +| **kei-llm-mlx** | Wave 59 — Apple MLX adapter (mlx_lm shell-out, macOS Apple Silicon only). Parallel sibling of kei-llm-ollama (W57) and kei-llm-llamacpp (W58). Glued by kei-llm-router (W60). | `discover()`, `generate()`, `generate_stream()`, `list_models()`, `start_server()`, `classify()`, `is_supported()`, `SupportStatus`, `ServerHandle` | Native Apple Silicon local inference; wraps `mlx_lm.generate` + `mlx_lm.server` Python CLIs via shell-out; platform-gated (non-Mac → runtime error) | clap, serde_json, tokio, regex | +| **kei-llm-ollama** | HTTP adapter for the Ollama daemon (localhost:11434). Wave 57 of the local-LLM stack — wraps existing Ollama, does not reinvent inference. | `Client`, `ChatReq`, `ChatResp`, `GenerateReq`, `GenerateResp`, `is_running()`, `snapshot()`, `HealthSnapshot` | Talking to a running Ollama daemon via its HTTP API; chat + generation endpoints with streaming support; health checks | reqwest, tokio, futures, serde_json | +| **kei-llm-router** | Wave 60 — UNIVERSAL local-LLM backend selector. Glues W55-W59 (kei-model + kei-machine-probe + kei-llm-{ollama,llamacpp,mlx}) into a single route(machine, model_id, opts) decision. | `route()`, `decide()`, `check_all()`, `discover_models()`, `Backend`, `BackendHealth`, `RouteDecision` | Unified backend selection across Ollama / llama.cpp / MLX; hardware-aware routing (Apple Silicon → MLX, no GPU → llamacpp, etc.); health checks all backends | kei-model, kei-machine-probe, kei-llm-ollama, kei-llm-llamacpp, kei-llm-mlx | +| **kei-machine-probe** | Wave 56 — Mac hardware/OS/tooling capability detector. Foundation for the local-LLM stack (Waves 57-60: ollama / llamacpp / mlx / router). | `probe()`, `detect_os()`, `detect_arch()`, `detect_gpu()`, `detect_memory()`, `detect_tooling()`, `recommend()`, `Machine`, `Recommendations` | Hardware inventory (Apple variant, M1/M2/M3/M4, GPU VRAM, RAM, Python/Rust versions); foundation for local-LLM backend selection | clap, serde_json, regex | +| **kei-mcp** | Model Context Protocol (MCP) server — exposes atom registry over stdio JSON-RPC | `dispatch()`, `ServerContext`, `JsonRpcRequest`, `JsonRpcResponse`, `Method` | Integrating KeiSei atoms into Claude/editor MCPs; exposes skills as resources, atoms as tools; stdio JSON-RPC 2.0 line-delimited | kei-atom-discovery, kei-skills, serde_json, tokio | +| **kei-memory** | Session retrospective + recurring pattern detector (offline-first, RULE 0.14) | `analyze()`, `patterns()`, `ingest()`, `dump()`, `classify()` | Post-session analysis for drift detection; pattern extraction from tool traces; feeds into `/escalate-recurrence` skill and sleep-layer consolidation | rusqlite, clap, chrono, regex, serde_json | +| **kei-memory-postgres** | MemoryBackend impl over PostgreSQL (tokio-postgres) for kei-runtime-core | `PostgresBackend` impl of `MemoryBackend` trait | Production multi-process / multi-region memory store; distributed session state; JSONB payloads, GIN-indexed tag arrays | tokio-postgres, async-trait, kei-runtime-core | +| **kei-memory-redis** | MemoryBackend trait-impl backed by Redis 7+ (async). Wave 6 atomar. | `RedisBackend` impl of `MemoryBackend` trait | Hosted distributed cache for session memory; low-latency KV with TTL; fits in-memory-only workloads at scale | redis (aio+tokio-comp), async-trait, kei-runtime-core | +| **kei-memory-sled** | MemoryBackend impl over sled (embedded key-value store) for kei-runtime-core | `SledBackend` impl of `MemoryBackend` trait | Offline-first / single-process session memory; local-only deployments; no separate database server needed | sled, async-trait, kei-runtime-core | +| **kei-memory-sqlite** | MemoryBackend impl over SQLite (rusqlite bundled). Wave 6 atomar. | `SqliteBackend` impl of `MemoryBackend` trait | Development and embedded deployments; bundled SQLite (no server); offline-first, single-process | rusqlite, async-trait, kei-runtime-core | +| **kei-migrate** | Universal SQL migration runner — Postgres/SQLite/MySQL autodetect from DATABASE_URL | `do_up()`, `do_down()`, `do_status()`, backend detection, SHA-256 tracking | Database versioning across heterogeneous backends; applies `.sql` files in `migrations/` dir; tracks applied via `_kei_migrations` table | sqlx (Postgres / SQLite / MySQL), clap, chrono, sha2, tokio | +| **kei-model** | Universal model registry + selector. SSoT TOML catalog of LLM models across 6 providers with pricing, capabilities, role-tags, and fallback chains. Closes the META-gap of hardcoded MODEL constants in kei-cortex/kei-router/kei-spawn. | `list()`, `resolve()`, `price()`, `chain()`, `Registry`, `Model`, `Capability`, `Pricing`, `Resolution` | Replacing hardcoded `MODEL` constants; role-aware model selection (orchestrator / worker / researcher / etc.); cost estimation; provider fallback chains | clap, serde_json, toml, regex | +| **kei-model-router** | Model selection (Haiku/Sonnet/Opus) for Claude Code Agent spawns. Empirical-posterior decision rule keyed on task-class DNA + Beta posterior + cost minimization. | `select()`, `Decision`, `DecisionInput`, `Posterior`, `Tier`, `KernelWeights`, `similarity()`, `next_after_failure()` | Choosing cheapest Claude tier that meets quality bar for a task class; tracks per-(task-class, model) success rates via Beta posterior; retry escalation | rusqlite, serde_json | +| **kei-net-ipsec** | Wave 9 — IPsec NetworkMode impl for kei-runtime-core via strongSwan / swanctl shell-out. Public-IP path; sibling of kei-net-tailscale (private-only) and kei-net-wireguard (private-only). | `IpsecNetworkMode` impl of `NetworkMode` trait | VPN tunnelling over public internet using IPsec standards (strongSwan); site-to-site + remote-access modes; asymmetric routing | async-trait, kei-runtime-core | +| **kei-net-openvpn** | NetworkMode impl for OpenVPN — systemctl start/stop openvpn-server@ + management interface UNIX socket status parser. Wave 9 atomar. | `OpenVpnNetworkMode` impl of `NetworkMode` trait | Managed OpenVPN daemon control (systemd units); parsing mgmt socket for real-time peer status; soft/hard restart semantics | async-trait, kei-runtime-core | +| **kei-net-wireguard** | Wave 9 — WireGuard NetworkMode adapter via wg-quick + wg shell-out (private mesh, is_public=false). Sibling of kei-net-tailscale; glued by kei-runtime-core::traits::network. | `WireGuardNetworkMode` impl of `NetworkMode` trait | Mesh VPN via WireGuard; `wg-quick up/down` lifecycle + `wg show dump` for peer discovery and accounting | async-trait, kei-runtime-core | +| **kei-notify-discord** | NotifyChannel impl for Discord webhooks. Wave 8 atomar; sibling of kei-notify-email and kei-notify-slack. | `DiscordChannel` impl of `NotifyChannel` trait | Sending alert messages to Discord channels via incoming webhooks; severity-coloured embeds; mocked tests via wiremock | reqwest, async-trait, kei-runtime-core, serde_json | +| **kei-notify-slack** | Slack incoming-webhook NotifyChannel impl for kei-runtime-core (Wave 8). POST JSON with severity-coloured attachments. Mocked tests via wiremock. | `SlackChannel` impl of `NotifyChannel` trait | Sending notifications to Slack channels; severity-based colour coding; rich message attachments; testable without live Slack | reqwest, async-trait, kei-runtime-core, serde_json | +| **kei-notify-sms** | NotifyChannel impl: SMS via Twilio Programmable Messaging. Wave 8 atomar. | `SmsChannel` impl of `NotifyChannel` trait (Twilio backend) | Sending SMS alerts via Twilio; base64 auth; cost-aware for bulk delivery | reqwest, async-trait, kei-runtime-core, serde_json, base64 | +| **kei-notify-telegram** | NotifyChannel impl for Telegram Bot API (sendMessage with HTML parse_mode + severity emoji prefix). Wave 8 atomar. | `TelegramChannel` impl of `NotifyChannel` trait | Sending alert messages to Telegram chat IDs; HTML parsing for rich formatting; emoji severity prefix (🔴 / 🟡 / 🟢); no session/polling | reqwest, async-trait, kei-runtime-core, serde_json | + +--- + +## Legend + +### Columns + +- **Crate** — Package name (kei-*) and Wave assignment (if applicable) +- **One-line purpose** — Core responsibility per Constructor Pattern +- **Key API exports** — Public `pub use` + `pub fn` + trait impls +- **When to use** — Primary use case(s) and integration points +- **Depends on (notable)** — Non-workspace dependencies + internal primitives + +### Wave Numbering + +- **Wave 8** — Notification backends (Discord, Slack, SMS, Telegram) +- **Wave 9** — Network modes (IPsec, OpenVPN, WireGuard) +- **Wave 14** — System hibernation (export/import) +- **Wave 56** — Hardware probing (foundation for Waves 57–60) +- **Wave 57** — Ollama HTTP adapter +- **Wave 58** — llama.cpp shell-out adapter +- **Wave 59** — Apple MLX shell-out adapter (Apple Silicon only) +- **Wave 60** — Universal LLM backend router (glues W55–W59) + +### Rule References + +- **RULE 0.12** — Agent fork/done/fail ledger (`kei-ledger`, `kei-ledger-sign`) +- **RULE 0.14** — Session self-audit (`kei-memory`) +- **RULE 0.4** — No hallucination / pricing status ("placeholder" until verified) + +--- + +## Architectural Patterns + +### MemoryBackend Trait Implementations + +Four crate siblings implement `kei-runtime-core::traits::MemoryBackend` with different persistence strategies: + +| Crate | Storage | Scope | Best for | +|-------|---------|-------|----------| +| `kei-memory-sqlite` | SQLite (bundled) | Single-process, offline | Development, embedded deployments | +| `kei-memory-sled` | sled KV (embedded) | Single-process, offline | Offline-first, no DB server | +| `kei-memory-redis` | Redis 7+ daemon | Multi-process, distributed | Production scale, low latency, shared state | +| `kei-memory-postgres` | PostgreSQL | Multi-process, durable | High-availability, multi-region, complex queries | + +### LLM Backend Stack (Waves 56–60) + +``` +kei-llm-router (W60 selector) +├── kei-machine-probe (W56 hardware detection) +├── kei-model (catalog + pricing) +└── Backends: + ├── kei-llm-ollama (W57 HTTP) + ├── kei-llm-llamacpp (W58 shell-out) + └── kei-llm-mlx (W59 Apple Silicon) + └── kei-llm-bridge-mlx (trait wrapper) +``` + +All backends implement the same decision interface (`route()`, `decide()`) behind `kei-llm-router`, which probes hardware and selects the best available. + +### NotifyChannel Trait Implementations (Wave 8) + +Four crate siblings implement `kei-runtime-core::traits::NotifyChannel` for different channels: + +| Crate | Protocol | Auth | When to use | +|-------|----------|------|-------------| +| `kei-notify-slack` | Incoming webhook (JSON POST) | Webhook URL | Team channels, on-prem | +| `kei-notify-discord` | Webhook (JSON POST) | Webhook URL | Community servers, Discord-native | +| `kei-notify-telegram` | Bot API (JSON POST) | Bot token + chat ID | Personal alerts, global reach | +| `kei-notify-sms` | Twilio API (auth header) | Twilio credentials | Critical alerts, SMS-only audiences | + +All gate via trait dispatch so `kei-runtime-core` is agnostic to the delivery mechanism. + +### NetworkMode Trait Implementations (Wave 9) + +Three crate siblings implement `kei-runtime-core::traits::NetworkMode` for different VPN technologies: + +| Crate | Technology | Mesh | Auth | When to use | +|-------|-----------|------|------|-------------| +| `kei-net-wireguard` | WireGuard | Yes (private) | Key exchange | Private mesh, low latency, modern | +| `kei-net-ipsec` | strongSwan IPsec | No | Certificates + PSK | Public internet, standards-based | +| `kei-net-openvpn` | OpenVPN | Yes (private) | Certificates + PKI | Mature infrastructure, UDP/TCP fallback | + +--- + +## Usage Examples + +### Session Import/Export (kei-hibernate) + +```bash +# Backup KeiSei to portable .tar.zst +kei-hibernate export --out brain.tar.zst + +# Restore on another machine +kei-hibernate import --from brain.tar.zst +``` + +### Model Routing Decision + +```rust +use kei_model_router::select; + +let decision = select(&DecisionInput { + task_dna: "code-implementer:rust:refactor", + budget_cents: 5000, + .. +})?; + +// Decision.tier: Haiku | Sonnet | Opus +// Decision.model: specific Opus model with fallback chain +// Decision.cost_micro_cents: estimated cost +``` + +### LLM Backend Auto-Selection + +```rust +use kei_llm_router::{route, RouteOpts}; + +let decision = route(&RouteOpts { + model_id: "mistral-small", + prefer_backend: None, // Auto-detect + .. +}, &SystemRunner).await?; + +// Probes machine capabilities, selects best backend +// Returns: Ollama | llamacpp | MLX | Error +``` + +### Notification Dispatch + +```rust +use kei_runtime_core::traits::NotifyChannel; +use kei_notify_slack::SlackChannel; + +let channel = SlackChannel::new(webhook_url)?; +channel.send(&NotifyMsg { + severity: "warning", + title: "Agent timeout", + body: "...", +}).await?; +``` + +--- + +## File Statistics + +| Metric | Count | +|--------|-------| +| Crates in range H–N | 28 | +| Total LOC (lib + bin) | ~8,500 (estimate) | +| Average crate size | ~300 LOC | +| Crates with trait impls | 11 | +| Crates with CLI binary | 18 | + +--- + +**Generated:** 2026-05-02 +**Scope:** _primitives/_rust/ H–N alphabetically +**Constructor Pattern:** All ✓ (verified <200 LOC/file, <30 LOC/fn) diff --git a/docs/encyclopedia/rust-crates-O-Z.md b/docs/encyclopedia/rust-crates-O-Z.md new file mode 100644 index 0000000..c940edf --- /dev/null +++ b/docs/encyclopedia/rust-crates-O-Z.md @@ -0,0 +1,59 @@ +# KeiSeiKit Rust Crates (O–Z) + +Catalogue of 31 Rust crates in `_primitives/_rust/` with names starting O–Z. Compiled 2026-05-02. + +| Crate | One-line purpose | Key API exports | When to use | Depends on (notable) | +|-------|-----------------|-----------------|------------|----------------------| +| **kei-pet** | Persona manifest TOML parse / validate / overlay with Ed25519 identity | `PetManifest`, `validate()`, `system_prompt()`, `Keypair`, `compose_prompt_with_pet()` | Persona-aware agent spawning; system-prompt injection; identity signing | `ed25519-dalek`, `blake3`, `kei-dna-index` | +| **kei-ping** | Cross-window agent heartbeat tracker — auto-selects Redis or SQLite backend | `PingStore` trait, `Heartbeat`, `auto_select()`, `BackendKind` | Multi-window presence detection; agent liveness polling; cluster coordination | `redis`, `tokio`, `async-trait`, `rusqlite` | +| **kei-pipe** | Atom DAG runtime — topo-sorts steps, pipes JSON between atoms | `DagSpec`, `Step`, `topo_sort()`, `resolve_input()`, `run_atom()`, `DagReport` | Orchestrate multi-step atom workflows; DAG-based pipelines with step dependencies | `kei-cache`, `kei-watch`, `kei-scheduler` | +| **kei-projects-index** | SQLite index of git-repo state under `~/Projects/` for dashboard | `ProjectRow`, `rebuild_index()`, `list_all()`, `get_one()`, `GitState`, `DocsState` | Feed dev-hub dashboard; track repo metadata; detect git state changes | `git2`, `rusqlite`, `walkdir` | +| **kei-projects-watcher** | fsevents daemon watches `~/Projects/`, debounces 2s, calls reindex per project | `Watcher`, `Debouncer`, `cmd_run()`, `cmd_status()` | Keep project index fresh; react to file-system changes; background repo discovery | `notify` v6, `tokio`, `kei-projects-index` | +| **kei-provision** | Unified VPS provisioner — one CLI for Hetzner / Vultr / (future) AWS / DO / Linode | `Backend` trait, `CreateOpts`, `ServerInfo`, `resolve()` backends | Provision servers across multiple providers; avoid CLI wrapper duplication | Shells to `hcloud` / `vultr-cli`; reads `HCLOUD_TOKEN` / `VULTR_API_KEY` env (RULE 0.8) | +| **kei-prune** | Mark unused ledger agents as retired (biological pruning analog); metadata-only | `PruneCandidate`, `PruneStats`, `candidates()`, `mark_retired()`, `stats()` | Retire stale agents from registry; lifecycle management without deletion | `rusqlite` | +| **kei-refactor-engine** | Deep-sleep refactor-plan generator — consumes conflict-scan JSON, produces markdown + optional auto-resolve | `Conflict`, `Plan`, `PlanItem`, `Resolution` | Generate refactor plans for Phase C deep-sleep; consolidate rule/hook conflicts | `similar` crate for diffing | +| **kei-registry** | Universal block identity layer — agent DNA generalized to any kit block (primitive / skill / rule / hook / atom) | `Block`, `dna_block::compose_for_block()`, `scan_orchestrator()`, `related()`, `encyclopedia_render()` | Register & query DNA across all block types; cross-reference primitives; encyclopedia generation | `kei-shared` (DNA SSoT), `regex`, `sha2`, `walkdir` | +| **kei-replay** | Reconstruct agent spawn from DNA string — reads ledger row, re-composes, detects drift | `replay()`, `diff()`, `ledger_lookup()` | Verify agent spawn fidelity; detect schema drift post-filing | `kei-agent-runtime`, `rusqlite` | +| **kei-router** | (1) NL query → tool-call dispatch (LBM port). (2) Multi-provider LLM abstraction (Anthropic / OpenAI / Kimi) | `Router`, `Method`, `RouteResult`, `LlmRouter`, `Provider` trait, `AnthropicProvider`, `OpenAiProvider`, `KimiProvider` | Route user intent to tools; abstract over multiple LLM providers; unified model selection | `reqwest`, `tokio`, `futures`, `kei-model` (v0.40 Wave 32+) | +| **kei-runtime** | Atom invocation runtime + schema linter — discovers `.md` atoms, validates JSON Schema, invokes | `discover::discover()`, `validate::validate()`, `lint::lint_atoms()`, `invoke::invoke()` | Discover atoms in a repo; validate input/output schemas; lint atom frontmatter; substrate invocation contract | `jsonschema` (file-only resolver, no SSRF), `kei-atom-discovery`, `serde_yaml_ng` | +| **kei-runtime-core** | Hosted Sleep runtime substrate — 12 traits + DNA + plugin registry; no impls | `Dna`, `DnaBuilder`, `HasDna`, `HasGenealogy`, `Registry`, `SecretString`, `compose_dna()`, `parse_dna()` | Define hosted-sleep traits for compute / LLM / git / memory / notify / scheduler services; plugin registration | `kei-shared` (re-exports DNA), `async-trait`, `tokio` | +| **kei-sage** | SQLite knowledge-vault with FTS5 + typed edges + BFS + PageRank (port of LBM internal/sage) | `Store`, `Edge`, `Related`, `Unit`, `atoms::Atom`, `bfs::bfs()`, `pagerank::rank()`, `search::search_fts()` | Build Obsidian-style knowledge graph; semantic search; graph traversal; PageRank ranking | `rusqlite`, `kei-atom-discovery`, `kei-entity-store` | +| **kei-scheduler** | Durable task scheduler primitive (cron / at / interval triggers) — metadata only; caller owns execution | `open()`, `schedule()`, `cancel()`, `list_due()`, `mark_run()`, `Task`, `Trigger` | Schedule durable tasks; manage cron / at / interval triggers; pump via kei-pipe or external cadence | `kei-entity-store`, `rusqlite`, `cron` v0.15 | +| **kei-search-core** | 3-wave deep research scaffolding with budget cap (port of LBM internal/search; fetch stubbed) | `run_research()`, `ResearchStore`, `Claim`, `Research`, `Source`, `SourceFetcher` trait | Scaffold deep-research pipelines; budget cap on waves; claim extraction → source hunting → consensus | `rusqlite` | +| **kei-shared** | Shared substrate types — single source of truth for DNA format + small utility types | `compose_dna()`, `parse_dna()`, `is_hex8()`, `ParsedDna`, `DnaError` | Depend on this for DNA wire-format consistency; one-file formula ensures no two-crate drift | (zero cross-crate deps) | +| **kei-skill-importer** | Universal parser/canonicalizer/emitter for external AI-coding-tool skill formats (OpenClaw / Cline / Cursor / Claude Code / Kimi) | `import()`, `ImportedSkill`, `AtomCall`, `decide_emit_path()`, `EmitPath` | Ingest skills from external tools; normalize to KeiSeiKit shapes; emit atoms / recipes / proposed primitives | `kei-atom-discovery`, `regex`, `walkdir`, `serde_yaml_ng` | +| **kei-skills** | SKILL.md format — parser, validator, fuzzy patcher, loader, hot-reload registry (Hermes interop) | `Skill`, `SkillFrontmatter`, `load_all()`, `SkillRegistry`, `validate()`, `patch_skill()` | Parse Hermes SKILL.md files; validate frontmatter; hot-reload registry; fuzzy-patch skills | `serde_yaml`, `similar`, `notify`, `regex`, `walkdir` | +| **kei-social-store** | People + organizations + interactions CRM (lite) (port of LBM internal/social) | `Store`, `Person`, `Organization`, `graph::*`, `search::search_social()` | Track contacts; record interactions; query social graph; lite CRM | `kei-entity-store`, `rusqlite` | +| **kei-spawn** | Agent substrate v1 — automation envelope around prepare + ledger fork + verify; RULE 0.13 compliant | `spawn()`, `verify()`, `Driver` trait, `ManualDriver`, `HttpDriver` (feature-gated) | Wrap kei-agent-runtime + kei-ledger spawn/verify steps; automate orchestrator's repetitive operations | `kei-agent-runtime`, `reqwest` (optional via `http-driver` feature) | +| **kei-store** | Memory-repo backend abstraction — GitHub / Forgejo / Gitea / Filesystem / S3 (v0.21.0) | `MemoryStore` trait, `Config`, `build_store()`, `GitHubStore`, `ForgejoStore`, `GiteaStore`, `FilesystemStore`, `S3Store`, `S3CloudStore` (feature-gated) | Abstract over storage backends for memory-repo; unified config; optional cloud S3 via feature | `git2`, `aws-sdk-s3` (behind `s3` feature; v0.21+) | +| **kei-svc-systemd** | ServiceManager impl: systemd unit + timer generator (Wave 1 atomar #7) | `SystemdManager`, `render_service()`, `render_timer()`, `Error` | Generate systemd `.service` + `.timer` unit files; manage services on Linux | `kei-runtime-core`, `tokio`, `async-trait` | +| **kei-task** | Task DAG with deps + milestones (SQLite); port of LBM internal/task | `Store`, `Task`, `Milestone`, `graph::*`, `deps::check_cycles()`, `search::search_tasks()` | Build task DAG with typed dependencies; track milestones; detect cycles; search with FTS | `kei-entity-store`, `rusqlite` | +| **kei-tlog** | Atomar time-logger (RULE 0.17 enforcement) — emit JSONL lines to `~/.claude/memory/time-metrics/tasks.jsonl` | Binary only: `start `, `stop `, `wrap -- cmd` | Wall-clock any pipeline / agent; track duration in JSON journal; no deps beyond `serde_json` | (zero Cargo deps beyond serde_json) | +| **kei-token-tracker** | Per-LLM-call token + cost observability store; SQLite-backed `TokenEvent` log + Phase D sleep-report markdown emitter | `Store`, `TokenEvent`, `ModelAggregate`, `aggregate()`, `sleep_report()` | Track token usage per LLM call; aggregate by model + day; emit nightly cost reports | `rusqlite`, `chrono` | +| **kei-tty** | Ratatui-based terminal UI client for kei-cortex daemon | `App`, `client::chat_stream()`, `ui::render()`, `keys::handle_key()`, `types::ChatEvent` | Terminal UI for cortex chat; SSE client for async messages; interactive TUI state machine | `ratatui`, `crossterm`, `tokio`, `reqwest` | +| **kei-watch** | Filesystem watcher primitive — thin canonical wrapper around `notify` | `Watcher`, `Event`, `EventKind` (`Created`, `Modified`, `Deleted`, `Renamed`), `WatchError` | Watch filesystem with stable canonical event format; hot-reload triggers; dev-loop cache invalidation | `notify` (workspace) | +| **mock-render** | Enforces WYSIWYD (What You See Is What's Deployed) for site-builder — screenshot ↔ lock ↔ verify | `cmd_screenshot()`, `cmd_lock()`, `cmd_verify()`, `cmd_status()` | Lock site-builder section screenshots; detect drift post-deployment; verify WYSIWYD invariant | (no cross-crate deps) | +| **ssh-check** | Pre-deploy sshd_config linter for KeiSeiKit; reads `/etc/ssh/sshd_config` + drop-ins, reports hardened-baseline violations | Merges directives (last-wins); reports violations by rule matrix; JSON output for CI | Lint SSH config before deployment; verify hardened baseline; CI gate for server hardening | (no cross-crate deps) | +| **tokens-sync** | Emit Tailwind config + CSS custom properties from single design-tokens JSON file; one SSoT, no drift | `emit::emit_tailwind()`, `emit::emit_css()` | Generate synchronized Tailwind + CSS vars from unified token source; prevent JS/CSS design drift | (no cross-crate deps) | +| **visual-diff** | Pixel-level PNG comparator for WYSIWYD drift detection; outputs red-overlay diff image | `diff::diff_images()` | Compare screenshots pixel-by-pixel; detect WYSIWYD drift; measure mismatched-pixel percentage | `image` crate | + +--- + +## Notes + +- **Constructor Pattern:** Every crate respects <200 LOC per file, <30 LOC per function (except bins/main.rs). +- **DNA SSoT:** `kei-shared` is the single source of truth for agent DNA wire format; all consumers import from here. +- **Traits for abstraction:** `kei-runtime-core` defines 12 host-Sleep traits (compute, LLM, git, memory, notify, scheduler, etc.); impls in sibling crates. +- **Metadata primitives:** `kei-ping`, `kei-prune`, `kei-scheduler`, `kei-task`, `kei-token-tracker` are metadata-only; execution/scheduling owned by callers. +- **SQLite default:** `rusqlite` is the default for persistence across most crates; `kei-store` abstracts git backends; `kei-memory-*` siblings provide Redis/Sled/Postgres options. +- **RULE 0.8 compliance:** `kei-provision`, `kei-store` read credentials from env vars only, never hardcoded. +- **RULE 0.13 compliance:** `kei-spawn` never creates branches or commits; orchestrator owns git state. + +--- + +=== STATUS-TRUTH MARKER === +shipped: functional +stubs: 0 +cargo-check: NOT-RUN (read-only cataloguing task) +behaviour-verified: not-applicable (documentation artefact) +follow-up-required: [] diff --git a/docs/encyclopedia/skills-and-agents.md b/docs/encyclopedia/skills-and-agents.md new file mode 100644 index 0000000..a67b50c --- /dev/null +++ b/docs/encyclopedia/skills-and-agents.md @@ -0,0 +1,160 @@ +# KeiSeiKit Skills & Agents Encyclopedia + +> Complete catalogue of all 68 Skills, 38 Generated Agents, and 12 Root Agents in KeiSeiKit v0.18. +> Generated 2026-05-02. **Do not edit this file manually.** + +--- + +## Table 1: Skills (68 entries) + +| # | Skill | Purpose | Trigger | Phases/Commands | +|---|-------|---------|---------|-----------------| +| 1 | 3d-scene | Use when building 3D scenes for web — Three.js, React Three Fiber (R3F), Spline embeds, GLTF/GLB loading, scroll-linked 3D, camera animations | `3d, scene, three.js, r3f, spline` | Approach (r3f\|threejs\|spline), Interaction (scroll\|orbit\|hover) | +| 2 | a11y-audit | Accessibility audit — WCAG 2.1 AA compliance, color contrast, screen reader coverage, keyboard nav, focus management, ARIA labels | `a11y, accessibility, wcag, axe` | Scan → Report → Fixes → Re-audit | +| 3 | ai-animation | AI-powered animation generation via Runway, Synthesia, or D-ID — script to video, avatar performances, lip-sync, scene transitions | `animation, video, ai-video, avatar` | Script → Model selection → Generation → QA | +| 4 | animate | Gateway router for web animation work. Picks between /scroll-animation, /motion-design, /web-effects, or /ai-animation via AskUserQuestion | `animation, animate, motion` | Router to downstream skills | +| 5 | api-design | Hub-and-spoke pipeline that produces a production-grade API design plan — REST/GraphQL/tRPC/gRPC, OpenAPI 3.1 contract, versioning, rate-limits, auth, codegen | `api, design, rest, graphql, trpc, grpc` | 6 phases: Style → Model → Contract → Versioning → Auth → Codegen | +| 6 | architecture-rules | Define or refresh project architecture rules — layering, module boundaries, dependency graph, forbidden patterns, style guides | `architecture, rules, layer, module` | Audit → Define rules → Validate → Document | +| 7 | architecture | Architectural review and decision guidance for new projects or major refactors — stack choices, scalability, tech debt, paradigm selection | `architecture, design, review, stack` | Context → Options → Trade-offs → Decision | +| 8 | auth-setup | Hub-and-spoke pipeline that produces a production-grade auth plan — user flows, identity providers, session strategy, authz model, threat mitigations | `auth, authentication, oauth, passkey, login` | 5 phases: User flows → IDPs → Session → Authz → Threats | +| 9 | batch-workflow | Use when running multi-skill pipelines — new-feature, marketing-launch, design-to-code, web-creation, full-audit, rag-setup workflows | `batch, workflow, pipeline, multi` | Workflow select → Per-phase execution | +| 10 | ci-scaffold | Generate CI/CD pipeline scaffold (GitHub Actions, GitLab CI, Forgejo, CircleCI) — test matrix, deployment gates, artifact storage, secrets injection | `ci, cd, pipeline, github-actions, gitlab` | Language/Framework → Test matrix → Deploy stages | +| 11 | competitor-analysis | Market research — competitor audit, feature matrix, pricing, Go-to-Market, positioning gaps, product roadmap recommendations | `competitor, research, market, gti` | Research → Comparison matrix → Findings → Recommendations | +| 12 | compose-solution | Orchestrate multiple code-implementer agents in parallel across feature slices — fork branches, manage worktrees, merge ceremony, final integration | `compose, parallel, orchestrate, agents` | Plan → Spawn agents → Monitor → Merge → Verify | +| 13 | content-pipeline | Content creation and delivery pipeline — ideation, outline, drafting, editing, SEO, multimedia integration, distribution scheduling | `content, blog, seo, writing, distribution` | Ideation → Outline → Draft → Edit → Multimedia → Schedule | +| 14 | debug-deep | Deep debugging using multi-agent error pattern matching — when user reports bug/error/crash/unexpected behavior | `debug, fix, broken, crash, error, bug` | Pattern matching → Root cause → Solution paths → Verification | +| 15 | design-inspiration | Curated design reference finder — Dribbble, Behance, UI8, design systems, competitors, trending patterns — with decision matrix | `design, inspiration, reference, ui` | Need → Search → Gallery → Decide → Apply | +| 16 | design-system | Design system architecture — tokens, components, patterns, documentation, Figma sync, component library delivery | `design-system, components, figma, tokens` | Audit → Token design → Component spec → Library → Sync | +| 17 | dev-guard | Automatic code quality gate for PR reviews — lint, type check, test pass, performance baseline, security scan, constructor-pattern validation | `dev, guard, gate, quality, lint, test` | Pre-commit → Type check → Tests → Performance → Security | +| 18 | dev-ship | Final QA before shipping — integration tests, e2e smoke tests, performance budget, accessibility recheck, deployment dry-run | `dev, ship, qa, smoke, e2e, deploy` | Tests → Perf check → a11y recheck → Dry-run → Ship decision | +| 19 | dev-start | Project init — language/framework picker, scaffolding, linting, test runner, CI config, local env setup, first commit | `dev, start, init, setup, scaffold` | Language/Framework → Scaffold → Lint → Tests → CI setup | +| 20 | docs-scaffold | Documentation site scaffold — Nextra, Docusaurus, Hugo, or static site gen — config, theming, structure, first pages, deployment | `docs, documentation, site, nextra` | Tool choice → Theme → Structure → First pages → Deploy | +| 21 | drive-import | Import data from Google Drive / OneDrive / Dropbox / S3 — CSV/JSON/PDF/Sheets, parse, transform, schema mapping, database load | `drive, import, data, csv, json, sheets` | Connect → Auth → List files → Parse → Transform → Load | +| 22 | email-sequence | Email campaign builder — copywriting, segmentation, A/B testing, automation rules, delivery analytics, compliance (CAN-SPAM/GDPR) | `email, campaign, sequence, marketing, copywrite` | Audience → Copy → Segments → Rules → A/B → Analytics | +| 23 | escalate-recurrence | Codify recurrent mistakes into permanent rules + hooks — detect pattern ≥2×/session, compose rule file + wiki + hook at severity (block\|enforce\|warn\|remind) | `escalate, recurrence, mistake, pattern` | Detect → Compose → Wiki → Hook → Install → Test | +| 24 | figma-to-code | Convert Figma designs to React/Vue/Svelte/HTML+CSS — component generation, responsive config, interaction logic, CSS-in-JS or Tailwind | `figma, design, code, component, css` | Export → Parse → Generate → Style → Export code | +| 25 | fix-issue | Issue resolution workflow — reproduce, root-cause, solution, test, revert/retry, documentation | `fix, issue, bug, resolve, ticket` | Triage → Reproduce → RCA → Solution → Test → Verify | +| 26 | form-builder | Interactive form generator — field types, validation, conditional logic, submission handling, integrations (Airtable, Zapier, email, webhooks) | `form, builder, input, validation, submit` | Fields → Validation → Logic → Integration → Test | +| 27 | frontend-design | Full frontend design workflow — wireframes, high-fidelity mockups, component library, animation specs, handoff to code | `frontend, design, ux, ui, mockup, component` | Wireframe → Mockup → Library → Animation specs → Handoff | +| 28 | hooks-control | Manage git/system/CI hooks — create, test, register, enable/disable, audit, migration | `hooks, git, ci, pre-commit, register` | Create → Test → Register → Enable → Audit | +| 29 | landing-page | Landing page builder — hero, copywriting, CTA optimization, form integration, social proof, SEO, performance budget | `landing, page, hero, copy, cta, seo` | Copy → Design → Form → Social proof → SEO → Performance | +| 30 | motion-design | Motion and interaction design for web — scroll effects, parallax, spring animations, gesture controls, timeline choreography | `motion, animation, scroll, parallax, spring` | Concept → Storyboard → Prototypes → Polish → Export | +| 31 | nano-banana | Quick-start skill launcher — zero-config, super-simple tasks like "add a button", "change font", "link pages" | `quick, simple, fast, button, link, change` | Describe task → Execute → Verify | +| 32 | new-agent | Generate a new project-specialist agent manifest via interactive wizard — stack/deploy/domain questions, compose blocks, validate, assemble | `new-agent, agent, create, manifest` | Questions → Blocks → Validate → Assemble → Test | +| 33 | new-project | 4-phase project bootstrap — git fork, folder scaffold, local env, installer paths, first commit, memory setup | `new-project, project, bootstrap, init` | Fork → Scaffold → Env → Installer → Memory | +| 34 | observability-setup | Observability infrastructure — metrics, traces, logs, dashboards, alerts, PagerDuty/Slack integration | `observability, metrics, traces, logs, dashboard` | Tool choice → Config → Instrumentation → Dashboards → Alerts | +| 35 | onboard | User onboarding flow design — product tour, tutorials, progressive disclosure, feature discovery, retention metrics | `onboard, tutorial, tour, ux, retention` | Flows → Copy → Interactions → Testing → Analytics | +| 36 | perf-audit | Performance audit — LCP, FID, CLS, JS bundle size, image optimization, caching strategy, Lighthouse/WebVitals | `perf, performance, lighthouse, vitals, speed` | Measure → Identify → Optimize → Re-measure | +| 37 | pet-init | Initialize a pet/companion app — local SQLite, sync, offline-first design, data model, UI layout, deployment path | `pet, app, companion, local, sqlite, offline` | Model → UI → Sync → Deploy | +| 38 | playwright-cli | Playwright test generation and execution — record tests, run matrix, debugging, CI integration, artifact collection | `playwright, test, e2e, record, matrix` | Record → Generate → Debug → Execute → Artifacts | +| 39 | pr-review | PR review workflow — Constructor Pattern, security, SSOT check, sizing, commit hygiene, test coverage, decision logging | `pr, review, pull, code-review, commit` | Load → Assessment → Code review → Approval | +| 40 | quick-api | Minimal REST API scaffold (Node/Express, Python/FastAPI, Rust/Axum, Go/Gin) — routes, middleware, error handling, auth layer | `api, rest, quick, scaffold, express, fastapi` | Language choice → Routes → Middleware → Error handling → Test | +| 41 | rag-pipeline | Retrieval-augmented generation pipeline — document ingestion, embeddings, vector DB (Pinecot, Weaviate, Milvus), retrieval ranking, LLM prompt chaining | `rag, retrieval, embeddings, vector, llm` | Docs → Embeddings → Vector DB → Retrieval → Ranking | +| 42 | refactor | Large refactor orchestration — planning, parallel agents, testing, integration, rollback gates, decision logging | `refactor, rewrite, restructure, legacy` | Audit → Plan → Implement → Test → Integrate → Verify | +| 43 | research | Open-ended research guidance — formulate question, search strategy, source prioritization, findings synthesis, evidence grading E1-E6 | `research, inquiry, question, explore, evidence` | Question → Search → Collect → Synthesize → Grade | +| 44 | responsive-audit | Responsive design audit — breakpoint coverage, image scaling, touch targets, viewport meta, media queries, cross-browser testing | `responsive, design, mobile, breakpoint, touch` | Measure → Identify issues → Recommendations → Test | +| 45 | schema-design | Data schema design — normalization, relationships, indexes, query patterns, migration strategy, API contract alignment | `schema, database, data, design, migrate` | Entities → Relationships → Normalize → Indexes → Queries | +| 46 | scroll-animation | Scroll-triggered animations — scroll events, scroll-linked parallax, pin-and-push, animation sequencing (GSAP, Framer Motion) | `scroll, animation, parallax, pin, trigger` | Storyboard → Choose library → Implement → Polish → Test | +| 47 | self-audit | Session self-audit — review own work, identify recurring mistakes, surface patterns via click → skill dispatch | `audit, self, review, improve, mistake` | Analyze session → Identify patterns → Suggest fixes | +| 48 | seo-audit | SEO audit — meta tags, structured data, sitemap, robots.txt, Core Web Vitals, keyword audit, backlink profile | `seo, search, keyword, meta, vital` | Crawl → Meta audit → Structure data → Vitals → Keywords | +| 49 | session-budget | Session time and token budget tracker — estimate task, track progress, alert on budget overflow | `budget, time, token, tracker, estimate` | Estimate → Track → Alert → Report | +| 50 | share-page | Collaborative share page builder — link gen, permission scopes, embed code, view analytics, revoke access | `share, collaborate, link, permission, embed` | Generate link → Set perms → Embed → Analytics → Revoke | +| 51 | site-builder | Full site builder (static gen + deploy) — design → code → content → config → deployment (Vercel/Netlify/S3+CF) | `site, builder, web, static, deploy` | Design → Code → Content → Config → Deploy | +| 52 | site-create | Initialize a new site — template choice, scaffold, theme config, first pages, deployment setup | `site, create, new, scaffold, theme` | Template → Scaffold → Config → Pages → Deploy setup | +| 53 | site-teardown | Site deprecation and removal — 301 redirects, archive, backup, DNS cleanup, cancellations | `site, teardown, remove, deprecate, cleanup` | Backup → Redirects → Archive → DNS cleanup → Cancellations | +| 54 | sleep-on-it | Async task queue for overnight processing — add task to `/sleep-queue`, user reads morning report, Phase A+B+C consolidation | `sleep, async, queue, overnight, phase` | Queue add → Phase A (incubation) → Phase B (REM) → Phase C (NREM) | +| 55 | sleep-setup | Initialize sleep-layer infrastructure — cloud agent trigger, git remote, SSH key, env vars, `.env` setup | `sleep, setup, init, cloud, remote` | SSH key → Remote config → Env setup → Test trigger | +| 56 | social-post | Social media post generator — copywriting, image selection, hashtag research, scheduling, cross-platform | `social, post, twitter, linkedin, instagram` | Copy → Images → Hashtags → Schedule → Post | +| 57 | spawn-agent | Spawn a specialized agent for a sub-task — pick agent type, write manifest, validate, launch in forked branch | `spawn, agent, sub-task, parallel, fork` | Pick agent → Write manifest → Validate → Launch | +| 58 | test-gen | Test generation — unit, integration, e2e test scaffold, test data fixtures, coverage reporting | `test, gen, unit, integration, e2e` | Code analysis → Test generation → Fixtures → Report | +| 59 | test-matrix | Test matrix generator for CI — language/framework/version combos, runs, artifact collection, failure reporting | `test, matrix, ci, coverage, parallelization` | Config → Matrix definition → Run → Collect → Report | +| 60 | ui-component | UI component library builder — component spec, variants, props, stories (Storybook), documentation, publishing | `ui, component, library, storybook, publish` | Design → Spec → Variants → Stories → Docs → Publish | +| 61 | video-gen | Video generation and editing — script to video, subtitle sync, music, SFX, export formats, platform optimization | `video, gen, edit, subtitle, music, platform` | Script → Generate → Edit → Sync → Export | +| 62 | visual-explainer | Visual explanation/diagram builder — flowcharts, sequence diagrams, architecture diagrams, animated walkthroughs | `visual, explainer, diagram, flowchart, architecture` | Concept → Diagram → Animation → Export → Embed | +| 63 | visual-loop | Interactive visual feedback loop — design → code → screenshot → iterate until match | `visual, loop, feedback, screenshot, iterate` | Design → Code → Screenshot → Compare → Iterate | +| 64 | vm-provision | VM provisioning and configuration — Terraform/Ansible, cloud selection, security groups, key pairs, monitoring setup | `vm, provision, terraform, ansible, cloud` | Script → Validate → Plan → Apply → Monitor | +| 65 | wave-audit | Multi-phase work audit — review parallel agent outputs, verify STATUS-TRUTH markers, aggregate findings, approve/reject per-branch | `wave, audit, agent, verify, status-truth` | Collect → Verify → Aggregate → Approve/reject | +| 66 | web-assets | Web asset optimization — image compression, WebP conversion, SVG optimization, sprite generation, CDN setup | `assets, image, optimization, webp, cdn` | Audit → Compress → Convert → Optimize → Upload | +| 67 | web-deploy | Web deployment orchestration (Vercel, Netlify, AWS, S3+CF) — config, secrets, domain setup, SSL, preview/prod gates | `deploy, web, vercel, netlify, aws` | Config → Secrets → Domain → SSL → Gates → Deploy | +| 68 | web-effects | Web visual effects — CSS filters, backdrop effects, glassmorphism, 3D transforms, transition choreography | `effects, css, visual, transform, blur` | Design → CSS → Polyfills → Test → Polish | + +--- + +## Table 2: Generated Agents (38 entries) + +| # | Agent | Role | Domain In | Forbidden Domain | +|---|-------|------|-----------|------------------| +| 1 | architect | Architectural decision + new-system design. Structural refactors, tech-debt resolution, scaling decisions. | All projects: Stack review, paradigm choice, layer design | NOT code-writing, NOT infra/deploy | +| 2 | code-implementer | Generic Rust/Swift/Python/Go/Flutter/TS implementation. Constructor Pattern, Rust-first, Test-First, Plan Mode. | All projects with source code | NOT ML training, NOT infra, NOT physics | +| 3 | code-implementer-flutter | Flutter-specific implementation. Riverpod, BLoC, state management, iOS/Android native bridges, animations. | Flutter projects only | Rust-first exception: Flutter projects mandate Dart | +| 4 | code-implementer-go | Go-specific implementation. Goroutines, channels, interfaces, error handling, CLI tooling, backend services. | Go codebases | Rust-first exception: Go projects only | +| 5 | code-implementer-python | Python-specific implementation. FastAPI/Django, ORM, async, testing, ML-adjacent. | Python codebases | Rust-first exception: Python projects only | +| 6 | code-implementer-rust | Rust-specific implementation. Ownership rules, trait system, async/await, macros, cargo workspace. | Rust codebases | Rust-first mandate: all Rust projects | +| 7 | code-implementer-swift | Swift-specific implementation. iOS/macOS development, SwiftUI, Combine, concurrency, package linking. | Swift/Obj-C projects | Rust-first exception: Swift iOS/macOS projects only | +| 8 | code-implementer-typescript | TypeScript-specific implementation. React, Vue, Svelte, Node, deno, browser/backend polymorphism, types. | TS/JS codebases | Rust-first exception: browser/DOM projects | +| 9 | cost-guardian | API cost auditor. Modal, AWS, GCP, fal.ai, Apify, ElevenLabs budget tracking, spend alerts, optimization. | All paid-compute projects | NOT deployment decisions | +| 10 | critic | Ruthless code critic. Anti-patterns, tech debt, bugs, security, performance. Read-only outputs with file:line evidence. | All codebases | NOT fixes, NOT edits | +| 11 | critic-anti-pattern | Anti-pattern specialist. Design smell detection, architectural red flags, refactoring candidates. | All codebases | NOT fixes | +| 12 | critic-bug | Bug specialist. Logic errors, race conditions, null pointer traps, boundary violations. | All codebases | NOT fixes | +| 13 | critic-perf | Performance specialist. CPU, memory, I/O bottlenecks, algorithmic inefficiency, bundle bloat. | All codebases | NOT fixes | +| 14 | critic-tech-debt | Tech-debt specialist. Deprecation, scaffolding stubs, temporary hacks, migration paydown. | All codebases | NOT fixes | +| 15 | fal-ai-runner | fal.ai API orchestration. Image generation, video, voice, upscaling, VFX, cost tracking. | fal.ai + model discovery projects | NOT code-writing, NOT training | +| 16 | frontend-validator | Frontend QA validator. Accessibility (a11y), responsive breakpoints, interaction patterns, visual parity. | Frontend projects | NOT code-writing | +| 17 | infra-implementer | Infrastructure/deploy implementation. CI/CD, containers, IaC, secrets, K8s, databases, monitoring. | All deploy-enabled projects | NOT source code, NOT ML training, NOT code-gen | +| 18 | infra-implementer-cicd | CI/CD pipeline specialist. GitHub Actions, GitLab CI, Forgejo, CircleCI, matrix builds, artifact storage. | All projects with CI needs | NOT container-only, NOT IaC-only | +| 19 | infra-implementer-container | Container specialist. Docker, Podman, multi-stage, image optimization, registry, scanning, K8s workloads. | Containerized projects | NOT CI-only, NOT IaC-only | +| 20 | infra-implementer-iac | Infrastructure-as-code specialist. Terraform, Ansible, CloudFormation, state management, drift detection. | All infrastructure projects | NOT manual ops, NOT CI-only | +| 21 | infra-implementer-secrets | Secrets + credentials management. Vault, sealed-secrets, env rotations, compliance (GDPR, SOC2, HIPAA). | All projects handling sensitive data | NOT code-writing, NOT deployment gates | +| 22 | ml-implementer | ML training/inference implementation. Modal jobs, experiment runners, Math-First paradigm, Pre-Experiment Check, observability-first. | ML/RL/CfC projects | NOT physics, NOT code-writing generically, NOT infra-only | +| 23 | ml-researcher | ML research specialist. Paper analysis, experiment design, architecture search, hyperparameter optimization, literature synthesis. | ML/RL projects | NOT production training, NOT code-writing | +| 24 | modal-runner | Modal infrastructure orchestration. Job spawning, volume management, GPU selection, cost estimation, result collection. | Modal-based ML/compute projects | NOT code-writing, NOT training logic | +| 25 | researcher | Open-ended research. Literature, web, code analysis. Evidence grading E1-E6. | All research-heavy projects | NOT code-writing, NOT decision-making | +| 26 | researcher-code | Code research specialist. API contracts, library docs, source-code archaeology, integration patterns. | Code-heavy projects | NOT writing code, NOT code-gen | +| 27 | researcher-hybrid | Hybrid research combining code + web sources. Documentation + API specs, standard compliance, library versioning. | Any multi-source research need | NOT code-writing, NOT synthesis | +| 28 | researcher-web | Web-only research. Official docs, vendor pricing, RFCs, blogs, GitHub issues, E1-E6 grading, zero-fabrication policy. | Research projects with public sources | NOT code access, NOT fabrication | +| 29 | security-auditor | Security code audit. Risk classification (HIGH/MEDIUM/LOW), differential review, variant analysis, supply-chain checks. | All projects with security concerns | NOT fixes, NOT pen-testing, NOT deployment | +| 30 | security-auditor-differential | Differential security review. Side-by-side code comparison, vulnerability checklist, auth/crypto/network focus. | Code-change security audits | NOT supply-chain, NOT variant-analysis | +| 31 | security-auditor-supply-chain | Supply-chain security specialist. Dependency audits, CVE tracking, maintainer signals, transitive deps, license checks. | All projects with dependencies | NOT code-level security, NOT auth | +| 32 | security-auditor-variant | Variant analysis for security. Root-cause abstraction, similar-location patterns, structural vulnerability hunt. | Post-vulnerability deep-dives | NOT differential-only, NOT supply-chain-only | +| 33 | validator | RULE 0.4 enforcement. Fact-checker, hallucination detector. VERIFIED/UNVERIFIED/FALSE/PARTIALLY TRUE verdicts with evidence URLs. | All claims in patents, commits, reports | NOT fabrication, NOT guessing | +| 34 | validator-api | API contract validator. Endpoint existence, signature match, version compatibility, breaking-change detection. | API projects, integrations | NOT code-writing | +| 35 | validator-benchmark | Benchmark validator. Experimental design, baseline comparison, statistical significance (p-value, effect-size). | ML/perf/physics experiments | NOT re-running experiments | +| 36 | validator-code-reality | Code-reality validator. Runtime vs docs matching, actual behavior vs claims, integration point verification. | All codebases with potential drift | NOT code-writing | +| 37 | validator-doc | Documentation validator. Completeness, accuracy, code examples execution, deprecation tracking, consistency. | All documentation | NOT doc-writing | +| 38 | validator-version | Version compatibility validator. Semantic versioning, breaking changes, deprecation timeline, API surface shifts. | All versioned libraries/APIs | NOT version bumping | + +--- + +## Table 3: Root Agents (12 entries) + +| # | Agent | Role | Model | Description | +|---|-------|------|-------|-------------| +| 1 | kei-architect | Architectural decision specialist (root Opus variant) | Opus 4.7 | Same as `architect` but spawned directly from user with Opus-tier reasoning for major design decisions | +| 2 | kei-code-implementer | Generic implementation specialist (root Opus variant) | Opus 4.7 | Same as `code-implementer` but spawned directly from user with Opus-tier context window for complex implementations | +| 3 | kei-cost-guardian | API cost auditor (root Opus variant) | Opus 4.7 | Same as `cost-guardian` but with full session context for multi-project cost tracking and optimization planning | +| 4 | kei-critic | Code critic (root Opus variant) | Opus 4.7 | Same as `critic` but with extended context for deep architectural smell detection across large codebases | +| 5 | kei-fal-ai-runner | fal.ai orchestration (root Opus variant) | Opus 4.7 | Same as `fal-ai-runner` but spawned directly for complex multi-model pipelines with full cost tracking | +| 6 | kei-infra-implementer | Infrastructure specialist (root Opus variant) | Opus 4.7 | Same as `infra-implementer` but spawned directly for complex multi-layer infrastructure changes and deployments | +| 7 | kei-ml-implementer | ML implementation specialist (root Opus variant) | Opus 4.7 | Same as `ml-implementer` but spawned directly with full experiment context, Math-First paradigm, Modal Protocol authority | +| 8 | kei-ml-researcher | ML researcher (root Opus variant) | Opus 4.7 | Same as `ml-researcher` but spawned directly for extended research sessions with literature synthesis and architecture search | +| 9 | kei-modal-runner | Modal orchestration (root Opus variant) | Opus 4.7 | Same as `modal-runner` but spawned directly for multi-hour Modal job orchestration with full GPU/cost decision authority | +| 10 | kei-researcher | Open-ended researcher (root Opus variant) | Opus 4.7 | Same as `researcher` but spawned directly with full context window for deep synthesis and evidence grading across domains | +| 11 | kei-security-auditor | Security auditor (root Opus variant) | Opus 4.7 | Same as `security-auditor` but spawned directly for comprehensive security audits with differential, variant, and supply-chain analysis | +| 11 | kei-validator | Validator/fact-checker (root Opus variant) | Opus 4.7 | Same as `validator` but spawned directly with full context for RULE 0.4 enforcement across patents, derivations, and claims | + +--- + +## Notes + +- **Skills** are reusable workflows — `/skill-name` invocations trigger hub-and-spoke decision trees, modal routers, or multi-phase pipelines. +- **Generated Agents** are specialized sub-roles spawned by `compose-solution`, `spawn-agent`, or user direct invocation in Sonnet (balanced cost/reasoning) or Opus (extended context). +- **Root Agents** are Opus-tier variants spawned directly from the main session for high-stakes decisions, extended reasoning, or full-context synthesis. +- All agents enforce [[~//../rules/code-implementer.md|RULE 0 Constructor Pattern]], [[~//../rules/dev-workflow.md|RULE 0.5 Plan Mode First]], and [[~//../rules/git-conventions.md|git commit hygiene]]. +- Read `_generated/_manifests/*.toml` source files for exact tool permissions, token budgets, and stage-gating details. +- Skill `.md` source: `/skills/*/SKILL.md` (68 files). +- Generated agent `.md` source: `/_generated/*.md` (38 files). +- Root agent `.md` source: `/kei-*.md` (12 files). + +--- + +**Last updated:** 2026-05-02 | **Catalogue version:** 1.0 | **KeiSeiKit:** v0.18 diff --git a/docs/encyclopedia/substrate-overview.md b/docs/encyclopedia/substrate-overview.md new file mode 100644 index 0000000..a84dad4 --- /dev/null +++ b/docs/encyclopedia/substrate-overview.md @@ -0,0 +1,425 @@ +# KeiSeiKit Substrate Overview & Encyclopedia + +**Last updated:** 2026-05-02 +**Status:** Comprehensive reference for substrate architecture, manifests, capabilities, roles, packages, and installer components. + +--- + +## 1. Architecture Overview + +KeiSeiKit is a **multi-tier agent composition substrate** built on four mutually-dependent layers: + +``` +┌─────────────────────────────────────────────────┐ +│ _assembler (Rust binary) │ +│ Composes agent .md files from: │ +│ - _manifests/*.toml (agent spec) │ +│ - _blocks/*.md (reusable prompt fragments) │ +│ - _capabilities/*/text.md (capability rules) │ +│ - _roles/*.toml (capability bundles) │ +│ - ~/.claude/rules/*.md (user rules) │ +│ Outputs: .claude/agents/*.md (generated) │ +└─────────────────────────────────────────────────┘ + ↓ one-way dependency +┌─────────────────────────────────────────────────┐ +│ _capabilities/ (declarative + Rust gates) │ +│ 18 capability atoms in 6 categories: │ +│ - policy (git scope, no-git-ops) │ +│ - scope (files whitelist/denylist) │ +│ - quality (Constructor Pattern, cargo-check) │ +│ - safety (no-dep-bump) │ +│ - output (report-format, severity-grade) │ +│ - tools (read-only, bash-allowlist) │ +│ - verify (fork-audit) │ +│ Each: capability.toml + text.md + Rust impl │ +└─────────────────────────────────────────────────┘ + ↓ one-way dependency +┌─────────────────────────────────────────────────┐ +│ _roles/ (capability bundles) │ +│ 7 roles = ordered list of capabilities + │ +│ tool allowlist + escalation policy. │ +│ Examples: │ +│ - read-only (10 capabilities, Bash forbidden) │ +│ - edit-local (8 capabilities, cargo-check req) │ +│ - edit-shared (12 capabilities, workspace auth) │ +│ - git-ops (5 capabilities, full git-safe) │ +│ - auditor (6 capabilities, diff + audit focus) │ +│ - explorer (4 capabilities, research-first) │ +│ - merger (7 capabilities, merge ceremony) │ +└─────────────────────────────────────────────────┘ + ↓ many-to-one dependency +┌─────────────────────────────────────────────────┐ +│ _manifests/ (agent specs) │ +│ 38 agent manifests in 8 families: │ +│ - code-implementer (7 languages) │ +│ - critic (5 specializations) │ +│ - researcher (web/code/hybrid) │ +│ - ml-implementer & ml-researcher │ +│ - infra-implementer (4 specializations) │ +│ - security-auditor (3 specializations) │ +│ - validator (6 specializations) │ +│ - architect, cost-guardian, etc. │ +│ Each: role, model, domain_in/out, handoffs │ +└─────────────────────────────────────────────────┘ +``` + +**Data flow at design time:** +1. User edits manifest `_manifests/code-implementer-rust.toml` +2. `_assembler` reads manifest + referenced blocks/roles +3. `_assembler` invokes kei-registry to refresh `capabilities.toml` index +4. `_assembler` regenerates `.claude/agents/code-implementer-rust.md` +5. keimd re-indexes the new agent + +**Data flow at runtime:** +1. User invokes agent via Claude Code Agent tool +2. Pre-spawn: `kei-agent-runtime compose` builds final prompt from role + capabilities +3. Agent executes with constraints from role (tools, file scope, capabilities) +4. Pre-tool-use: `kei-capability check ` gates operations (Bash, Edit, Write) +5. On agent return: `kei-capability verify ` validates output against constraints +6. Post-merge: simulated-merge test ensures agent output compiles and tests pass after integration + +--- + +## 2. Agent Manifests Table + +| Agent | Family | Model | Domain | Handoffs | Stability | +|-------|--------|-------|--------|----------|-----------| +| **code-implementer** | impl-generic | sonnet | Rust/Swift/Python/Go/Flutter/TS | ml-impl / infra / architect / critic / security | stable | +| **code-implementer-rust** | impl-lang | sonnet | Rust default; Constructor Pattern; tests-first | ml-impl / infra / architect / critic | stable | +| **code-implementer-swift** | impl-lang | sonnet | SwiftUI + SPM; menubar/iOS; Constructor Pattern | infra / architect | stable | +| **code-implementer-python** | impl-lang | sonnet | Python (RULE 0.2 exception #N only) | ml-impl / infra / architect | stable | +| **code-implementer-go** | impl-lang | sonnet | Go; mesh/CLI/embedded; Constructor Pattern | infra / architect | stable | +| **code-implementer-flutter** | impl-lang | sonnet | Flutter/Dart; Riverpod; Clean Architecture | infra / architect | stable | +| **code-implementer-typescript** | impl-lang | sonnet | Next.js/Node/browser; type-safe contracts | ml-impl / infra / architect | stable | +| **critic** | critic-generic | opus | Ruthless code review; anti-patterns, tech debt, perf, security | code-impl (fixes) | stable | +| **critic-bug** | critic-spec | opus | Off-by-one, error-swallowing, race conditions | code-impl | stable | +| **critic-perf** | critic-spec | opus | N+1, hot loops, unbounded retention | code-impl | stable | +| **critic-anti-pattern** | critic-spec | opus | God classes, deep inheritance, shotgun surgery | code-impl | stable | +| **critic-tech-debt** | critic-spec | opus | Dead code, TODOs, version-skew, stale deps | code-impl | stable | +| **researcher** | research-generic | opus | Web + codebase research; E1-E6 grading | validator / code-impl | stable | +| **researcher-web** | research-spec | opus | WebFetch/WebSearch only; fact-finding | validator | stable | +| **researcher-code** | research-spec | opus | Glob/Grep/Read; codebase discovery | code-impl | stable | +| **researcher-hybrid** | research-spec | opus | Parallel web + code research orchestrator | validator / code-impl | stable | +| **ml-implementer** | ml | sonnet | Training/inference; Modal jobs; Math-First paradigm | code-impl / infra / modal-runner | stable | +| **ml-researcher** | ml | opus | Paper review, benchmarks, reproducibility, tooling | ml-impl | stable | +| **infra-implementer** | infra-generic | sonnet | Deploy, CI/CD, secrets, IaC; credential isolation | code-impl / security | stable | +| **infra-implementer-cicd** | infra-spec | sonnet | GitHub Actions, GitLab CI, build pipelines | code-impl | stable | +| **infra-implementer-container** | infra-spec | sonnet | Dockerfile, OCI, multi-stage, distroless | code-impl | stable | +| **infra-implementer-iac** | infra-spec | sonnet | Terraform, Pulumi, CDK; Constructor Pattern | code-impl | stable | +| **infra-implementer-secrets** | infra-spec | sonnet | Vault, sops, age, env-var injection | code-impl | stable | +| **security-auditor** | security | opus | Risk-classified (HIGH/MEDIUM/LOW) audit with 9-point review | code-impl | stable | +| **security-auditor-differential** | security-spec | opus | Auth bypass, injection, deserialization, race conditions | code-impl | stable | +| **security-auditor-variant** | security-spec | opus | Pattern grep after vuln found; systematic coverage | code-impl | stable | +| **security-auditor-supply-chain** | security-spec | opus | New deps: maintainers, CVE, transitive deps | code-impl | stable | +| **validator** | validation | opus | API existence, version compat, code reality, benchmarks | code-impl | stable | +| **validator-api** | validator-spec | opus | API/OpenAPI verification | code-impl | stable | +| **validator-benchmark** | validator-spec | opus | External benchmark claim verification | code-impl | stable | +| **validator-code-reality** | validator-spec | opus | Behavioural claims vs running code | code-impl | stable | +| **validator-version** | validator-spec | opus | Semver, MSRV, transitive dep compatibility | code-impl | stable | +| **validator-doc** | validator-spec | opus | Documentation claim verification | code-impl | stable | +| **architect** | specialization | opus | System design, dependency analysis, patterns | code-impl | stable | +| **cost-guardian** | specialization | sonnet | Modal/AWS/GCP/fal.ai/Apify cost pre-launch gate | modal-runner | stable | +| **fal-ai-runner** | specialization | sonnet | Image/video/3D generation; model catalog & pricing | code-impl | stable | +| **modal-runner** | specialization | sonnet | Modal jobs: cost est, GPU compat, observability | code-impl | stable | +| **frontend-validator** | specialization | sonnet | tsc --noEmit, eslint, kei-db-contract, visual snapshots | code-impl | stable | + +**Notes:** +- Model: `opus` (Opus 4 for complex reasoning), `sonnet` (Haiku 4.5 for routine implementation) +- All manifests reference role via `substrate_role = ""` (see §3 below) +- Handoff targets are agents the manifest explicitly names for task delegation +- All stable as of 2026-05-02; no experimental agents currently active + +--- + +## 3. Capabilities Table + +| Capability | Category | Description | Gate/Verify | Event | Severity | Bypass Env | +|------------|----------|-------------|-------------|-------|----------|-----------| +| **policy::no-git-ops** | policy | Forbid git, gh repo, gh api /repos | gate | PreToolUse:Bash | block | ORCHESTRATOR_META | +| **policy::git-ops-scope** | policy | Allow git within scope (branch, commit, push) | gate | PreToolUse:Bash | warn | GIT_OPS_OVERRIDE | +| **scope::files-whitelist** | scope | Restrict Edit/Write to declared file glob patterns | gate+verify | PreToolUse:Edit\|Write | block | — | +| **scope::files-denylist** | scope | Blacklist specific file globs from any touch | gate+verify | PreToolUse:Edit\|Write | block | — | +| **scope::read-only** | scope | Deny all Edit/Write/Bash; allow Read/Glob/Grep only | gate | PreToolUse:Edit\|Write\|Bash | block | — | +| **quality::constructor-pattern** | quality | Enforce: file ≤ 200 LOC, function ≤ 30 LOC | verify | on-return | fail-if-exceed | — | +| **quality::cargo-check-green** | quality | Verify: `cargo check --workspace` exits 0 | verify | on-return | fail-if-error | — | +| **quality::tests-green** | quality | Verify: `cargo test` passes; min test-count floor | verify | on-return | fail-if-error | — | +| **safety::no-dep-bump** | safety | Forbid Cargo.toml [dependencies] edits; no `cargo add` | gate+verify | PreToolUse:Edit / on-return | block | DEPENDENCY_OVERRIDE | +| **output::report-format** | output | Require: Files written / cargo-check / cargo-test / LOC-delta / blockers / next (fields) | verify | on-return | fail-if-missing | — | +| **output::severity-grade** | output | Require: E1-E6 evidence grade on all numeric/claim output | verify | on-return | warn-if-missing | — | +| **output::merge-result** | output | Require: STATUS-TRUTH MARKER (shipped:functional\|partial\|scaffolding) | verify | on-return | warn-if-missing | STATUS_TRUTH_BYPASS | +| **tools::read-only** | tools | Deny Edit/Write; Bash patterns restricted to read-only | gate | PreToolUse:Bash\|Edit\|Write | block | — | +| **tools::cargo-only-bash** | tools | Restrict Bash to cargo-only patterns + safe mkdirs | gate | PreToolUse:Bash | block | CARGO_ONLY_OVERRIDE | +| **tools::bash-allowlist** | tools | Restrict Bash to declared allow-patterns | gate | PreToolUse:Bash | warn | — | +| **tools::deny-tools** | tools | Blacklist specific tools (e.g. NotebookEdit) | gate | PreToolUse:Any | block | — | +| **verify::fork-audit** | verify | Validate 6-file artefact bundle on agent branch (RULE 0.12) | verify | on-return | warn-if-missing | AGENT_AUDIT_SKIP | + +**Notes:** +- **Gate vs Verify:** gate=PreToolUse (blocks before action), verify=on-return (validates after) +- **Severity:** block=exit 2 (fail), fail-if-*=test failure, warn=stderr advisory (exit 0), —=no bypass +- **Stage:** design-time (manifest load) vs runtime (agent execution) +- **Rust impl:** gates + verifies in `_primitives/_rust/kei-agent-runtime/src/{gates,verifies}/` + +--- + +## 4. Roles Table + +| Role | Spawnable | Capabilities | Tools Allowed | Bash Patterns | Description | +|------|-----------|--------------|---------------|---------------|-------------| +| **read-only** | yes | policy::no-git-ops, scope::read-only, tools::read-only, output::report-format | Read, Glob, Grep | none | Research/investigation only; no mutations. Read docs/files, find info, report findings. | +| **explorer** | yes | policy::no-git-ops, scope::read-only, tools::read-only | Read, Glob, Grep, Bash (log-only) | grep, find (log output) | Exploration+discovery; grep codebase, scan logs. Pre-implementation research gate. | +| **edit-local** | yes | no-git-ops, files-whitelist, files-denylist, constructor-pattern, cargo-check-green, tests-green, no-dep-bump, report-format | Read, Write, Edit, Glob, Grep, Bash | ^cargo, ^mkdir, ^rm -rf /tmp | Local file edits within whitelist; cargo check/test required; no git, no deps. Typical code-implementer sandbox. | +| **edit-shared** | yes | no-git-ops, files-whitelist, files-denylist, constructor-pattern, cargo-check-green, tests-green, no-dep-bump, workspace-auth, report-format | Read, Write, Edit, Glob, Grep, Bash, Agent | ^cargo, ^mkdir, ^rustc, ^tsc | Workspace edits; workspace-level auth; can spawn sub-agents. No git; Cargo workspace-level checks. | +| **git-ops** | no | git-ops-scope (allow safe git), files-whitelist, files-denylist, no-dep-bump, report-format, merge-result | Read, Write, Edit, Glob, Grep, Bash, Agent | git (scoped), gh (safe patterns), cargo | Orchestrator-only. Full git (branch, commit, push) within declared scope. Merge ceremony + verification gates. | +| **auditor** | yes | policy::no-git-ops, scope::read-only (plus diff access), constructor-pattern, cargo-check-green, tests-green, output::report-format | Read, Glob, Grep, Bash (cargo only) | ^cargo, audit-specific | Post-code-review role. Runs cargo check/test, diffs, double-audit protocol. Reports findings without fixes. | +| **merger** | no | git-ops-scope, constructor-pattern, cargo-check-green, tests-green, output::merge-result | all | all (git-scoped) | Orchestrator-only. Merges agent branches; runs pre-merge verification (cargo + test count); manages merge ceremony. | + +**Notes:** +- **Spawnable:** agents marked "no" cannot be invoked by user; only orchestrator can invoke them +- **Capabilities:** ordered list; text.md fragments concatenated in this order into final prompt +- **Tools Allowed:** hard deny if not in this list (except git-ops / merger which are orchestrator-only) +- **Bash Patterns:** additional restriction via regex; agent Bash restricted to matching patterns only +- **Escalation policy:** most roles escalate via file-return field list; git-ops/merger escalate via merge ceremony + +--- + +## 5. TypeScript Packages Table + +| Package | Version | Purpose | Key Dependencies | Build Target | +|---------|---------|---------|------------------|--------------| +| **@keisei/mcp-server** | 0.14.0 | MCP server exposing KeiSeiKit Rust primitives as tools | @modelcontextprotocol/sdk ^1.0.0, execa ^9.0.0, zod ^3.23.0 | Node.js ≥18.0.0; native binaries via bun compile (darwin/linux/windows) | +| **@keisei/gmail-adapter** | 0.5.2 | Gmail API integration for email-based task intake | @google-cloud/gmail ^1.3.0, nodemailer-mock ^2.0.0 | Node.js ≥18.0.0 | +| **@keisei/grok-adapter** | 0.3.1 | Grok (xAI) LLM provider bridge | openai-compatible ^1.0.0 | Node.js ≥18.0.0 | +| **@keisei/telegram-adapter** | 0.6.0 | Telegram Bot API integration for notifications + input | telegram-typings ^4.10.0, node-telegram-bot-api ^0.65.0 | Node.js ≥18.0.0 | +| **@keisei/recall-adapter** | 0.2.1 | Recall.ai (browser automation) integration | recall-ai-sdk ^1.0.0 | Node.js ≥18.0.0; browser environment recommended | +| **@keisei/youtube-adapter** | 0.1.8 | YouTube API integration for video transcript + metadata | googleapis ^118.0.0 | Node.js ≥18.0.0 | + +**Notes:** +- All packages scoped under `@keisei/` on npm (published to keigit.com npm registry) +- All use TypeScript 5.5+ with strict mode; zod for runtime validation +- Build output lives in `dist/` (generated from `src/` via `tsc -b`) +- MCP server ships as multi-target native binaries (darwin/linux/windows arm64 + x64) + +--- + +## 6. Top-Level Docs (25 files) + +| Document | Purpose | Key Sections | +|----------|---------|--------------| +| **SUBSTRATE-SCHEMA.md** | Atom + capability schema SSoT | Core concept, file layout, Cargo.toml metadata, JSON Schema conventions, discovery | +| **AGENT-SUBSTRATE-SCHEMA.md** | Agent capability atoms (gates+verifies) | Core concept triplet, file layout, capability.toml shape, Rust trait contract | +| **ARCHITECTURE.md** | Stack overview + trait impl matrix | Compute/Git/Memory/Auth/Notify/Network/LLM/ServiceManager impls per backend | +| **AGENT-ROLES.md** | Generated role matrix (human-readable) | 7 roles × capabilities × tools × escalation (auto-generated from _roles/*.toml) | +| **PHILOSOPHY.md** | Substrate philosophy + design principles | Single-source-of-truth, Constructor Pattern, no overlays, decomposability | +| **DNA-INDEX.md** | Agent DNA (deterministic identity) format | 80-char DNA breakdown: role::caps::scope-sha8::body-sha8-nonce | +| **IMPORT-RUNTIME.md** | Foreign-project ingestion pipeline | Decompose → match → extract skills → plan → execute (Hermes proof-of-concept) | +| **PUBLISHING.md** | Community npm registry + scoped package publishing | keigit.com npm, OAuth, per-user PAT, `npm publish` / `npm install` flow | +| **RULES-AS-BLOCKS.md** | How user ~/.claude/rules/*.md become prompt blocks | Rule fragment extraction, RULE re-composition, link-tracking | +| **QUICKSTART.md** | 60-second install guide | 11 install profiles (minimal/core/full + MCP/Cortex/Cursor/Continue/etc) | +| **INSTALL.md** | Full installation docs | Prereqs, profiles, lib-*.sh breakdown, troubleshooting | +| **CONVERGENCE-PLAN.md** | Multi-stream parallel work roadmap | UI / Atoms refactor / Graph / Runtime phases (2026-06 closure target) | +| **REFERENCE.md** | Command-line reference for kei-* binaries | kei-runtime, kei-sage, kei-registry, kei-import, kei-capability, kei-forge | +| **SCHEMA-LOCKED.md** / **SCHEMA-UNLOCKED.md** | Lock markers for breaking change gates | SUBSTRATE-SCHEMA.md locked 2026-05-02; 6-week parallel window | +| **AGENT-SCHEMA-LOCKED.md** | Lock marker for agent substrate | AGENT-SUBSTRATE-SCHEMA.md locked 2026-04-23; 3-week parallel window | +| **SECURITY.md** | Substrate security model | Sandboxing, capability enforcement, audit gates, simulated-merge verification | +| **SLEEP-LAYER.md** | Three-phase nightly consolidation (REM/NREM) | Phase A incubation (tasks), Phase B consolidation (reports), Phase C deep-sleep (conflicts) | +| **TAXONOMY.md** | Metadata taxonomy for all substrate objects | kingdom, mechanism, domain, layer, stage, stability, language (Dublin Core + custom) | +| **HANDOFF-WAKE.md** | Agent handoff orchestration + wake-on-complete | Async task queue, named handoffs, web hook triggers | +| **WHY.md** | Motivation + pain points solved | Context loss, agent collisions, duplicate work, no fork hygiene | +| **USB-BRAIN-GUIDE.md / *.md** | Portable offline KeiSeiKit on USB | Self-contained substrate on external drive; no internet required | + +--- + +## 7. Installer Components (install/lib-*.sh) + +| Library | Purpose | Functions | +|---------|---------|-----------| +| **lib-log.sh** | Logging + output formatting | color_info, color_success, color_warn, color_error, log_step | +| **lib-prereqs.sh** | System prerequisite checks + installation | check_rust, check_node, check_git, install_cargo_deps | +| **lib-profile.sh** | Profile selection (minimal/core/full + target) | detect_client, show_profiles, select_profile | +| **lib-pathway.sh** | Pathway builder (install → scaffold → verify) | build_pathway, run_checks, verify_install | +| **lib-primitives.sh** | Rust primitive crate building | cargo_build_release, strip_binaries, copy_to_bin | +| **lib-agents.sh** | Agent manifest + generated .md setup | copy_agents, generate_agents, verify_agent_dna | +| **lib-hooks.sh** | Hook installation + registration | install_hooks, register_hook_event, test_hook | +| **lib-bridges.sh** | Cross-tool bridge generation | generate_cursorrules, generate_windsurf_rules, generate_github_copilot | +| **lib-scaffold.sh** | Workspace scaffolding (dirs, templates, config) | create_workspace, setup_config_toml, init_memory_db | +| **lib-wizard.sh** | Interactive setup wizard | show_welcome, ask_profile, ask_features, summary | +| **lib-plan.sh** | Install plan builder + preview | build_plan, show_plan, confirm_plan | +| **lib-dev-hub-*.sh** | Optional dev-hub tools | restic (backup), mdbook (docs), gdrive-import (foreign-project), datasette (SQL UI) | + +**Key entry points:** +- `install.sh` — main orchestrator; sources lib-*.sh files and runs pathway +- `install.sh --profile=minimal` — install without optional tools +- `install.sh --profile=full --with-cortex` — full suite + Cortex stack +- `install.sh --dry-run` — preview without mutating filesystem + +--- + +## 8. Manifest Block References + +The `_blocks/` directory contains reusable prompt fragments included via TOML `blocks = [...]` list in manifests. Examples: + +| Block Name | Used By | Content | +|------------|---------|---------| +| **baseline** | ALL manifests (obligatory) | RULE -1, RULE 0.1, RULE 0.2, RULE 0.4, RULE 0.5, RULE ZERO, KARPATHY (full text, never summarized) | +| **evidence-grading** | ALL manifests (obligatory) | E1-E6 grading table + rules for claims | +| **memory-protocol** | ALL manifests (obligatory) | 3-layer memory architecture + session save protocol | +| **rule-pre-dev-gate** | code-implementer* | Pre-Dev Gate (analogues, stack compat, duplication checks) | +| **rule-test-first** | code-implementer* | Test-First discipline (TDD for critical, alongside for rest) | +| **rule-error-budget** | code-implementer* | 3-Level Escalation (2 fails → review, 3 → research) | +| **rule-double-audit** | code-implementer*, critic* | Phase 1/2/3/4 double-audit protocol | +| **rule-no-patching** | code-implementer* | Root-cause fixes (never overlay patches) | +| **rule-constructor-pattern** | code-implementer*, infra-impl* | Constructor Pattern enforcement + split criteria | +| **rule-no-hallucination** | researcher*, validator* | RULE 0.4 citation verify + confidence grading | +| **rule-git-conventions** | infra-impl, git-ops | Commit types + timing SSoT | +| **rule-api-cost-guard** | cost-guardian, modal-runner | Dashboard check, price estimation, batch caution | +| **rule-security-review** | security-auditor* | Risk classification + 9-point differential review | + +**Discovery:** blocks are indexed in `_blocks/blocks.toml` (auto-generated by assembler) with versions + keywords for kei-sage search. + +--- + +## 9. Capability Text Fragments + +Each capability has a `text.md` fragment (≤ 200 words) that is concatenated into the final agent prompt. Examples: + +### policy::no-git-ops (text.md) + +```markdown +## No git operations + +You MUST NOT invoke git, gh repo, gh api /repos, or any shell command +that modifies git state. Orchestrator handles all git operations +(commits, branches, pushes, rebases). + +If your task requires staging a change, describe it in the return +file-list — the orchestrator will commit on your behalf. + +Bypass exists for orchestrator-meta agents only; it is not available here. +``` + +### quality::constructor-pattern (text.md) + +```markdown +## Constructor Pattern compliance + +Every file you write or edit MUST stay under 200 lines of code. +Every function MUST stay under 30 lines of code. No exceptions. + +If your change pushes a file past 200 LOC or a function past 30 LOC, +split it on the spot. Never commit with "TODO: refactor later". + +Comments, blank lines, and use statements count toward LOC — +the verifier counts lines as wc -l sees them. +``` + +These are concatenated in role-declared order, with `\n\n---\n\n` separators between fragments. + +--- + +## 10. Runtime Execution Flow + +**At agent spawn time:** +``` +1. Orchestrator: kei-agent-runtime compose → prompt.md + - Reads role + ordered capability list + - Concatenates blocks + capability text.md fragments + - Outputs final prompt to stdout / file +2. Orchestrator: Agent({ prompt, model, tools }) ← Claude Code Agent tool +3. Agent runs with PreToolUse hooks wired to kei-capability check +``` + +**On each tool call:** +``` +4. Claude Code: PreToolUse:Bash → calls hook +5. Hook: exec kei-capability check --capability "tools::cargo-only-bash" < +6. kei-capability: dispatch to Rust gate impl +7. Gate: Allow | Deny | NotApplicable +8. Hook: exit 0 (allow) | exit 2 (deny) | return +9. Tool call proceeds or fails +``` + +**On agent return:** +``` +10. Agent emits final report with file list + status-truth-marker +11. Orchestrator: kei-capability verify --capability "quality::cargo-check-green" \ + --worktree /path --mode simulated-merge +12. kei-capability: dispatch to Rust verify impl +13. Verify: run `cargo check` in worktree OR on temp merge branch +14. Verify: Pass | Fail {reason} +15. Orchestrator: if any verify fails, block merge + report findings +16. User: approves merge or requests fixes (via return questionnaire) +``` + +--- + +## 11. Substrate Invariants + +| Invariant | Enforced By | Failure Mode | +|-----------|-------------|--------------| +| Manifests declare role + handoffs | assembler `validate-manifest` + CI | spawn rejected; agent unknown | +| Role names match _roles/*.toml | assembler `validate-role` + CI | manifest ref error | +| Capabilities exist + have both toml + text.md | assembler `validate-capability` + CI | compose fails; missing fragments | +| JSON Schemas are valid draft-07 | `kei-schema-lint` + CI | atom discovery skips malformed schema | +| Gates/verifies registered in kei-agent-runtime | `cargo test --all` | gate lookup fails; aborting capability | +| Agent DNA is 80 chars unique per invocation | kei-ledger fork + DNA-INDEX.md | collision risk; query ambiguity | +| No file edits outside files-whitelist | scope::files-whitelist gate | merge blocked; scope violation | +| Files ≤ 200 LOC, functions ≤ 30 LOC | quality::constructor-pattern verify | merge blocked; pattern violation | +| cargo check + tests green | quality::cargo-check-green + tests-green | merge blocked; integration failure | +| No Cargo.toml [dependencies] edits | safety::no-dep-bump gate | merge blocked; supply-chain review pending | +| STATUS-TRUTH MARKER present | output::merge-result verify | merge blocked; shipped-vs-functional drift flagged | + +--- + +## 12. Key Concepts Glossary + +| Term | Definition | Example | +|------|-----------|---------| +| **Atom** | One verb operation on a primitive; independently composable | `kei-task::create`, `kei-task::search` | +| **Capability** | Declarative bundle (TOML + text.md + Rust impl) that enforces agent constraint | `quality::constructor-pattern`, `scope::files-whitelist` | +| **Block** | Reusable prompt fragment referenced by multiple manifests | `rule-pre-dev-gate`, `baseline` | +| **Manifest** | Agent spec declaring role, tools, domain_in/out, handoffs | `code-implementer.toml`, `ml-implementer.toml` | +| **Role** | Bundle of capabilities + tool allowlist + escalation policy | `edit-local`, `read-only`, `git-ops` | +| **Gate** | PreToolUse Rust impl blocking tool calls (deny) | `policy::no-git-ops` blocks `^git` | +| **Verify** | On-return Rust impl validating agent output + integration | `quality::cargo-check-green` runs `cargo check --workspace` | +| **DNA** | 80-char deterministic agent identity; "did this run before?" without embeddings | `code-impl::edit-local::7f1a2c3d::5e9b4f2a-nonce` | +| **Simulated merge** | Orchestrator creates test branch, applies agent diff, runs checks from there | catches integration regressions pre-merge | +| **Hands off** | Agent delegates work to another agent via manifest `[[handoff]]` table | `code-implementer` hands off to `ml-implementer` | + +--- + +## 13. Status & Completeness + +**As of 2026-05-02:** + +| Layer | Stability | Notes | +|-------|-----------|-------| +| Manifests (38 agents) | stable | All roles + models declared; DNA format frozen | +| Capabilities (18 atoms) | stable | All gates + verifies implemented; tests green | +| Roles (7 roles) | stable | edit-local / read-only / git-ops / etc. locked; no churn | +| Assembler (compose logic) | stable | Generates .md from TOML + blocks; keimd integration active | +| Cortex stack | beta | kei-cortex (HTTP) + kei-tty (TUI) build clean; browser/VSCode frontends concept | +| MCP Server (@keisei/mcp-server) | stable | Exports Rust atoms as MCP tools; published to keigit.com npm | +| Bridges | stable | 11 cross-tool format generators (.cursorrules, .windsurf/rules, GEMINI.md, etc.) | +| Sleep Layer (Phase A/B/C) | stable | Incubation (tasks), REM consolidation (reports), NREM deep-sleep (conflicts) | +| Foreign-project ingestion | stable | kei-import proof-of-concept via Hermes validation | +| Nightly consolidation | active | Running 2026-05-02; Phase A + B + C observed; reports 495 DNA indices | + +**Roadmap:** +- Model router (Bayesian posterior, currently manual routing) +- Hosted Phase B/C (cross-machine memory sync) +- Encyclopedia-as-API (query substrate by DNA / role / capability) +- Browser + VSCode frontends for Cortex + +--- + +## References + +- `docs/SUBSTRATE-SCHEMA.md` — atom + capability SSoT +- `docs/AGENT-SUBSTRATE-SCHEMA.md` — agent gate/verify triplet spec +- `docs/ARCHITECTURE.md` — implementation trait matrix +- `docs/DNA-INDEX.md` — agent identity format +- `.claude/rules/orchestrator-branch-first.md` (RULE 0.13) — agent git model +- `.claude/rules/agent-git-model.md` (RULE 0.12) — fork/ledger lifecycle +- `_primitives/_rust/kei-agent-runtime/src/` — gate + verify impls +- `_primitives/_rust/kei-registry/` — atom discovery + indexing +- `_assembler/src/` — manifest → agent.md composition