Closes the loop on "without full tracking the system can't make decisions"
(user pushback on partial coverage). Three gaps that left the inference
layer blind are now wired:
GAP #1 — agent toolStats / token counts / cache hits captured
================================================================
`agent-outcome-backfill.sh` now appends one JSONL row per spawn to
`~/.claude/memory/time-metrics/agent-toolstats.jsonl` with:
agent_id, outcome, stubs, ts,
tool_use_count, duration_ms, tool_stats {Read:N, Bash:M, ...},
tokens_in, tokens_out, cache_read, cache_write
Sidecar journal (no schema migration). Production payload's
.tool_response.totalToolUseCount / totalDurationMs / toolStats / usage
fields land directly. Smoke-tested with synthetic spawn — row written.
GAP #2 — skill_invocations table actually receives writes
================================================================
The `skill_invocations` table (schema v8) had 0 rows because no caller
existed for `skill_metrics::record_invocation`. Added two pieces:
(a) `kei-ledger record-skill <name> --success {0|1}` CLI subcommand
Mirrors record-cost; same dispatch shape. Optional `--agent-id`,
`--trajectory-id`, `--duration-ms`, `--db`. Validates non-empty
name + duration ≥ 0. Outputs `{"ok":true,"skill":"...","ts":N}`.
(b) `hooks/skill-record.sh` — PostToolUse:Skill hook. 50 LOC POSIX.
Detects Skill tool calls, derives success heuristic from
tool_response (exit_code / status / content non-empty), shells
out to `kei-ledger record-skill`. Bypass via SKILL_RECORD_BYPASS=1.
83 kei-ledger tests pass (16 unit + 67 integration). Smoke-tested
end-to-end: `kei-ledger record-skill test-skill --success 1` inserts
a row with correct fields.
Phase D nightly skill-metrics decisions (archive if unused N days,
re-extract if success<60% over M days, validated if >20 calls + >90%
success) now have data to consume.
GAP #3 — numeric-claims.jsonl receives every evidence-tagged claim
================================================================
RULE 0.18 mandated three markers `[REAL:]` / `[FROM-JOURNAL:]` /
`[ESTIMATE-HTC:]` on every numeric/duration/cost claim, but no hook
appended valid claims to the journal — the calibration data RULE 0.18
promised never accumulated.
`hooks/numeric-claims-record.sh` — Stop hook, 140 LOC POSIX. Reads
transcript_path from stdin, locates the last assistant message via
recursive flatten (same pattern as agent-outcome-backfill.sh after
the production-payload-shape fix), regex-extracts every `<phrase>
[<TIER>: <pointer>]` triple, appends one JSONL row per claim.
Idempotent within 1-second window to avoid double-recording on
repeat Stop fires. Bypass via NUMERIC_CLAIMS_RECORD_BYPASS=1.
Smoke test: synthetic transcript with 3 markers (REAL + ESTIMATE-HTC
+ FROM-JOURNAL) produced exactly 3 well-formed JSONL rows.
Settings.json
================================================================
- PostToolUse:Skill matcher created (or augmented if already
present) with skill-record.sh.
- Stop:* matcher gains numeric-claims-record.sh after the existing
chain (stop-verify, task-timer, session-end-dump, extract-task-
durations, chat-numeric-postflag, affect-threshold-check,
enrich-from-jsonl).
What this does NOT do (deferred):
- Backfill `skill_invocations` from past traces (history started
today; Phase D cohort builds forward from now).
- Migrate the agent toolStats sidecar JSONL into a proper ledger
column. Append-only file is fine for the current scale.
- Refactor main.rs (now 233 LOC, was 212; pre-existing CP debt
flagged by skill-record agent — separate cleanup PR).
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- kei-ledger main.rs Constructor Pattern split (212→233 LOC)
- Verify in next session: skill_invocations gets rows from real
Skill tool use; numeric-claims.jsonl gets rows from real assistant
messages with markers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two parallel agents (both Sonnet 4.6 via the just-activated tier system)
extended the substrate-unified-registry. First end-to-end proof that the
Phase 4 router refactor saves money: no Opus spawns this round.
PART 1 — `kei-registry secrets` subcommand (Agent A — code-implementer)
Reads env-var NAMES from `~/.claude/secrets/.env` (RULE 0.8 SSoT) and
per-project `secrets/*.env`, greps the kit tree for usages, reports
orphans (defined but unreferenced). Live run on this kit found 26 keys,
11 ORPHAN — actionable cleanup candidates incl. GitHub OAuth client
creds, Godaddy keys, KeiGit admin creds, KEI_MEMORY_TOKEN.
Files:
- `_primitives/_rust/kei-registry/src/secrets.rs` (152 LOC) — pure
read-side cube. SecretsReport + KeyRow types, env-file parser
(KEY=value lines, validates `^[A-Z][A-Z0-9_]*$`), walkdir-based
scanner with skips (target/ node_modules/ .git/ _generated/),
word-boundary regex per key. ASCII + JSON render.
- `_primitives/_rust/kei-registry/src/secrets_tests.rs` (125 LOC) —
5 unit tests covering env parse, scan correctness, word-boundary
regression (`MY_KEY` ≠ `MY_KEY_EXTRA`), JSON roundtrip, ORPHAN marker.
- `_primitives/_rust/kei-registry/src/secrets_handler.rs` (58 LOC) —
CLI dispatch handler.
- `cli.rs`, `handlers.rs`, `lib.rs` extended with Secrets variant.
Resolves the asymmetry called out in the design discussion: paths got
atomization (commit f135ece), keys get a query-layer instead. Reason:
env-var NAMES are already public and stable; opaque atom-DNA over them
adds zero security and full overhead. Orphan detection is the unique
value, and a 30-LOC subcommand delivers it without a per-key atom file.
PART 2 — kei-model catalog extension (Agent B — fal-ai-runner)
Adds 10 generation-model entries with VERIFIED pricing per RULE 0.4:
- google: gemini-3-1-flash-image, gemini-3-pro-image
- fal.ai: flux-2-pro, flux-pro-1-1, kling-o3, veo-3, ideogram-v3, recraft-v3
- elevenlabs: elevenlabs-v3, elevenlabs-multilingual-v2
Pricing sourced from each provider's public pricing page (URLs cited
per row in `notes` + `source_url` fields); 8/10 verified, 2 marked
needs-verification (gemini-3-pro-image price not found on public page).
Schema additions to `_primitives/_rust/kei-model/src/model.rs` to
support the new entries without `provider = "local"` placeholder:
- Provider enum + 3 variants: Google, Fal, Elevenlabs (with as_str
+ parse impls).
- Capability enum + 9 variants: image-gen, text-to-image, image-edit,
video-gen, text-to-video, image-to-video, voice-gen, text-to-speech,
voice-clone (with serde rename + as_str + parse).
Pricing struct unchanged: per-image / per-second / per-1k-chars unit
costs ride existing `output_per_mtok_micro` field with the unit
documented in `notes` (e.g. "Per-image cost. 1 unit = 1 image."). A
proper Pricing.unit field is a follow-up.
Files:
- `_primitives/_rust/kei-model/src/model.rs` (+24 LOC enum extensions)
- `_primitives/_rust/kei-model/data/models.toml` (+216 LOC, 471 total)
`kei-model list` returns the full 21-model catalog incl. new providers.
Tests:
- kei-registry: 25 passed (existing + 5 secrets tests + 10 status)
- kei-model: 0 (no unit tests in crate, parser smoke via list)
- agent-assembler: 29 passed (no regressions)
Verification (cited):
- `./target/release/kei-registry secrets --env-file ~/.claude/secrets/.env`
emits real report 26/11 orphan.
- `./target/release/kei-model list` parses all 21 entries cleanly.
- `cargo build --release --workspace` clean.
What this does NOT do (deferred):
- Pricing.unit field (per-mtok / per-image / per-second / per-1k-chars
discriminator) — needs Rust struct refactor + cost-estimator update.
- `secrets` skip-list extension (worktrees, _ts_packages/node_modules
duplicate counts) — minor noise.
- gemini-3-pro-image pricing (no public page; vendor-specific quote
needed).
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- Pricing.unit field for cost-estimator correctness on gen models
- secrets scan: skip .claude/worktrees/ to avoid duplicate counts
- gemini-3-pro-image price verification
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 of substrate-unified-registry: turn on the existing
kei-model-router by changing manifest defaults from `model = "opus"`
to `model = "sonnet"` for routine agents, and give every git branch
a deterministic DNA in the kei-status dashboard.
The model-tier system was BUILT (`_primitives/_rust/kei-model-router/`
crate with Beta posterior, complexity τ-estimator, escalate ladder,
calibrate subcommand) and the advisor hook
(`~/.claude/hooks/model-router-advisor.sh`) was REGISTERED. But every
ledger row from this session ran on Opus because:
1. All 38 manifests hard-coded `model = "opus"` → no chance for the
router to recommend cheaper.
2. The orchestrator (me) ignored the stderr advisory.
This commit closes (1). (2) is a behavioural change tracked separately.
Manifest reclassification (4 Opus + 34 Sonnet):
Opus (hard reasoning):
- architect (system-design synthesis)
- ml-implementer (Math-First paradigm)
- ml-researcher (literature analysis)
- security-auditor (deep risk synthesis)
Sonnet (everything else):
- 8 code-implementer-* + code-implementer
- 5 critic-* + critic
- 6 infra-implementer-* + infra-implementer
- 4 researcher-* + researcher
- 6 validator-* + validator
- 3 security-auditor-{differential,supply-chain,variant}
- cost-guardian, fal-ai-runner, frontend-validator, modal-runner
Regenerated all 38 `_generated/*.md` so the YAML frontmatter `model:`
field matches the manifest.
Branch DNA (kei-registry status):
- New `compute_branch_dna(name, commit_sha)` in `status.rs`. Format
`branch::<sha8(name)>::<sha8(commit)>`, mirrors kei-shared
DNA wire layout `<role>::<caps>::<scope_sha8>::<body_sha8>`.
- Deterministic — same `(name, commit)` → same DNA. Changes when
either changes. No DB persistence: the underlying truth lives in
`.git/refs/heads/<name>`.
- 3 new unit tests cover format, determinism, name-change, commit-
change. `cargo test status::tests` → 10 passed.
`kei-registry status` output now shows DNA prefix per branch alongside
ahead/behind, last commit. Combined with existing per-block DNA in the
[Blocks] and [Path Atoms] sections + `dna` column on `agents` table in
kei-ledger, every artefact in the dashboard has an identifier:
Atoms (incl path-atoms) → atom::<caps>::<scope>::<body> (registry)
Skills/Rules/Hooks/Prim → <role>::<caps>::<scope>::<body> (registry)
Agent forks → row.dna in agents table (ledger)
Local branches → branch::<sha8>::<sha8> (computed)
What this does NOT do:
- No outcome backfill — the 205 NULL outcomes in ledger still prevent
the Beta posterior from learning. Router falls back to top-tier
until ≥1 datapoint per (task_class, model) accumulates. Tracked as
follow-up.
- No post-checkout hook to auto-register branches in kei-ledger. Live
shell-out to `git for-each-ref` is fast enough for the dashboard;
persistence buys nothing the .git tree doesn't already give.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- Outcome backfill hook (writes outcome to ledger after agent done)
- User /model claude-sonnet-4-6 for current session (5x cheaper)
- Push the orchestrator (me) to read advisor stderr in real-time
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3 of substrate-unified-registry: a single command shows every
live artefact across the three sources without merging stores.
`kei-registry status` joins:
1. `blocks` table (kei-registry SQLite) — active counts per BlockType,
plus the registered path-atoms with DNA prefix + body sha8.
2. `git for-each-ref refs/heads` (shell-out, no DB persistence) — local
branches, current marker, ahead/behind via `upstream:track,nobracket`.
3. `agents` table (kei-ledger SQLite) — fork counts per status
(running/done/failed/merged/rejected). Missing ledger DB → section
skipped, never an error.
Output: ASCII multi-section table by default; `--format json` for
machine consumption.
Files:
- `_primitives/_rust/kei-registry/src/status.rs` — new module, ~270
LOC. Pure read-side per Constructor Pattern. 7 unit tests cover
`parse_track` (in sync / ahead / behind / both / "gone"), DNA prefix
rendering, and empty-status section presence.
- `_primitives/_rust/kei-registry/src/cli.rs` — new `Status` variant
with `--db`, `--git-repo`, `--ledger-db`, `--format` flags.
- `_primitives/_rust/kei-registry/src/handlers.rs` — `handle_status`
dispatcher, ASCII/JSON branching.
- `_primitives/_rust/kei-registry/src/lib.rs` — module export.
End-to-end run from kit root shows the prior gap: 17 local branches
(many `worktree-agent-*` orphans), kei-ledger summary 4 running /
158 done / 35 failed / 7 merged / 0 rejected — visibility the user
asked for ("в каждой сессии видеть, чтобы не бегать по диску в
поисках несмерженных").
What this does NOT do (Phase 4):
- No orphan detection (`kei-status orphans`) — counts only.
- No auto-registration of branches into kei-ledger (Phase 2). Branches
come from live `git for-each-ref` shell-out; if the repo moves or
is deleted the row vanishes from the dashboard. Acceptable for v1.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- Phase 2 (post-checkout hook → kei-ledger auto-register)
- Phase 4 (orphan detection: branches with no commits in N days,
path-atoms with no consumers, agent forks stuck running)
- --filter flags (--type, --status) for targeted queries
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root-cause of the username-path leak in DNA-INDEX.md (107 atom rows
in v0.17 — sed-patched in 8fcba39). The encyclopedia render's
short_path() prefix list omitted every top-level dir except
`_primitives/`, `skills/`, `hooks/`, `rules/` — so atom and capability
rows fell through to the absolute path stored in the registry DB,
leaking the maintainer's home prefix into the public encyclopedia.
Fix: add `_blocks/`, `_manifests/`, `_generated/`, `_atoms/`,
`_assembler/`, `_roles/`, `_capabilities/`, `agents/`, `docs/` to
the prefix list. 8 unit tests cover the new prefixes (fixtures use
CI-style paths like `/srv/ci/build/...` so the source file does not
contain a maintainer-shaped path that would itself trip the local
pre-commit hook + leak-check CI).
Verified: regenerated docs/DNA-INDEX.md has 0 absolute-path hits.
Source fix supersedes the sed hot-fix in 8fcba39 — the next
`kei-registry encyclopedia` invocation will not regress.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
