b332b571bf
Closes the loop on "without full tracking the system can't make decisions" (user pushback on partial coverage). Three gaps that left the inference layer blind are now wired: GAP #1 — agent toolStats / token counts / cache hits captured ================================================================ `agent-outcome-backfill.sh` now appends one JSONL row per spawn to `~/.claude/memory/time-metrics/agent-toolstats.jsonl` with: agent_id, outcome, stubs, ts, tool_use_count, duration_ms, tool_stats {Read:N, Bash:M, ...}, tokens_in, tokens_out, cache_read, cache_write Sidecar journal (no schema migration). Production payload's .tool_response.totalToolUseCount / totalDurationMs / toolStats / usage fields land directly. Smoke-tested with synthetic spawn — row written. GAP #2 — skill_invocations table actually receives writes ================================================================ The `skill_invocations` table (schema v8) had 0 rows because no caller existed for `skill_metrics::record_invocation`. Added two pieces: (a) `kei-ledger record-skill <name> --success {0|1}` CLI subcommand Mirrors record-cost; same dispatch shape. Optional `--agent-id`, `--trajectory-id`, `--duration-ms`, `--db`. Validates non-empty name + duration ≥ 0. Outputs `{"ok":true,"skill":"...","ts":N}`. (b) `hooks/skill-record.sh` — PostToolUse:Skill hook. 50 LOC POSIX. Detects Skill tool calls, derives success heuristic from tool_response (exit_code / status / content non-empty), shells out to `kei-ledger record-skill`. Bypass via SKILL_RECORD_BYPASS=1. 83 kei-ledger tests pass (16 unit + 67 integration). Smoke-tested end-to-end: `kei-ledger record-skill test-skill --success 1` inserts a row with correct fields. Phase D nightly skill-metrics decisions (archive if unused N days, re-extract if success<60% over M days, validated if >20 calls + >90% success) now have data to consume. GAP #3 — numeric-claims.jsonl receives every evidence-tagged claim ================================================================ RULE 0.18 mandated three markers `[REAL:]` / `[FROM-JOURNAL:]` / `[ESTIMATE-HTC:]` on every numeric/duration/cost claim, but no hook appended valid claims to the journal — the calibration data RULE 0.18 promised never accumulated. `hooks/numeric-claims-record.sh` — Stop hook, 140 LOC POSIX. Reads transcript_path from stdin, locates the last assistant message via recursive flatten (same pattern as agent-outcome-backfill.sh after the production-payload-shape fix), regex-extracts every `<phrase> [<TIER>: <pointer>]` triple, appends one JSONL row per claim. Idempotent within 1-second window to avoid double-recording on repeat Stop fires. Bypass via NUMERIC_CLAIMS_RECORD_BYPASS=1. Smoke test: synthetic transcript with 3 markers (REAL + ESTIMATE-HTC + FROM-JOURNAL) produced exactly 3 well-formed JSONL rows. Settings.json ================================================================ - PostToolUse:Skill matcher created (or augmented if already present) with skill-record.sh. - Stop:* matcher gains numeric-claims-record.sh after the existing chain (stop-verify, task-timer, session-end-dump, extract-task- durations, chat-numeric-postflag, affect-threshold-check, enrich-from-jsonl). What this does NOT do (deferred): - Backfill `skill_invocations` from past traces (history started today; Phase D cohort builds forward from now). - Migrate the agent toolStats sidecar JSONL into a proper ledger column. Append-only file is fine for the current scale. - Refactor main.rs (now 233 LOC, was 212; pre-existing CP debt flagged by skill-record agent — separate cleanup PR). === STATUS-TRUTH MARKER === shipped: functional stubs: 0 cargo-check: PASS behaviour-verified: yes follow-up-required: - kei-ledger main.rs Constructor Pattern split (212→233 LOC) - Verify in next session: skill_invocations gets rows from real Skill tool use; numeric-claims.jsonl gets rows from real assistant messages with markers Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| _rust | ||
| templates | ||
| design-scrape.sh | ||
| figma-tokens.sh | ||
| frontend-inspect.sh | ||
| harden-base.sh | ||
| kei-ci-lint.sh | ||
| kei-docs-scaffold.sh | ||
| kei-doctor.sh | ||
| kei-sleep-queue.sh | ||
| kei-sleep-setup.sh | ||
| kei-sleep-sync.sh | ||
| live-preview.sh | ||
| log-ship.sh | ||
| MANIFEST.toml | ||
| metrics-scrape.sh | ||
| provision-hetzner.sh | ||
| provision-vultr.sh | ||
| README.md | ||
| screenshot-decode.sh | ||
| tomd.sh | ||
_primitives — first-class building blocks
_primitives/ holds standalone utilities that agents, hooks, and skills
(including /compose-solution) depend on. Unlike _blocks/ (behavioral
markdown) or _manifests/ (agent TOML), primitives are executable shell
programs installed at $HOME/.claude/agents/_primitives/ by install.sh.
Current primitives
| Primitive | Purpose | Invocation |
|---|---|---|
tomd.sh |
Universal non-native-format → markdown converter (PDF, DOCX, XLSX, PPTX, CSV, images, code). | ~/.claude/agents/_primitives/tomd.sh <file> |
tomd.sh is a first-class primitive. Universal non-native-format →
markdown converter with configurable cache directory
(KEISEI_TOMD_CACHE) and KeiSeiKit-style error tags ([tomd]).
Hook integration
hooks/tomd-preread.sh is a PreToolUse(Read) hook that auto-redirects
Claude to the converted markdown when a Read targets .docx / .doc / .xlsx / .pptx / .csv. Cached under $KEISEI_TOMD_CACHE (default
/tmp/keisei-tomd-cache).
/compose-solution discovery
Phase 3 prior-art sweep greps _primitives/ alongside _blocks/,
_manifests/, skills/, _bridges/, hooks/. If a user task involves
file-format parsing, the meta-composer surfaces tomd automatically —
reuse over rewrite (RULE "No Patching").