feat(outcome-hook): PostToolUse:Agent backfills outcome + stubs in ledger

Phase 4 follow-up: the outcome-backfill hook the kei-model-router
needs to learn from. Without an outcome signal the Beta posterior
sees 205 NULL rows and can never converge → router falls back to
top-tier on every spawn. This hook closes that loop.

Spawned by orchestrator as a code-implementer agent (Sonnet 4.6 by
default after the manifest refactor in cb1fdde — first dogfood proof
that the tier system works end-to-end). Agent returned cleanly with
STATUS-TRUTH MARKER `shipped: functional, stubs: 0`.

Files (3):
- `~/.claude/hooks/agent-outcome-backfill.sh` (73 LOC, /bin/sh) —
  reads PostToolUse:Agent stdin JSON, parses STATUS-TRUTH MARKER from
  `tool_response`, runs `UPDATE agents SET outcome = ?, stubs_count = ?
  WHERE id = ?` via sqlite3 CLI. Defensive on every step (never blocks,
  exits 0 on missing jq / sqlite3 / DB / marker). Bypass:
  `OUTCOME_BACKFILL_BYPASS=1`. Lives outside the kit (system-level).
- `tests/hook-outcome-backfill-test.sh` (79 LOC, /bin/sh) — 8 assertions
  cover: 4 valid outcomes, idempotent re-run, missing marker, bypass
  env, missing sqlite3 (PATH stripped). Run via
  `sh tests/hook-outcome-backfill-test.sh` → "Passed: 8 Failed: 0".
- `_blocks/path-user-hooks.md` — third path-atom following
  user-memory / user-rules convention. Resolves to `~/.claude/hooks/`.
  Lets future manifests reference hook files via
  `path:user-hooks/<file>.sh` opaquely. Registered in registry as
  `atom::md::331b9a34::023e5a08`.

Wiring:
- `~/.claude/settings.json` PostToolUse:Agent matcher chain — appended
  the hook idempotently (jq update preserves existing
  `agent-stub-scan.sh`, `task-timer.sh`, `agent-fork-done.sh`).
- DNA-INDEX regenerated; new path-atom appears in `## Atom (120)`
  section.

Effect: every Agent tool call from now on writes outcome + stubs to
ledger. After ~10-20 invocations the Beta posterior has a usable
prior; after ~50 the router stops defaulting Sonnet to Opus on
unfamiliar tasks. The advisor hook (`model-router-advisor.sh`)
already prints stderr when current model > recommended — orchestrator
needs to actually pass `model:` parameter on next spawn (behavioural,
not a code change).

=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN
behaviour-verified: yes
follow-up-required:
  - PR feat/substrate-path-atoms-2026-05-01 → main when ready

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Parfii-bot 2026-05-01 23:24:02 +08:00
parent cb1fddeabb
commit be1a864629
3 changed files with 116 additions and 7 deletions

View file

@ -0,0 +1,28 @@
---
type: atom
kind: path
name: user-hooks
template: ~/.claude/hooks
expand_at: render
---
# Path atom — user-hooks
Resolves to the user's `~/.claude/hooks/` directory (PreToolUse / PostToolUse / UserPromptSubmit / Stop hook scripts like `agent-stub-scan.sh`, `agent-outcome-backfill.sh`, `numeric-claims-guard.sh`, etc.).
Used by agent manifests (`_manifests/*.toml`) to reference hook scripts without leaking the absolute path (with the maintainer's home `/Users/<user>/...`) into public artefacts under `_generated/`.
**Usage in manifests:**
```toml
[references]
extra = [
"path:user-hooks/agent-outcome-backfill.sh",
"path:user-hooks/agent-stub-scan.sh",
]
```
**Resolution:** the assembler detects the `path:user-hooks/` prefix, looks up this atom in the registry, and emits an opaque DNA reference into the rendered `_generated/<agent>.md`. Same content-addressing semantics as `path-user-memory` and `path-user-rules` — published artefact has DNA hashes, not paths. A reader with a local kit + registry resolves the DNA back to the file; a reader without the kit sees only the opaque hash.
**Expand timing:** `render` — substitution happens at `_assembler` time, before the `_generated/` markdown is written.
**Constructor Pattern:** one cube, one path. No code, no logic. Body bytes + frontmatter ARE the atom. Hash → DNA via standard registry pipeline.

View file

@ -1,11 +1,11 @@
# KeiSeiKit DNA Encyclopedia # KeiSeiKit DNA Encyclopedia
> Auto-generated from kei-registry. Last regenerated: 2026-05-01T14:25:35Z. > Auto-generated from kei-registry. Last regenerated: 2026-05-01T15:21:20Z.
> Total blocks: 507. Per-type breakdown: > Total blocks: 509. Per-type breakdown:
| Type | Count | | Type | Count |
|---|---:| |---|---:|
| atom | 119 | | atom | 121 |
| hook | 40 | | hook | 40 |
| primitive | 106 | | primitive | 106 |
| rule | 174 | | rule | 174 |
@ -882,7 +882,7 @@ Sorted alphabetically by name.
| task-timer | shell | hook::shell::dda5e94… | hooks/task-timer.sh | | task-timer | shell | hook::shell::dda5e94… | hooks/task-timer.sh |
| tomd-preread | shell | hook::shell::8a95b76… | hooks/tomd-preread.sh | | tomd-preread | shell | hook::shell::8a95b76… | hooks/tomd-preread.sh |
## Atom (119) ## Atom (121)
Sorted alphabetically by name. Sorted alphabetically by name.
@ -984,6 +984,7 @@ Sorted alphabetically by name.
| foo | atom::md::63a73aa1::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa | | foo | atom::md::63a73aa1::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa |
| foo | atom::md::0f507ef3::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa | | foo | atom::md::0f507ef3::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa |
| foo | atom::md::40c9240c::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa | | foo | atom::md::40c9240c::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa |
| foo | atom::md::077f9361::… | _primitives/_rust/kei-registry/tests/fixtures/atom-sample/atom.md | 309b88fa |
| git-ops | atom::_::6da713d3::d… | _roles/git-ops.toml | da80a8e7 | | git-ops | atom::_::6da713d3::d… | _roles/git-ops.toml | da80a8e7 |
| merger | atom::_::183b6add::a… | _roles/merger.toml | af2bf880 | | merger | atom::_::183b6add::a… | _roles/merger.toml | af2bf880 |
| output::merge-result | atom::output::d58ef5… | _capabilities/output/merge-result/capability.toml | 91cb9245 | | output::merge-result | atom::output::d58ef5… | _capabilities/output/merge-result/capability.toml | 91cb9245 |
@ -1004,6 +1005,7 @@ Sorted alphabetically by name.
| tools::cargo-only-bash | atom::_::692833ce::9… | _capabilities/tools/cargo-only-bash/capability.toml | 98e70f68 | | tools::cargo-only-bash | atom::_::692833ce::9… | _capabilities/tools/cargo-only-bash/capability.toml | 98e70f68 |
| tools::deny-tools | atom::tools::d64414a… | _capabilities/tools/deny-tools/capability.toml | 8f342dd8 | | tools::deny-tools | atom::tools::d64414a… | _capabilities/tools/deny-tools/capability.toml | 8f342dd8 |
| tools::read-only | atom::_::eded5636::2… | _capabilities/tools/read-only/capability.toml | 22bba452 | | tools::read-only | atom::_::eded5636::2… | _capabilities/tools/read-only/capability.toml | 22bba452 |
| user-hooks | atom::md::331b9a34::… | _blocks/path-user-hooks.md | 023e5a08 |
| user-memory | atom::md::1a771d51::… | _blocks/path-user-memory.md | b8f9e85f | | user-memory | atom::md::1a771d51::… | _blocks/path-user-memory.md | b8f9e85f |
| user-rules | atom::md::97292045::… | _blocks/path-user-rules.md | bc8e0acf | | user-rules | atom::md::97292045::… | _blocks/path-user-rules.md | bc8e0acf |
| verify::fork-audit | atom::verify::81e519… | _capabilities/verify/fork-audit/capability.toml | 3fb8694d | | verify::fork-audit | atom::verify::81e519… | _capabilities/verify/fork-audit/capability.toml | 3fb8694d |
@ -1025,7 +1027,7 @@ Sorted alphabetically by name.
- `alignment-check` — 2 versions: 4e7389b1 → b1e18549 - `alignment-check` — 2 versions: 4e7389b1 → b1e18549
- `extract-task-durations` — 2 versions: e6854ef5 → 859873eb - `extract-task-durations` — 2 versions: e6854ef5 → 859873eb
- `firewall-diff` — 2 versions: e42f1e32 → 8260ffc0 - `firewall-diff` — 2 versions: e42f1e32 → 8260ffc0
- `foo` — 10 versions: 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa - `foo` — 11 versions: 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa → 309b88fa
- `frustration-matrix` — 2 versions: 0923b30a → d51e63c8 - `frustration-matrix` — 2 versions: 0923b30a → d51e63c8
- `kei-agent-runtime` — 2 versions: 708830d4 → 33b44d6c - `kei-agent-runtime` — 2 versions: 708830d4 → 33b44d6c
- `kei-artifact` — 2 versions: 2c55b84a → a33abf97 - `kei-artifact` — 2 versions: 2c55b84a → a33abf97
@ -1106,8 +1108,8 @@ Sorted alphabetically by name.
- `kei-provision` — 2 versions: 1d613e5d → cfa53bb3 - `kei-provision` — 2 versions: 1d613e5d → cfa53bb3
- `kei-prune` — 2 versions: 7c0a0c11 → 4454513b - `kei-prune` — 2 versions: 7c0a0c11 → 4454513b
- `kei-refactor-engine` — 2 versions: 90048888 → 92e83ce0 - `kei-refactor-engine` — 2 versions: 90048888 → 92e83ce0
- `kei-registry`2 versions: 7d9570ad → 5a2e79d8 - `kei-registry`3 versions: 7d9570ad → 5a2e79d8 → 5a2e79d8
- `kei-registry::kei-registry`12 versions: a9d4104f → 4110ba86 → 6e2dc3fd → 1f486539 → f10a08ba → 48886c98 → 6aeaf85c → ca0c09e0 → 130372c0 → f69680b3 → 50364568 → 30e6dee3 - `kei-registry::kei-registry` — 21 versions: a9d4104f → 4110ba86 → 6e2dc3fd → 1f486539 → f10a08ba → 48886c98 → 6aeaf85c → ca0c09e0 → 130372c0 → f69680b3 → 50364568 → 30e6dee3 → 3bb6d4f8 → 26a25696 → 0951d355 → 3261f321 → 5a190e74 → 80762a78 → d2bd49f3 → 99859be7 → b134cecf
- `kei-replay` — 2 versions: 420ceb46 → 74f2fcc4 - `kei-replay` — 2 versions: 420ceb46 → 74f2fcc4
- `kei-router` — 2 versions: fc8c6820 → 2cfaa362 - `kei-router` — 2 versions: fc8c6820 → 2cfaa362
- `kei-router::kei-router` — 15 versions: 186634e6 → d91e8a11 → 80d4f8c6 → f8677f1d → a2e47f61 → 299a5afe → 675effa4 → 1fa6b4bb → 89c81c79 → 29340bbb → 51682c29 → ec0a1bfb → f4fce214 → 184e4f53 → 98ab93cd - `kei-router::kei-router` — 15 versions: 186634e6 → d91e8a11 → 80d4f8c6 → f8677f1d → a2e47f61 → 299a5afe → 675effa4 → 1fa6b4bb → 89c81c79 → 29340bbb → 51682c29 → ec0a1bfb → f4fce214 → 184e4f53 → 98ab93cd

View file

@ -0,0 +1,79 @@
#!/bin/sh
# hook-outcome-backfill-test.sh — exercises agent-outcome-backfill.sh
# against a temp ledger DB. Asserts UPDATE behaviour for the 4 outcomes,
# missing-marker no-op, bypass no-op, and missing-sqlite3 no-op.
set -u
HOOK="$HOME/.claude/hooks/agent-outcome-backfill.sh"
[ -x "$HOOK" ] || { echo "FAIL: hook not executable at $HOOK"; exit 1; }
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
DB="$TMP/ledger.sqlite"
export KEI_LEDGER_DB="$DB"
sqlite3 "$DB" "CREATE TABLE agents (
id TEXT PRIMARY KEY,
outcome TEXT CHECK (outcome IN ('functional','partial','scaffolding','fail')),
stubs_count INTEGER DEFAULT 0
);"
PASS=0; FAIL=0
assert_eq() {
if [ "$1" = "$2" ]; then PASS=$((PASS+1));
else FAIL=$((FAIL+1)); echo " FAIL: $3 — got '$1' expected '$2'"; fi
}
run_case() {
# $1=id $2=shipped $3=stubs_count_in_marker
sqlite3 "$DB" "INSERT OR REPLACE INTO agents(id,outcome,stubs_count) VALUES('$1',NULL,0);"
BODY="prelude text
=== STATUS-TRUTH MARKER ===
shipped: $2
stubs: $3
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- none"
PAYLOAD=$(jq -nc --arg id "$1" --arg body "$BODY" \
'{tool_use_id:$id, tool_response:$body}')
printf '%s' "$PAYLOAD" | "$HOOK"
OUT=$(sqlite3 "$DB" "SELECT outcome||'|'||stubs_count FROM agents WHERE id='$1';")
assert_eq "$OUT" "$2|$3" "outcome=$2"
}
echo "[1] 4 valid outcomes update correctly"
run_case "id-func" "functional" 0
run_case "id-part" "partial" 3
run_case "id-scaf" "scaffolding" 7
run_case "id-fail" "fail" 12
echo "[2] idempotent re-run produces same row"
run_case "id-func" "functional" 0
echo "[3] missing marker → no-op"
sqlite3 "$DB" "INSERT OR REPLACE INTO agents(id,outcome,stubs_count) VALUES('id-bare',NULL,0);"
printf '%s' '{"tool_use_id":"id-bare","tool_response":"just a plain reply"}' | "$HOOK"
OUT=$(sqlite3 "$DB" "SELECT IFNULL(outcome,'NULL')||'|'||stubs_count FROM agents WHERE id='id-bare';")
assert_eq "$OUT" "NULL|0" "no-marker no-op"
echo "[4] bypass env → no-op"
sqlite3 "$DB" "INSERT OR REPLACE INTO agents(id,outcome,stubs_count) VALUES('id-byp',NULL,0);"
BODY="=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0"
PAYLOAD=$(jq -nc --arg id "id-byp" --arg body "$BODY" '{tool_use_id:$id,tool_response:$body}')
printf '%s' "$PAYLOAD" | OUTCOME_BACKFILL_BYPASS=1 "$HOOK"
OUT=$(sqlite3 "$DB" "SELECT IFNULL(outcome,'NULL') FROM agents WHERE id='id-byp';")
assert_eq "$OUT" "NULL" "bypass no-op"
echo "[5] missing sqlite3 → no-op (PATH stripped)"
sqlite3 "$DB" "INSERT OR REPLACE INTO agents(id,outcome,stubs_count) VALUES('id-nosql',NULL,0);"
PAYLOAD=$(jq -nc --arg id "id-nosql" --arg body "$BODY" '{tool_use_id:$id,tool_response:$body}')
JQ_DIR=$(dirname "$(command -v jq)")
printf '%s' "$PAYLOAD" | env -i HOME="$HOME" KEI_LEDGER_DB="$DB" PATH="$JQ_DIR" "$HOOK" 2>/dev/null
OUT=$(sqlite3 "$DB" "SELECT IFNULL(outcome,'NULL') FROM agents WHERE id='id-nosql';")
assert_eq "$OUT" "NULL" "no-sqlite3 no-op"
echo "Passed: $PASS Failed: $FAIL"
[ "$FAIL" -eq 0 ] || exit 1