Pre-public Phase 1. Remove personal/IP traces that should not ship in a
general-purpose kit; keep only intended author attribution.
- no-github-push.sh + hooks-and-blocks.md + ci-scaffold: drop "KeiTech
unfiled patent IP / trade secrets / priority date" wording; reword as a
generic opt-in guard for keeping code on a private remote.
- check-error-patterns.sh: remove author-local absolute path from the
tombstone comment.
- graph-export-watcher.sh: default viz dir to ~/.local/share/kei/graph-viz
(was a personal project path).
- agent manifests (cost-guardian, modal-runner, infra/ml/code-implementer)
+ ci.yml: strip private memory references and dated personal incidents;
keep the generic cost/ops lessons. Snapshots regenerated; golden 3/3.
Kept intentionally: author attribution (NOTICE / README / Cargo / plugin).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Any Claude Code session can message any other (not just Agent-Teams teammates),
no tmux. Append-only jsonl bus + UserPromptSubmit hook pulling unread per turn.
kei-message.sh (send/inbox/list, address by cwd-basename or "all"),
mailbox-inject.sh (cursor dedup, first-turn baseline, no self-echo), bin/kei
`message` dispatch, lib-scaffold copies all scripts/*.sh, snippet wires the hook.
Bypass KEI_MAILBOX_BYPASS=1. Verified by 2-session simulation.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SessionStart first-run hook now injects an ordered post-install checklist Claude
walks the user through: (1) /onboard projects → per-project agents,
(2) /sleep-setup → nightly REM (recommend local-only), (3) /cortex-setup (only
if cortex daemon installed). Confirm + run each, skippable. Fires once.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. SessionStart hook first-run-onboard.sh: on first Claude Code session after
install, nudges user/Claude to run `/onboard ~/Projects/*` (scan stack +
create a project-specialist agent per project; delegates to /new-agent).
Fires once (marker), then silent. Wired in settings-snippet under SessionStart.
2. lib-summary next-steps: lead with /onboard ~/Projects/*, then /new-agent.
3. lib-hooks merge: normalize null/absent matcher → "" (Claude Code /doctor
rejects null; pre-kit hooks often lack a matcher field).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
12-agent audit (2 waves Opus+Sonnet, 6 slices each) flagged 3 HIGH-tier
issues that BOTH waves agreed on, plus 5 doc-honesty findings. This
batch fixes the lot.
== CI green (was failing on main 1207cf5) ==
- _primitives/_rust/Cargo.toml — workspace tokio gains `io-std` feature
(needed by kei-mcp/src/main.rs which calls tokio::io::{stdin,stdout})
- _primitives/_rust/kei-mcp/Cargo.toml — dev-deps tokio gains `test-util`
feature (needed by tests/tools_call_timeout.rs for tokio::time::advance
and Builder::start_paused). Both verified locally:
`cargo check -p kei-mcp` ✓
`cargo test --no-run -p kei-mcp` ✓ (3 test binaries link)
[REAL: ran 2026-05-03 in this session]
== HIGH-tier audit fixes (consensus across waves) ==
1. SQLi escape in agent-outcome-backfill.sh:110
- 4 of 12 agents flagged: TOOL_USE_ID was JSON-derived and
interpolated raw into SQL. Allowlist on $SHIPPED protected today
but a future case-statement removal opened the surface.
- Fix: tiny `_sql_esc` helper that doubles single-quotes (SQL-99
standard escape), applied to SHIPPED + TOOL_USE_ID. STUBS already
integer-validated.
2. PRAGMA user_version=9 in install/sql/outcome-only-schema.sql
- W1 outcome-only critic flagged: the SQL fallback installed a
v9-equivalent flat schema but left user_version=0. A LATER
`kei-ledger init` (e.g. when user upgrades to full kit) would
re-run migrations v1-v9 and ALTER TABLE ADD COLUMN duplicate-error
mid-migration → broken DB.
- Fix: set PRAGMA user_version=9 before COMMIT so the binary's
migration runner sees current ≥ target and short-circuits.
3. backup_file mv→cp + uninstall macOS-portable awk
- W1+W2 outcome-only flagged: lib-backup.sh uses `mv` which DELETES
the target before _jq_merge_hooks runs; `|| true` swallowed the
subsequent jq read-error → silent settings.json loss.
- Fix in lib-profile-outcome-only.sh: `cp -p` aside, drop `|| true`,
return 1 on merge failure (trap restores).
- PROFILE-OUTCOME-ONLY.md uninstall used GNU sed `,+1` extension
which BSD sed (macOS) does not support — uninstall silently
no-op'd on macOS, leaving orphan CLAUDE.md text.
- Fix: replace with portable `awk` recipe; also added `rm -f` for
the agent-toolstats.jsonl sidecar (privacy completeness).
== Doc honesty pass (RULE 0.18 numerics + RULE 0.4 citations) ==
4. README.md count drift — verified all values against filesystem:
* 102→105 Rust crates (Cargo.toml workspace `members` count)
* 67→68 skills (`ls skills/ | wc -l`)
* 35→38 hooks (`grep -c '"command":' settings-snippet.json`)
* 37→38 agent manifests (`ls _manifests/*.toml | wc -l`)
* 82→85 substrate blocks (`find _blocks/ -name '*.md' | wc -l`)
* 18 capability atoms VERIFIED via `find _capabilities/ -name '*.md'`
(encyclopedia §3 row count of 17 is in a separate file and is a
known internal display issue, not changed in this commit)
* 495→565 active DNAs (per docs/DNA-INDEX.md header 2026-05-03)
Each value now carries a `[REAL: <command>]` style trailer per
RULE 0.18.
5. README.md DNA "80-char identity" → "≥33-char variable-length"
- W1+W2 reviewer-pass flagged FALSE: docs/DNA-FORMAT.md SSoT says
minimum 33 chars; 80 was nowhere in code or spec
- Fix in README.md:36 + docs/PHILOSOPHY.md:39 + docs/DNA-INDEX.md:1352
6. README.md "Eleven install profiles (... Cursor / Continue / Zed /
Aider / Docker / Nix)" — Cursor/Continue/Zed/Aider/Docker/Nix were
never install profiles, they were bridge targets
- Fix: list 12 actual profiles from _primitives/MANIFEST.toml,
mention bridges as separate concept
7. .claude-plugin/plugin.json license MIT → Apache-2.0
- W2-Sonnet reviewer flagged: LICENSE file is Apache-2.0 (since
2026-04-30 per NOTICE), but plugin.json still declared MIT —
plugin marketplace would show wrong license
8. docs/ARCHITECTURE.md:318 placeholder URL `https://example.invalid/...`
- W2-Sonnet reviewer flagged: dead link in published docs
- Fix: remove the bad href, describe ssl-rule-file as per-user
install outside the public repo
9. skills/sleep-on-it/SKILL.md Wagner et al. 2004 citation
- W1+W2 reviewer flagged RULE 0.4 violation: citation without
verification marker
- Fix: added [VERIFIED: doi:10.1038/nature02223] + clarification
that the original paper showed slow-wave-sleep (not strictly REM)
insight gain — our metaphor is a loose mapping
10. encyclopedia/substrate-overview.md §5 fabricated TS deps
- W1-Opus doc-consistency flagged RULE 0.4.b violation: 5 of 6
package rows had INVENTED dependency strings
(`recall-ai-sdk ^1.0.0`, `nodemailer-mock ^2.0.0`,
`telegram-typings ^4.10.0`, etc — none exist in the actual
package.json files)
- Fix: regenerated table from real `package.json` reads via
`node -p "require(...).dependencies"` for each of the 6 packages
- Fix: also corrected version drift (5 packages all 0.14.0 now)
Verification:
- Outcome-only end-to-end install against fake $HOME succeeds:
hooks installed, ledger schema at user_version=9, settings.json
created cleanly, all 5 documented files present
[REAL: ran 2026-05-03 in this session]
- `cargo check -p kei-mcp` + `cargo test --no-run -p kei-mcp` clean
Audit findings NOT yet addressed (deferred to next batch):
- README:65 git clone github URL — repo is private; reviewer flagged
external strangers cannot clone; will resolve via Quick Start rewrite
- npm.pkg.github.com / @keisei84 leftover sweep — both waves verified
ZERO refs, no fix needed
- safeEqual timing leak in TS server (W2 sec MEDIUM)
- HTTP server bind 0.0.0.0 (W2 sec MEDIUM)
- Unbounded request body (W2 ci MEDIUM)
- --dry-run silent ignored on non-outcome profiles (W1+W2 MEDIUM)
- Doc-link missing for MEMORY/DNA/LEDGER format specs from README
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three independent security hardenings from cross-cutting audits.
1. cortex /term PTY env leak + bind guard (HIGH — Sonnet Cross-cutting + Opus)
- kei-cortex/src/handlers/term_pty.rs: PTY spawn was inheriting daemon's
full process env (KEI_AUTH_KEY, ANTHROPIC_API_KEY, FAL_KEY, etc.) into
every authenticated /term shell. Combined with default cors_origin =
https://keisei.app, one stored XSS on keisei.app + one bearer token =
full local shell with all daemon secrets.
Added apply_safe_env() helper: env_clear() + re-set only HOME, PATH,
USER, LANG, TERM. Spawn helper invokes it before spawn_command.
- kei-cortex/src/main.rs: extracted build_config() helper; added
enforce_loopback_or_local_cors() guard called before serve.bind. Refuses
to start if bind addr is non-loopback AND cors_origin is a public
domain — prevents the XSS-to-shell scenario in production.
2. agent-stub-scan.sh stdin parsing (HIGH — multiple audits)
- hooks/agent-stub-scan.sh: previously read $CLAUDE_AGENT_TRANSCRIPT env
var which Claude Code does NOT set on PostToolUse:Agent. Hook silently
exited 0 — RULE 0.16 enforcement was dead-code in production.
Rewrote to read stdin JSON via jq, flatten .tool_response recursively
(string|array|object via the same pattern as agent-event-done.sh),
guard on .tool_name == "Agent" and command -v jq. Maintained WARN-tier
exit-0 with TODO marker for ENFORCE flip on 2026-05-05 (per RULE 0.16
§2 ladder).
3. magiclink revoke() silent no-op (HIGH — Opus Rust + Sonnet Cross-cutting)
- kei-auth-magiclink/src/{error,provider}.rs: revoke() previously returned
Ok(()) without doing anything. Operators expecting "revoke a session"
semantics from the AuthProvider trait got false success. Stolen magic-
link URLs remained valid until the 15-minute TTL.
Added Error::Unsupported variant. revoke() now returns
Err(Unsupported(...)) with explicit guidance: "rotate KEI_MAGICLINK_HMAC_
KEY to invalidate all live tokens, or maintain a deny-list at the caller
layer". Test provider_revoke_returns_unsupported_error confirms the
error variant is wired.
Tests: cargo check + cargo test both PASS. 444 functional tests across
kei-cortex (428 lib) + kei-auth-magiclink (16 lib + smoke). Pre-existing
openai_loop_wiring.rs 502 failures in routes/openai/{chat,responses}.rs are
NOT introduced by these fixes — separate unrelated triage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three independent shell hardening fixes from Opus Shell + Sonnet Shell audits.
1. disk-reclaim.sh Guard 3 — protect branches without upstream tracking (HIGH)
File: hooks/disk-reclaim.sh:88-101
Bug: when a worktree branch has no upstream tracking ref, `git log @{u}..`
exited non-zero and `unpushed=""` (empty). The check
`[ -n "$unpushed" ] && [ "$unpushed" != "0" ]` evaluated FALSE, so the
worktree fell through Guard 3 and was eligible for mtime-based pruning.
Local-only branches with committed work were silently deleted.
Fix: explicit two-branch logic. Run `git rev-parse --abbrev-ref @{u}` first;
only run the unpushed-count check if upstream exists. If no upstream, log
SKIP[no-upstream] and `continue` conservatively. New
`worktrees_skip_unpushed` counter increments in both unpushed paths.
2. secrets-pre-guard.sh — placeholder allowlist scope-narrow (MEDIUM)
File: hooks/secrets-pre-guard.sh:43-103
Bug: word "placeholder" anywhere in content disabled all secret-pattern
scanning for that whole Write. Allowlist was too broad — a doc with the
word "placeholder" in its prose could mask a real sk-ant- token elsewhere.
Fix: replaced global early-exit with per-line awk scan. New scan_pattern()
helper walks content line-by-line; each line matching a secret regex is
allowed ONLY if the SAME line also matches ALLOWLIST_RE. Doc prose can no
longer mask cross-line secrets. Added `dummy[_-]?(key|token|secret)` to
allowlist for legitimate test fixtures.
3. lib-rust-prebuild.sh — sha256 fail-closed (HIGH supply-chain)
File: install/lib-rust-prebuild.sh:75-88
Bug: when ${url}.sha256 404'd, installer printed WARNING and proceeded with
unverified tarball. A compromised github release uploader could ship a
malicious tarball, omit .sha256, and the installer would extract it into
~/.cargo/bin/.
Fix: missing .sha256 → ERROR + abort. Path A install fails → falls back to
Path B (cargo build from source). Override via KEI_ALLOW_UNVERIFIED_TARBALL=1
(visible per-call, intentional friction).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pushback: live-graph showed only "main" node, no pulses on agents.
Root cause: hook stdin doesn't carry parent_tool_use_id for sub-agent
tool calls — we only get the sub-agent's own session_id, which doesn't
link back to the spawn's tool_use_id.
Sequential heuristic via shared state file:
- agent-event-spawn.sh appends tool_use_id to /tmp/kei-active-children.tsv
- tool-use-event.sh reads the LAST line of that file → uses that
tool_use_id as agent_id for the emitted event
- agent-event-done.sh removes the spawn's line (grep -v + atomic mv)
Verified end-to-end: a code-implementer agent ran 5 Bash calls during
its lifetime — all 5 tool_use events were correctly attributed to the
spawn's tool_use_id. After agent_done, subsequent orchestrator-direct
tool calls correctly fall back to agent_id="main".
Limitation: parallel agents may misattribute. The "most recent live
spawn" heuristic works for single-agent-at-a-time which is the common
case. Parallel spawns share /tmp/kei-active-children.tsv and a sub-
agent's tool calls all attribute to whichever spawn appended last.
Acceptable for v1 demo; proper parent-tool-use-id propagation requires
Claude Code to expose it in sub-agent stdin (upstream change).
The `mv` after `grep -v` runs UNCONDITIONALLY (not gated on grep's
exit code) — grep -v returns 1 when ALL lines match, which would
otherwise leave the stale file in place.
Bypass: `KEI_EVENTS_BYPASS=1` (existing) covers all 3 hooks.
Override path: `KEI_ACTIVE_SPAWNS_FILE=/path/to/file`.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN
behaviour-verified: yes
follow-up-required:
- Parallel-agent attribution would need parent_tool_use_id from
Claude Code sub-agent stdin (not currently exposed).
- Race condition window between spawn append and done remove is
millisecond-scale; observed clean in single-agent demo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pushback: "транслирует в онлайне какие агенты создаются? основное
окно агента, а дальше при запусках появляются новые ветки, мы показываем
в онлайне как агенты собираются и работают"
Earlier `kei-graph-export` rendered the static SUBSTRATE (all 581 atoms,
catalog-style). User wanted the LIFECYCLE: orchestrator at center, every
new agent as a fading-in branch, every tool call as a pulse, every
completion as a fade-out. TTL = until done; pure online, no history
accumulation per user direction.
Three-layer architecture, all conforming to schema /tmp/agent-events-schema.md:
LAYER 1 — Event emitters (4 hooks)
hooks/agent-event-spawn.sh PreToolUse:Agent → agent_spawn event
hooks/agent-event-done.sh PostToolUse:Agent → agent_done event
(parses STATUS-TRUTH MARKER for outcome,
computes cost_usd from token×pricing table)
hooks/tool-use-event.sh PreToolUse:Bash|Read|Edit|Write|Grep|Glob|NotebookEdit
→ tool_use event
hooks/skill-record.sh EXTENDED — second emit step writes skill_use
event in addition to existing kei-ledger
record-skill call
All 4 are POSIX /bin/sh, defensive (never block, exit 0), bypass via
KEI_EVENTS_BYPASS=1. Append-only JSONL to
~/.claude/memory/agent-events.jsonl.
Smoke: 4 synthetic invocations cover spawn/done/tool/filter cases.
LAYER 2 — kei-graph-stream Rust daemon
_primitives/_rust/kei-graph-stream/ (~480 LOC, 5 files + 1 test)
- Tails events.jsonl every 200ms (poll-based, no notify dep).
- Parses each event, updates AliveState (insert on spawn, remove on done).
- Broadcasts {"type":"event","data":<event>} to all WebSocket clients.
- On client connect: sends {"type":"snapshot","alive":[...]} first.
- Heartbeat: {"type":"ping"} every 30s.
- axum 0.7 + ws feature (already in Cargo.lock via kei-cortex).
- Bypass: KEI_GRAPH_STREAM_BYPASS=1.
Bound to 127.0.0.1:8201 (loopback only). Endpoints:
GET /stream → WebSocket upgrade
GET /health → "kei-graph-stream alive"
4 unit + 1 integration test. cargo build clean.
Installed binary: ~/.cargo/bin/kei-graph-stream
Launchd plist: io.keisei.graph-stream (RunAtLoad, KeepAlive)
Loaded as PID 52678, /health 200 OK verified.
LAYER 3 — live-graph.html (single-file frontend)
~/Projects/lbm-graph-viz/live-graph.html (~464 LOC, self-contained)
- SVG full-viewport, dark #0f172a, CSS grid background.
- Pinned center node "main" (orchestrator), gold #fbbf24, glowing.
- Agents radiate via D3 force-simulation; color-by-model
(sonnet=green, opus=red, haiku=blue, default=gray).
- On agent_spawn: fade-in 300ms, edge from main to new node.
- On tool_use: pulse on agent node (r 8→12→8 over 400ms) +
floating tool name label fades 800ms.
- On agent_done: outcome-color flash → fade-out 800ms → remove.
- WebSocket client: ws://127.0.0.1:8201/stream, exponential-backoff
reconnect (1s→30s).
- Top-right status badge: ● connected | ○ reconnecting | ✕ disconnected.
- Bottom counters: alive / spawned / tool calls / done / last event age.
- No build step. D3 v7 from CDN. Pure HTML+JS+CSS.
End-to-end smoke (this machine, just now):
- daemon health 200 OK
- hook injected agent_spawn → daemon broadcasts → AliveState=1
- hook injected agent_done → daemon broadcasts → AliveState=0
- frontend file syntax-checked clean
What this does NOT do (deferred, by user direction "это онлайн"):
- History persistence — agents who finished are GONE from the graph.
Per-session log remains in events.jsonl + sleep-sync if user wants
to consult later, but the live view is RIGHT NOW only.
- Sub-agent attribution beyond "main" — orchestrator-direct tool calls
show on the orchestrator node. Sub-agent's internal tool calls would
need session-id correlation; current schema has agent_id="main"
placeholder for non-Agent tool calls.
- Replay mode — no time-scrubber. Possible follow-up if useful.
- Auth on WebSocket — bound to 127.0.0.1 only. Local-only by design.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- Sub-agent tool-call attribution (correlate session_id chain)
- Replay mode with time scrubber (if user finds use)
- Tool aggregator nodes ("Bash bucket" with N) instead of per-agent pulses
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pushback: "можно нашего Кейси подключить к обсидиан? будет в
онлайне строить граф из всех наших агентов?"
Closer-to-question architecture: don't build new Obsidian plugin —
re-use the legacy `~/Projects/lbm-graph-viz/` D3 viewer (lineage:
keicode → living-graph → lbm → lbm-graph-viz → keisei-graph). Strip
its Hebbian/co-change edges, replace with DNA-derived edges from the
kei-registry + kei-ledger. Open in any browser, file://...index.html.
NEW Rust crate `_primitives/_rust/kei-graph-export/` (~440 LOC, 5 files)
Reads:
~/.claude/registry.sqlite (730 active blocks)
~/.claude/agents/ledger.sqlite (6 agents post-cleanup)
_manifests/*.toml (38 agent manifests)
Emits 581-node, 291-edge graph. Edge types:
block_dep 171 manifest → atom (blocks=[])
path_ref 99 manifest → atom (path:NAME refs)
branch_lineage 11 parent_branch → branch
agent_uses_manifest 10 agent → manifest (slug from branch name)
Output formats:
--format spaces-fragment → `window.RUNTIME_SPACE = {...}` JS file
--format json → raw {nodes, links} for downstream tools
Block-name lookup is multi-resolution: each block is registered under
display name + lowercased + file-stem slug (from path basename) so
manifest references like `blocks = ["baseline"]` resolve to a registry
row whose `name` column holds "BASELINE — inherit from Main Claude".
Without this fix the graph had 0 block_dep edges; with it, 171.
NEW background updater `hooks/graph-export-watcher.sh` + launchd plist
template `_primitives/templates/io.keisei.graph-export.plist`
5-second loop:
while true; do
kei-graph-export --format spaces-fragment --output <viz>/data-runtime.js.tmp
mv <viz>/data-runtime.js.tmp <viz>/data-runtime.js # atomic
sleep 5
done
launchd plist substitutes `HOME_DIR` and `HOOKS_DIR` placeholders at
install time. RunAtLoad=true, KeepAlive=true. Logs to
~/.claude/memory/graph-export.log. Bypass: GRAPH_EXPORT_BYPASS=1.
Loaded into user-side launchd (PID 16474 confirmed running). File
mtime advances every 5s — live updates verified.
PATCH `~/Projects/lbm-graph-viz/index.html` (outside kit, surgical)
Three changes:
1. Add `<script src="data-runtime.js">` BEFORE `spaces.js` (window
global available when SPACES is defined).
2. After spaces.js: `if (window.RUNTIME_SPACE) SPACES.runtime = window.RUNTIME_SPACE;`
3. Auto-refresh setInterval(5s): fetch data-runtime.js, eval (re-
assigns window.RUNTIME_SPACE), hash-compare, re-render via
`rebuildGraph()` if currently viewing the runtime space.
window.RUNTIME_SPACE (not const RUNTIME_SPACE) avoids the
"const cannot be re-declared" error on subsequent eval() calls.
Effect: open file://~/Projects/lbm-graph-viz/index.html in any
browser, switch to "Runtime" space — full DNA graph of every agent /
atom / skill / branch / manifest / hook / primitive / rule, force-
laid-out by D3. Updates every 5 seconds without page reload.
What this does NOT do (deferred):
- Obsidian mirror — separate work, would emit .md per node into
~/Projects/KeiSeiVault/. Useful for backlinks navigation but
file-watcher latency similar to current 5s polling.
- Skill-invocation edges — table is empty until next Skill tool
use; will populate naturally.
- Scoped queries (orphan finder, hot-path PageRank). Out of scope
for v1; the JSON --format export feeds any downstream tool.
- `agent_uses_manifest` heuristic warns on unknown subagent slugs
(e.g. `physics-deriver` with no manifest yet). Non-fatal.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- Obsidian vault mirror (Phase C, separate work)
- Skill-edges populate from real Skill use (not blockered)
- Hot-path PageRank highlighting in viewer (cosmetic)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pushback: "Агент должен делать осмысленные выводы! С утра должен
быть отчет и пусть он приходит куда-то! На телеграмм, например, лучше
сразу после фазы сна, бот есть"
Wires the @KeiSeiBot Telegram bot as the delivery channel for nightly
Phase B reports, with a Claude Sonnet 4.6 reasoning step in front to
distil the multi-section markdown into a single actionable brief.
NEW — `hooks/sleep-report-tg.sh` (130 LOC POSIX bash)
Pipeline:
1. Source ~/.claude/secrets/.env (umbrella SSoT — RULE 0.8)
2. POST report markdown to Claude API messages endpoint with a
system prompt mandating: TL;DR + numbers + 3-5 actionable
findings + rule-candidates if any cross-session pattern ≥3×.
Sonnet 4.6, max_tokens=1500, 120s timeout.
3. Send distilled summary via Telegram sendMessage to whitelisted
chat_id (defaults to TELEGRAM_ALLOWED_CHAT_ID env, falls back
to 86059912).
4. Cap message at 3900 chars (TG limit 4096).
5. Fallback if Markdown parse_mode fails (orphan * / [ in body) →
retry without parse_mode so the user still sees the report.
6. Defensive on every step: missing API key → send raw excerpt;
missing curl/jq → log + exit 0; HTTP failure → log + exit 0.
7. Bypass: SLEEP_REPORT_TG_BYPASS=1.
WIRE — `hooks/phase-b-rem.sh`
Step 7 (new) calls sleep-report-tg.sh after the existing commit/push
step. Failure of TG delivery never affects Phase B's exit code —
the local report + memory-repo push remain the source-of-truth;
TG is convenience.
CONFIG (already done outside this commit, documented for completeness)
- ~/.claude/secrets/.env now has TELEGRAM_BOT_TOKEN +
TELEGRAM_ALLOWED_CHAT_ID (single-user whitelist 86059912).
- ~/.claude/tg-webhook.py whitelist locked to {86059912}; group
chat (-1003758632751) and partner (10954083) removed per
user request "сделай боту только один вайт адрес". Blocked
senders land in /var/log/tg-webhook/blocked.jsonl, no auto-reply.
- ~/.claude/tg-contacts.json shrunk from 3 contacts to 1.
Smoke verified: today's sleep-2026-05-02.md → cloud agent emitted
TL;DR ("Opus burned $1239 across 117 runs with 100% unknown outcomes")
+ 5 findings + 3 rule-candidates → delivered to chat_id 86059912 as
msg_id 1129 (HTTP 200). Cost: 3955 in + 897 out tokens on Sonnet
≈ $0.025/run. At 1 run/night that is ~$0.75/month for full reasoning
on every nightly report.
What this does NOT yet do:
- No retry on Telegram rate-limit (429). Single nightly call
is well below the 30/sec limit, but if the system ever bursts
multiple reports it would lose them.
- No multi-day digest mode (each run is independent; future:
weekly Sunday recap aggregating 7 reports).
- Cloud agent prompt is hard-coded inline; future: extract to
a path-atom-style block (post-2026-05-02 substrate work).
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN (pure shell)
behaviour-verified: yes
follow-up-required:
- Phase B prompt template extracted to atom (low priority)
- Weekly recap mode (Sunday)
- 429 rate-limit retry (defensive)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pushback: "что теперь делает сон? все связано?" — Sleep Phase B
was reading only `traces/`, ignoring the four tracking journals shipped
in the previous commit. Cloud agent had a partial view of what happened.
This commit closes the loop. Sleep now sees everything that's tracked.
PUSH SIDE — `kei-sleep-sync.sh` (called on every Stop event)
Now mirrors the full observability surface into the memory-repo:
~/.claude/memory/time-metrics/sessions.jsonl → time-metrics/
~/.claude/memory/time-metrics/tasks.jsonl → time-metrics/
~/.claude/memory/time-metrics/numeric-claims.jsonl → time-metrics/
~/.claude/memory/time-metrics/agent-toolstats.jsonl→ time-metrics/
~/.claude/agents/ledger.sqlite agents table → ledger/agents.jsonl
~/.claude/agents/ledger.sqlite skill_invocations → ledger/skill_invocations.jsonl
Format: JSONL (one row per object). The two ledger tables are dumped
via `sqlite3 + json_object()` so cloud agents can stream-parse into
pandas / duckdb without binary-file handling.
First sync moved 6 files / 638 rows from local to remote — verified
by `git show --stat` of the resulting `memory: session traces` commit.
CONSUME SIDE — `phase-b-rem.sh` REM-consolidation report
Each nightly `reports/sleep-YYYY-MM-DD.md` now ends with a "Tracking
observability (last 7 days)" section containing four jq-aggregated
digests:
1. Agent outcomes — per-model: n, functional/partial/scaffolding/fail
counts + total_cost_usd. Lets the agent see whether the model-tier
refactor (50c9e76) actually paid off and whether Sonnet success
rate justifies routing more task classes to it.
2. Skill success rates — per-skill: n, successes, rate_pct. Drives
Phase D nightly decisions (archive unused / re-extract failing /
mark validated). Empty until Skill tool is invoked in the next
session.
3. Numeric-claims tier breakdown — REAL / FROM-JOURNAL / ESTIMATE-HTC
counts. High ESTIMATE-HTC ratio = orchestrator under-calibrated.
Cloud agent's job: spot frequent ESTIMATE-HTC categories and
propose conversion to FROM-JOURNAL via measured runs.
4. Agent tool-call patterns — mean tool_use_count, mean duration_ms,
per-tool total calls. Lets the agent see "this code-implementer
spawn made 30 Read but 1 Edit — was tier-allocation correct?".
All four sections gracefully skip if the source JSONL is missing or
empty. jq is the only new dependency (already present per existing
phase-b checks).
What is NOT yet automated:
- The cloud agent's prompt template doesn't yet INSTRUCT it to act
on these digests. Currently the digest is data; whether the agent
proposes rule + hook codification based on it depends on the
free-text instructions in the schedule. Follow-up: codify a Phase B
instruction block that maps each digest to a recommendation pattern.
- Idempotency on `cp` for time-metrics: I use plain `cp` (not `cp -n`)
so the latest local state always overwrites remote. The journals are
append-only on the local side, so this is safe — but if two machines
ever share one memory-repo it would corrupt. Out of scope for
single-machine setup.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN (pure shell)
behaviour-verified: yes
follow-up-required:
- Phase B prompt template — instruct cloud agent to act on the four
digests (codify recurring patterns, calibrate ESTIMATE-HTC).
- skill_invocations.jsonl will populate from next session onward.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the loop on "without full tracking the system can't make decisions"
(user pushback on partial coverage). Three gaps that left the inference
layer blind are now wired:
GAP #1 — agent toolStats / token counts / cache hits captured
================================================================
`agent-outcome-backfill.sh` now appends one JSONL row per spawn to
`~/.claude/memory/time-metrics/agent-toolstats.jsonl` with:
agent_id, outcome, stubs, ts,
tool_use_count, duration_ms, tool_stats {Read:N, Bash:M, ...},
tokens_in, tokens_out, cache_read, cache_write
Sidecar journal (no schema migration). Production payload's
.tool_response.totalToolUseCount / totalDurationMs / toolStats / usage
fields land directly. Smoke-tested with synthetic spawn — row written.
GAP #2 — skill_invocations table actually receives writes
================================================================
The `skill_invocations` table (schema v8) had 0 rows because no caller
existed for `skill_metrics::record_invocation`. Added two pieces:
(a) `kei-ledger record-skill <name> --success {0|1}` CLI subcommand
Mirrors record-cost; same dispatch shape. Optional `--agent-id`,
`--trajectory-id`, `--duration-ms`, `--db`. Validates non-empty
name + duration ≥ 0. Outputs `{"ok":true,"skill":"...","ts":N}`.
(b) `hooks/skill-record.sh` — PostToolUse:Skill hook. 50 LOC POSIX.
Detects Skill tool calls, derives success heuristic from
tool_response (exit_code / status / content non-empty), shells
out to `kei-ledger record-skill`. Bypass via SKILL_RECORD_BYPASS=1.
83 kei-ledger tests pass (16 unit + 67 integration). Smoke-tested
end-to-end: `kei-ledger record-skill test-skill --success 1` inserts
a row with correct fields.
Phase D nightly skill-metrics decisions (archive if unused N days,
re-extract if success<60% over M days, validated if >20 calls + >90%
success) now have data to consume.
GAP #3 — numeric-claims.jsonl receives every evidence-tagged claim
================================================================
RULE 0.18 mandated three markers `[REAL:]` / `[FROM-JOURNAL:]` /
`[ESTIMATE-HTC:]` on every numeric/duration/cost claim, but no hook
appended valid claims to the journal — the calibration data RULE 0.18
promised never accumulated.
`hooks/numeric-claims-record.sh` — Stop hook, 140 LOC POSIX. Reads
transcript_path from stdin, locates the last assistant message via
recursive flatten (same pattern as agent-outcome-backfill.sh after
the production-payload-shape fix), regex-extracts every `<phrase>
[<TIER>: <pointer>]` triple, appends one JSONL row per claim.
Idempotent within 1-second window to avoid double-recording on
repeat Stop fires. Bypass via NUMERIC_CLAIMS_RECORD_BYPASS=1.
Smoke test: synthetic transcript with 3 markers (REAL + ESTIMATE-HTC
+ FROM-JOURNAL) produced exactly 3 well-formed JSONL rows.
Settings.json
================================================================
- PostToolUse:Skill matcher created (or augmented if already
present) with skill-record.sh.
- Stop:* matcher gains numeric-claims-record.sh after the existing
chain (stop-verify, task-timer, session-end-dump, extract-task-
durations, chat-numeric-postflag, affect-threshold-check,
enrich-from-jsonl).
What this does NOT do (deferred):
- Backfill `skill_invocations` from past traces (history started
today; Phase D cohort builds forward from now).
- Migrate the agent toolStats sidecar JSONL into a proper ledger
column. Append-only file is fine for the current scale.
- Refactor main.rs (now 233 LOC, was 212; pre-existing CP debt
flagged by skill-record agent — separate cleanup PR).
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
- kei-ledger main.rs Constructor Pattern split (212→233 LOC)
- Verify in next session: skill_invocations gets rows from real
Skill tool use; numeric-claims.jsonl gets rows from real assistant
messages with markers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hook never fired in production despite passing unit tests. Diagnosed
via debug-log + payload dump: real Claude Code PostToolUse:Agent sends
`tool_response` as an OBJECT (not string, not array), with the agent's
reply at `tool_response.content[0].text` — keys: agentId / agentType /
content / prompt / status / toolStats / totalDurationMs / totalTokens
/ totalToolUseCount / usage.
Original jq filter handled string + object (`$r.content // $r.text`)
but `$r.content` returns the array verbatim; `jq -r` then dumps the
JSON literal which has `\n` as escape sequences, defeating the
`grep -m1 '^shipped:'` line-anchor.
Fix: recursive `flatten` jq function:
string → as-is
array of any → recurse, join "\n"
object with .text → return .text
object with .content → recurse into content
anything else → ""
Verified end-to-end: latest 4 code-implementer spawns now write
outcome=functional to ledger correctly. Beta posterior in
kei-model-router begins receiving signal.
Production cleanup:
- Removed verbose debug-log + payload-dump diagnostic. Toggle via
`AGENT_OUTCOME_DEBUG=1` env if hook stops firing in some future
Claude Code version.
- Hook source committed to `hooks/agent-outcome-backfill.sh` so
`install.sh` deploys it on fresh installs (was only in user-home
previously — gap from `feat/substrate-path-atoms` agent run).
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN
behaviour-verified: yes
follow-up-required:
- none
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Parfii-bot
