User pushback: "что теперь делает сон? все связано?" — Sleep Phase B
was reading only `traces/`, ignoring the four tracking journals shipped
in the previous commit. Cloud agent had a partial view of what happened.
This commit closes the loop. Sleep now sees everything that's tracked.
PUSH SIDE — `kei-sleep-sync.sh` (called on every Stop event)
Now mirrors the full observability surface into the memory-repo:
~/.claude/memory/time-metrics/sessions.jsonl → time-metrics/
~/.claude/memory/time-metrics/tasks.jsonl → time-metrics/
~/.claude/memory/time-metrics/numeric-claims.jsonl → time-metrics/
~/.claude/memory/time-metrics/agent-toolstats.jsonl→ time-metrics/
~/.claude/agents/ledger.sqlite agents table → ledger/agents.jsonl
~/.claude/agents/ledger.sqlite skill_invocations → ledger/skill_invocations.jsonl
Format: JSONL (one row per object). The two ledger tables are dumped
via `sqlite3 + json_object()` so cloud agents can stream-parse into
pandas / duckdb without binary-file handling.
First sync moved 6 files / 638 rows from local to remote — verified
by `git show --stat` of the resulting `memory: session traces` commit.
CONSUME SIDE — `phase-b-rem.sh` REM-consolidation report
Each nightly `reports/sleep-YYYY-MM-DD.md` now ends with a "Tracking
observability (last 7 days)" section containing four jq-aggregated
digests:
1. Agent outcomes — per-model: n, functional/partial/scaffolding/fail
counts + total_cost_usd. Lets the agent see whether the model-tier
refactor (cb1fdde) actually paid off and whether Sonnet success
rate justifies routing more task classes to it.
2. Skill success rates — per-skill: n, successes, rate_pct. Drives
Phase D nightly decisions (archive unused / re-extract failing /
mark validated). Empty until Skill tool is invoked in the next
session.
3. Numeric-claims tier breakdown — REAL / FROM-JOURNAL / ESTIMATE-HTC
counts. High ESTIMATE-HTC ratio = orchestrator under-calibrated.
Cloud agent's job: spot frequent ESTIMATE-HTC categories and
propose conversion to FROM-JOURNAL via measured runs.
4. Agent tool-call patterns — mean tool_use_count, mean duration_ms,
per-tool total calls. Lets the agent see "this code-implementer
spawn made 30 Read but 1 Edit — was tier-allocation correct?".
All four sections gracefully skip if the source JSONL is missing or
empty. jq is the only new dependency (already present per existing
phase-b checks).
What is NOT yet automated:
- The cloud agent's prompt template doesn't yet INSTRUCT it to act
on these digests. Currently the digest is data; whether the agent
proposes rule + hook codification based on it depends on the
free-text instructions in the schedule. Follow-up: codify a Phase B
instruction block that maps each digest to a recommendation pattern.
- Idempotency on `cp` for time-metrics: I use plain `cp` (not `cp -n`)
so the latest local state always overwrites remote. The journals are
append-only on the local side, so this is safe — but if two machines
ever share one memory-repo it would corrupt. Out of scope for
single-machine setup.
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN (pure shell)
behaviour-verified: yes
follow-up-required:
- Phase B prompt template — instruct cloud agent to act on the four
digests (codify recurring patterns, calibrate ESTIMATE-HTC).
- skill_invocations.jsonl will populate from next session onward.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
125 lines
5 KiB
Bash
Executable file
125 lines
5 KiB
Bash
Executable file
#!/bin/sh
|
|
# kei-sleep-sync.sh — POSIX-sh helper called at session end.
|
|
#
|
|
# Stages any new session traces + backlog in the user's memory-repo and
|
|
# pushes via a dedicated deploy key. NEVER blocks the session: every
|
|
# failure path logs to ~/.claude/memory/sync-errors.log and exits 0.
|
|
#
|
|
# Config resolution order:
|
|
# 1. env var KEI_MEMORY_REPO_PATH / KEI_MEMORY_SSH_KEY
|
|
# 2. ~/.claude/secrets/.env (sourced if present)
|
|
# 3. sync-repo's .keisei-sync.toml (informational only)
|
|
#
|
|
# Emergency bypass: `KEI_SLEEP_SYNC_BYPASS=1 ...` — silent exit 0.
|
|
|
|
set -u
|
|
|
|
ERR_LOG="${HOME}/.claude/memory/sync-errors.log"
|
|
|
|
log_err() {
|
|
mkdir -p "$(dirname "$ERR_LOG")" 2>/dev/null || return 0
|
|
printf '[%s] %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*" >> "$ERR_LOG" 2>/dev/null || true
|
|
}
|
|
|
|
# ---- bypass + env -----------------------------------------------------------
|
|
|
|
[ "${KEI_SLEEP_SYNC_BYPASS:-0}" = "1" ] && exit 0
|
|
|
|
SECRETS_FILE="${HOME}/.claude/secrets/.env"
|
|
if [ -f "$SECRETS_FILE" ] && [ -z "${KEI_MEMORY_REPO_PATH:-}" ]; then
|
|
# shellcheck disable=SC1090
|
|
. "$SECRETS_FILE" 2>/dev/null || true
|
|
fi
|
|
|
|
REPO_PATH="${KEI_MEMORY_REPO_PATH:-}"
|
|
SSH_KEY="${KEI_MEMORY_SSH_KEY:-}"
|
|
|
|
# Silent no-op when sync isn't configured yet (most users).
|
|
[ -z "$REPO_PATH" ] && exit 0
|
|
[ -d "${REPO_PATH}/.git" ] || exit 0
|
|
|
|
# ---- stage, commit, push ---------------------------------------------------
|
|
|
|
# cd may fail (permissions / path vanished) — silent exit.
|
|
cd "$REPO_PATH" 2>/dev/null || exit 0
|
|
|
|
# Mirror traces from the canonical local dump dir into the repo.
|
|
TRACES_SRC="${HOME}/.claude/memory/traces"
|
|
if [ -d "$TRACES_SRC" ]; then
|
|
mkdir -p traces 2>/dev/null || true
|
|
# -n = never overwrite; append-only semantics.
|
|
cp -n "$TRACES_SRC"/*.jsonl traces/ 2>/dev/null || true
|
|
fi
|
|
|
|
# Mirror time-metrics journals (RULE 0.18 + post-2026-05-02 tracking).
|
|
# Append-only JSONL, OK to overwrite remote with local since local is the
|
|
# source-of-truth for this user's machine. Source files:
|
|
# sessions.jsonl — RULE 0.18 session-duration journal
|
|
# tasks.jsonl — task-timer.sh per-Agent durations
|
|
# numeric-claims.jsonl — RULE 0.18 evidence-tagged claims
|
|
# agent-toolstats.jsonl — agent-outcome-backfill.sh sidecar
|
|
TIME_METRICS_SRC="${HOME}/.claude/memory/time-metrics"
|
|
if [ -d "$TIME_METRICS_SRC" ]; then
|
|
mkdir -p time-metrics 2>/dev/null || true
|
|
for f in sessions.jsonl tasks.jsonl numeric-claims.jsonl agent-toolstats.jsonl; do
|
|
if [ -f "$TIME_METRICS_SRC/$f" ]; then
|
|
cp "$TIME_METRICS_SRC/$f" "time-metrics/$f" 2>/dev/null || true
|
|
fi
|
|
done
|
|
fi
|
|
|
|
# Snapshot kei-ledger: agents + skill_invocations as JSONL (sqlite3 .dump
|
|
# has too much noise + is binary-ordering-sensitive). Cloud agents can
|
|
# stream-parse JSONL straight into pandas/duckdb for analysis.
|
|
LEDGER_DB="${KEI_LEDGER_DB:-${HOME}/.claude/agents/ledger.sqlite}"
|
|
if [ -f "$LEDGER_DB" ] && command -v sqlite3 >/dev/null 2>&1; then
|
|
mkdir -p ledger 2>/dev/null || true
|
|
# `-newline` mode + `-cmd .mode json` would be cleaner but isn't
|
|
# universally available; emit one-row-per-line JSON via select+json_object.
|
|
sqlite3 "$LEDGER_DB" \
|
|
"SELECT json_object(
|
|
'id', id, 'branch', branch, 'parent_branch', parent_branch,
|
|
'spec_sha', spec_sha, 'status', status,
|
|
'started_ts', started_ts, 'finished_ts', finished_ts,
|
|
'summary', summary, 'worktree_path', worktree_path,
|
|
'dna', dna, 'creator_id', creator_id, 'fork_parent_id', fork_parent_id,
|
|
'cost_micro_cents', cost_micro_cents, 'provider', provider,
|
|
'model', model, 'tokens_in', tokens_in, 'tokens_out', tokens_out,
|
|
'stubs_count', stubs_count, 'outcome', outcome,
|
|
'escalation_depth', escalation_depth, 'task_class_dna', task_class_dna
|
|
) FROM agents ORDER BY started_ts" \
|
|
> ledger/agents.jsonl 2>/dev/null || true
|
|
sqlite3 "$LEDGER_DB" \
|
|
"SELECT json_object(
|
|
'id', id, 'skill_name', skill_name, 'ts', ts,
|
|
'agent_id', agent_id, 'success', success,
|
|
'trajectory_id', trajectory_id, 'duration_ms', duration_ms
|
|
) FROM skill_invocations ORDER BY ts" \
|
|
> ledger/skill_invocations.jsonl 2>/dev/null || true
|
|
fi
|
|
|
|
git add traces/ backlog.md time-metrics/ ledger/ 2>/dev/null \
|
|
|| { log_err "git add failed"; exit 0; }
|
|
|
|
# Nothing staged — silent exit.
|
|
if git diff --cached --quiet 2>/dev/null; then
|
|
exit 0
|
|
fi
|
|
|
|
COMMIT_MSG="memory: session traces $(date +%Y-%m-%dT%H:%M:%S%z)"
|
|
if ! git commit -q -m "$COMMIT_MSG" 2>/dev/null; then
|
|
log_err "git commit failed"
|
|
exit 0
|
|
fi
|
|
|
|
# Push via the dedicated deploy key so we don't clobber the user's default SSH.
|
|
if [ -n "$SSH_KEY" ] && [ -f "$SSH_KEY" ]; then
|
|
GIT_SSH_COMMAND="ssh -i $SSH_KEY -o StrictHostKeyChecking=accept-new" \
|
|
git push -q origin HEAD 2>/dev/null \
|
|
|| { log_err "git push failed via $SSH_KEY"; exit 0; }
|
|
else
|
|
git push -q origin HEAD 2>/dev/null \
|
|
|| { log_err "git push failed (no SSH_KEY set)"; exit 0; }
|
|
fi
|
|
|
|
exit 0
|