KeiSeiKit-1.0/_primitives/kei-sleep-sync.sh
Parfii-bot 9a3db14b90 feat(sleep-sync): mirror time-metrics + ledger snapshots, surface in Phase B report
User pushback: "что теперь делает сон? все связано?" — Sleep Phase B
was reading only `traces/`, ignoring the four tracking journals shipped
in the previous commit. Cloud agent had a partial view of what happened.

This commit closes the loop. Sleep now sees everything that's tracked.

PUSH SIDE — `kei-sleep-sync.sh` (called on every Stop event)

Now mirrors the full observability surface into the memory-repo:

  ~/.claude/memory/time-metrics/sessions.jsonl       → time-metrics/
  ~/.claude/memory/time-metrics/tasks.jsonl          → time-metrics/
  ~/.claude/memory/time-metrics/numeric-claims.jsonl → time-metrics/
  ~/.claude/memory/time-metrics/agent-toolstats.jsonl→ time-metrics/
  ~/.claude/agents/ledger.sqlite agents table        → ledger/agents.jsonl
  ~/.claude/agents/ledger.sqlite skill_invocations   → ledger/skill_invocations.jsonl

Format: JSONL (one row per object). The two ledger tables are dumped
via `sqlite3 + json_object()` so cloud agents can stream-parse into
pandas / duckdb without binary-file handling.

First sync moved 6 files / 638 rows from local to remote — verified
by `git show --stat` of the resulting `memory: session traces` commit.

CONSUME SIDE — `phase-b-rem.sh` REM-consolidation report

Each nightly `reports/sleep-YYYY-MM-DD.md` now ends with a "Tracking
observability (last 7 days)" section containing four jq-aggregated
digests:

  1. Agent outcomes — per-model: n, functional/partial/scaffolding/fail
     counts + total_cost_usd. Lets the agent see whether the model-tier
     refactor (cb1fdde) actually paid off and whether Sonnet success
     rate justifies routing more task classes to it.

  2. Skill success rates — per-skill: n, successes, rate_pct. Drives
     Phase D nightly decisions (archive unused / re-extract failing /
     mark validated). Empty until Skill tool is invoked in the next
     session.

  3. Numeric-claims tier breakdown — REAL / FROM-JOURNAL / ESTIMATE-HTC
     counts. High ESTIMATE-HTC ratio = orchestrator under-calibrated.
     Cloud agent's job: spot frequent ESTIMATE-HTC categories and
     propose conversion to FROM-JOURNAL via measured runs.

  4. Agent tool-call patterns — mean tool_use_count, mean duration_ms,
     per-tool total calls. Lets the agent see "this code-implementer
     spawn made 30 Read but 1 Edit — was tier-allocation correct?".

All four sections gracefully skip if the source JSONL is missing or
empty. jq is the only new dependency (already present per existing
phase-b checks).

What is NOT yet automated:

  - The cloud agent's prompt template doesn't yet INSTRUCT it to act
    on these digests. Currently the digest is data; whether the agent
    proposes rule + hook codification based on it depends on the
    free-text instructions in the schedule. Follow-up: codify a Phase B
    instruction block that maps each digest to a recommendation pattern.

  - Idempotency on `cp` for time-metrics: I use plain `cp` (not `cp -n`)
    so the latest local state always overwrites remote. The journals are
    append-only on the local side, so this is safe — but if two machines
    ever share one memory-repo it would corrupt. Out of scope for
    single-machine setup.

=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: NOT-RUN  (pure shell)
behaviour-verified: yes
follow-up-required:
  - Phase B prompt template — instruct cloud agent to act on the four
    digests (codify recurring patterns, calibrate ESTIMATE-HTC).
  - skill_invocations.jsonl will populate from next session onward.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 04:02:28 +08:00

125 lines
5 KiB
Bash
Executable file

#!/bin/sh
# kei-sleep-sync.sh — POSIX-sh helper called at session end.
#
# Stages any new session traces + backlog in the user's memory-repo and
# pushes via a dedicated deploy key. NEVER blocks the session: every
# failure path logs to ~/.claude/memory/sync-errors.log and exits 0.
#
# Config resolution order:
# 1. env var KEI_MEMORY_REPO_PATH / KEI_MEMORY_SSH_KEY
# 2. ~/.claude/secrets/.env (sourced if present)
# 3. sync-repo's .keisei-sync.toml (informational only)
#
# Emergency bypass: `KEI_SLEEP_SYNC_BYPASS=1 ...` — silent exit 0.
set -u
ERR_LOG="${HOME}/.claude/memory/sync-errors.log"
log_err() {
mkdir -p "$(dirname "$ERR_LOG")" 2>/dev/null || return 0
printf '[%s] %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*" >> "$ERR_LOG" 2>/dev/null || true
}
# ---- bypass + env -----------------------------------------------------------
[ "${KEI_SLEEP_SYNC_BYPASS:-0}" = "1" ] && exit 0
SECRETS_FILE="${HOME}/.claude/secrets/.env"
if [ -f "$SECRETS_FILE" ] && [ -z "${KEI_MEMORY_REPO_PATH:-}" ]; then
# shellcheck disable=SC1090
. "$SECRETS_FILE" 2>/dev/null || true
fi
REPO_PATH="${KEI_MEMORY_REPO_PATH:-}"
SSH_KEY="${KEI_MEMORY_SSH_KEY:-}"
# Silent no-op when sync isn't configured yet (most users).
[ -z "$REPO_PATH" ] && exit 0
[ -d "${REPO_PATH}/.git" ] || exit 0
# ---- stage, commit, push ---------------------------------------------------
# cd may fail (permissions / path vanished) — silent exit.
cd "$REPO_PATH" 2>/dev/null || exit 0
# Mirror traces from the canonical local dump dir into the repo.
TRACES_SRC="${HOME}/.claude/memory/traces"
if [ -d "$TRACES_SRC" ]; then
mkdir -p traces 2>/dev/null || true
# -n = never overwrite; append-only semantics.
cp -n "$TRACES_SRC"/*.jsonl traces/ 2>/dev/null || true
fi
# Mirror time-metrics journals (RULE 0.18 + post-2026-05-02 tracking).
# Append-only JSONL, OK to overwrite remote with local since local is the
# source-of-truth for this user's machine. Source files:
# sessions.jsonl — RULE 0.18 session-duration journal
# tasks.jsonl — task-timer.sh per-Agent durations
# numeric-claims.jsonl — RULE 0.18 evidence-tagged claims
# agent-toolstats.jsonl — agent-outcome-backfill.sh sidecar
TIME_METRICS_SRC="${HOME}/.claude/memory/time-metrics"
if [ -d "$TIME_METRICS_SRC" ]; then
mkdir -p time-metrics 2>/dev/null || true
for f in sessions.jsonl tasks.jsonl numeric-claims.jsonl agent-toolstats.jsonl; do
if [ -f "$TIME_METRICS_SRC/$f" ]; then
cp "$TIME_METRICS_SRC/$f" "time-metrics/$f" 2>/dev/null || true
fi
done
fi
# Snapshot kei-ledger: agents + skill_invocations as JSONL (sqlite3 .dump
# has too much noise + is binary-ordering-sensitive). Cloud agents can
# stream-parse JSONL straight into pandas/duckdb for analysis.
LEDGER_DB="${KEI_LEDGER_DB:-${HOME}/.claude/agents/ledger.sqlite}"
if [ -f "$LEDGER_DB" ] && command -v sqlite3 >/dev/null 2>&1; then
mkdir -p ledger 2>/dev/null || true
# `-newline` mode + `-cmd .mode json` would be cleaner but isn't
# universally available; emit one-row-per-line JSON via select+json_object.
sqlite3 "$LEDGER_DB" \
"SELECT json_object(
'id', id, 'branch', branch, 'parent_branch', parent_branch,
'spec_sha', spec_sha, 'status', status,
'started_ts', started_ts, 'finished_ts', finished_ts,
'summary', summary, 'worktree_path', worktree_path,
'dna', dna, 'creator_id', creator_id, 'fork_parent_id', fork_parent_id,
'cost_micro_cents', cost_micro_cents, 'provider', provider,
'model', model, 'tokens_in', tokens_in, 'tokens_out', tokens_out,
'stubs_count', stubs_count, 'outcome', outcome,
'escalation_depth', escalation_depth, 'task_class_dna', task_class_dna
) FROM agents ORDER BY started_ts" \
> ledger/agents.jsonl 2>/dev/null || true
sqlite3 "$LEDGER_DB" \
"SELECT json_object(
'id', id, 'skill_name', skill_name, 'ts', ts,
'agent_id', agent_id, 'success', success,
'trajectory_id', trajectory_id, 'duration_ms', duration_ms
) FROM skill_invocations ORDER BY ts" \
> ledger/skill_invocations.jsonl 2>/dev/null || true
fi
git add traces/ backlog.md time-metrics/ ledger/ 2>/dev/null \
|| { log_err "git add failed"; exit 0; }
# Nothing staged — silent exit.
if git diff --cached --quiet 2>/dev/null; then
exit 0
fi
COMMIT_MSG="memory: session traces $(date +%Y-%m-%dT%H:%M:%S%z)"
if ! git commit -q -m "$COMMIT_MSG" 2>/dev/null; then
log_err "git commit failed"
exit 0
fi
# Push via the dedicated deploy key so we don't clobber the user's default SSH.
if [ -n "$SSH_KEY" ] && [ -f "$SSH_KEY" ]; then
GIT_SSH_COMMAND="ssh -i $SSH_KEY -o StrictHostKeyChecking=accept-new" \
git push -q origin HEAD 2>/dev/null \
|| { log_err "git push failed via $SSH_KEY"; exit 0; }
else
git push -q origin HEAD 2>/dev/null \
|| { log_err "git push failed (no SSH_KEY set)"; exit 0; }
fi
exit 0