Closes engine bugs #1, #2, #3 from the user's backlog.md entry dated 2026-05-11 "kei-refactor-engine — 4 false-positive bugs". Bug #4 was fixed in6cd99982(wikilink path-norm + handoff scanner removal). ## Bug #1 — vendored marketplaces skip Engine was scanning `plugins/marketplaces/claude-plugins-official/` — vendored upstream code where Constructor Pattern thresholds don't apply. ~246 cp-violations were from this tree. Fix: `tree::should_skip_path()` central filter. Skips any path component named `marketplaces`, `target`, `node_modules`, or `.git`. Applied via `WalkDir::filter_entry()` in `collect_markdown`, `collect_with_ext`, `scanners::cp::scan`, `scanners::orphans::scan`, `scanners::orphans::all_basenames`. `scanners::cp::skip_dir` now delegates to `should_skip_path` (removed the older inline `/target/`-substring check). ## Bug #2 — hooks-share-matcher false-positive class Claude Code hook chains are designed to support N hooks per event by design. `scanners::hooks` was flagging every pair sharing a matcher as a "redundancy conflict" — 9 hooks/medium findings in the last deep-sleep run, every one false-positive. Fix: `scanners::hooks::scan` reduced to a no-op stub returning `Vec::new()`. Module docstring documents the retraction + future direction (a real `hooks-validity` scanner for broken shebangs, missing chmod, syntax errors would replace it). ## Bug #3 — `.patch` file not unified diff Already resolved in prior commit (v0.14.1 retraction in patch.rs): CLI default is `plan-autoresolve.md`, Phase C template references `-autoresolve.md` suffix, `write_patch` is deprecated shim. Only legacy `.patch` artefacts in sync-repo/reports/ remain — those are audit trail, not active. ## Phase C draft file emission (deep-sleep-trigger-prompt.md §6.d) The earlier Phase C template emitted `proposed_rule` markdown blocks only — no actionable artefacts. Extended §6 with step 6.d: when WITH_FORK=1 AND fork branch was created, ALSO write skeleton draft files into the branch: sync-repo/sleep-deep/YYYY-MM-DD/drafts/rules/<slug>.md sync-repo/sleep-deep/YYYY-MM-DD/drafts/hooks/<slug>.sh Drafts follow pattern-codifier-agent Phase 3 templates. Phase C does NOT register hooks — that's pattern-codifier's job via /sleep-review morning click-flow (skill Phase 3a added in ~/.claude commit 49a320d). This closes the loop: Phase C surfaces draft → morning review clicks approve → pattern-codifier installs → settings.json registered. Smoke-test required in §6.d: every emitted `.sh` MUST `bash -n` clean or be excluded from commit + listed in plan markdown. ## Results on ~/.claude/memory/sync-repo (live data) | Scanner | Before | After | Delta | |-----------|-------:|------:|------:| | orphans | 108 | 1 | -107 | | hooks | 2 | 0 | -2 | | cp | 174 | 0 | -174 | | **TOTAL** | 284 | 1 | -283 | On full ~/.claude scan: total drops from ~1614 (per 2026-05-11 backlog) to 983 (cp=186 + orphans=797 — orphan count high because ~/.claude tree has many memory/chatlogs/ refs out-of-tree). ## Tests 12/12 pass on kei-conflict-scan workspace (4 unit + 8 integration). Pre-existing `oversize_file_flagged` + `orphan_wikilinks_flagged` still green; new `cross_repo_wikilink_not_flagged` + `path_prefixed_wikilink_matches_basename` from6cd99982still green. Private mirror at ~/Projects/KeiSeiKit/_primitives/_rust/ synced (4 files: tree.rs, scanners/cp.rs, scanners/orphans.rs, scanners/hooks.rs). Closes backlog "engine-noise-2026-05-11" tag bugs #1, #2, #3.
175 lines
7.9 KiB
Markdown
175 lines
7.9 KiB
Markdown
# Deep-sleep NREM consolidation (KeiSeiKit v0.13.0)
|
||
|
||
<!--
|
||
Phase C of the three-phase nightly cycle. Runs OPTIONALLY, by default
|
||
every 7 days from install. Biological analog: NREM slow-wave-sleep
|
||
system consolidation — detect conflicts across rules / hooks / blocks /
|
||
memory, propose a clean refactor, optionally generate a ready-to-merge
|
||
fork branch for user review.
|
||
|
||
Placeholders:
|
||
{REPO_URL} — memory-repo SSH URL (git@host:org/repo.git)
|
||
{DEEP_SLEEP_CRON_DAYS} — integer; frequency in days (default: 7)
|
||
{WITH_FORK} — 0 or 1; if 1, engine also applies to a fork
|
||
branch when safe (zero-conflict guarantee)
|
||
-->
|
||
|
||
Clone: {REPO_URL}
|
||
Cadence: every {DEEP_SLEEP_CRON_DAYS} day(s), counted from first install
|
||
Fork-output mode: WITH_FORK={WITH_FORK} (1 = plan + fork; 0 = plan only)
|
||
|
||
## Cycle order with Phase C
|
||
|
||
Phase A (incubation, v0.12.0) → Phase B (REM consolidation, v0.12.0)
|
||
→ Phase C (deep-sleep NREM, THIS document, v0.13.0)
|
||
|
||
Phase C runs AFTER Phase B, and ONLY when today is a multiple of
|
||
`DEEP_SLEEP_CRON_DAYS` from the install date (file
|
||
`sync-repo/reports/install-anchor.txt`). If the file is missing on a
|
||
first run, Phase C silently no-ops and writes the anchor for next time.
|
||
|
||
If Phase A selected a `marathon: true` task, Phase B is skipped per
|
||
v0.12.0 rules AND Phase C is skipped too — the marathon owns the night.
|
||
|
||
## Phase C — Task
|
||
|
||
1. **Scan.** Run `kei-conflict-scan --path sync-repo/ --format=json
|
||
--exit-on-hit` and capture the JSON. Categories: rules, hooks,
|
||
blocks, orphans, cp.
|
||
|
||
2. **Plan.** Pipe the JSON into
|
||
`kei-refactor-engine --input - --plan-out
|
||
sync-repo/sleep-deep/YYYY-MM-DD-plan.md`.
|
||
The plan markdown always lists:
|
||
- Auto-apply candidates (safe; engine-proposed)
|
||
- "Requires human decision" items (zero-conflict guarantee: NEVER
|
||
included in the generated patch)
|
||
|
||
3. **Optional fork (only if `WITH_FORK=1`):**
|
||
|
||
a. `kei-refactor-engine --input - --apply-to-branch
|
||
deep-sleep/YYYY-MM-DD --patch-out sync-repo/sleep-deep/YYYY-MM-DD-autoresolve.md`
|
||
(re-run on same JSON; the auto-resolve markdown lists auto-apply
|
||
items only — NOT a unified diff, see note below).
|
||
|
||
b. Review the auto-resolve markdown and apply each change manually
|
||
in a new local branch:
|
||
`git checkout -b deep-sleep/YYYY-MM-DD`
|
||
Open `<autoresolve>.md`, edit the listed files accordingly, then
|
||
`git add <files> && git commit`.
|
||
|
||
c. Gate: `kei-graph-check --path sync-repo/`. If broken refs after
|
||
your edits → delete branch, append "graph check failed — fork
|
||
aborted, plan kept" note to the plan file.
|
||
|
||
d. If clean → push the fork branch for morning review.
|
||
|
||
> NOTE (v0.14.1 retraction): earlier docs claimed the engine emits a
|
||
> `git apply`-ready patch. It does not — see `patch.rs` header for
|
||
> the reason (engine cannot synthesise file-content hunks without
|
||
> reading source files, which risks RULE 0.4 hallucination). The
|
||
> companion file is a markdown summary reviewed and applied by hand.
|
||
|
||
4. **Commit + push.** The plan markdown is always committed to `main`
|
||
with message `NREM: deep-sleep YYYY-MM-DD`. If a fork branch was
|
||
produced, it is pushed as a separate ref for the user's morning
|
||
review. The user merges (or rejects) the fork manually.
|
||
|
||
5. **Model-router calibration (v0.14.0+).** Run
|
||
`kei-model-router calibrate` once Phase C reaches step 5. The CLI
|
||
reads the local ledger at `~/.claude/agents/ledger.sqlite` (or
|
||
`$KEI_LEDGER_DB`), performs leave-one-out grid search over kernel
|
||
weights, and prints baseline-vs-best MSE plus the calibrated
|
||
`(α_role, α_caps, α_scope, α_body)` tuple. Append the JSON line to
|
||
`~/.claude/memory/router-calibration.jsonl`:
|
||
|
||
{"ts":"YYYY-MM-DDThh:mm:ssZ","rows":N,"baseline_mse":B,
|
||
"best_mse":M,"weights":{...}}
|
||
|
||
If `rows < 5`, write `{"ts":"...","rows":N,"skipped":true}` and
|
||
continue — too few outcomes to recalibrate. The router falls back
|
||
to the last persisted weights or compile-time defaults.
|
||
|
||
6. **Affect-matrix aggregation (v0.14.1+).** Aggregate the per-session
|
||
affect files at `~/.claude/memory/affect/*.jsonl` (written live by
|
||
`affect-live-scan.sh` UserPromptSubmit hook) into a Phase C summary.
|
||
Steps:
|
||
|
||
a. Concatenate all affect files dated within the last
|
||
`DEEP_SLEEP_CRON_DAYS` interval.
|
||
b. Group by `category` and count. For any category whose count
|
||
exceeds its threshold (`repeat-signal ≥ 3`,
|
||
`frustration-tone ≥ 5`, `conservative-framing ≥ 4`,
|
||
`data-contamination ≥ 2`) AND whose top-3 quotes share a
|
||
semantic theme (manual review by the deep-sleep agent), emit a
|
||
`proposed_rule` block in the plan markdown listing:
|
||
- the recurring pattern (1 sentence)
|
||
- 2-3 example user quotes (with timestamps)
|
||
- suggested severity tier (remind / warn / enforce / block)
|
||
- suggested hook event (UserPromptSubmit / PreToolUse / Stop)
|
||
c. Append a one-line audit row to
|
||
`sync-repo/reports/affect-trends.jsonl`:
|
||
|
||
{"date":"YYYY-MM-DD","window_days":N,
|
||
"counts":{"repeat-signal":R,"frustration-tone":F,
|
||
"conservative-framing":C,"data-contamination":D},
|
||
"proposed_rules":N_proposed}
|
||
|
||
This is the LEARNING LOOP — Phase C surfaces affect patterns to the
|
||
user via the morning plan, the user clicks `/escalate-recurrence`
|
||
to codify them. Without step 6, the affect matrix is a passive log;
|
||
with it, the matrix feeds back into rule formation per RULE 0.10.
|
||
|
||
d. **(2026-05-12 extension) Emit DRAFT FILES, not just markdown.**
|
||
For each `proposed_rule` block in §6b, also write skeleton files
|
||
into the deep-sleep fork branch (only if `WITH_FORK=1` and the
|
||
branch was successfully created in §3):
|
||
|
||
sync-repo/sleep-deep/YYYY-MM-DD/drafts/rules/<slug>.md
|
||
sync-repo/sleep-deep/YYYY-MM-DD/drafts/hooks/<slug>.sh
|
||
|
||
Where `<slug>` = kebab-cased pattern name (e.g.
|
||
`response-conservatism-check`). File contents follow the
|
||
`pattern-codifier-agent` Phase 3 template — frontmatter + Why
|
||
+ Rule + Severity ladder + Bypass for `.md`; bash skeleton with
|
||
stdin JSON parsing + severity exit code + bypass env for `.sh`.
|
||
|
||
DO NOT register the hook in `settings.json` here. The morning
|
||
`/sleep-review` skill Phase 3a presents each draft pair to the
|
||
user via `AskUserQuestion`; user approval triggers
|
||
`pattern-codifier-agent` which performs the actual install +
|
||
registration. Phase C's job is just to provide click-ready
|
||
drafts so the morning review is a 30-second click flow, not a
|
||
30-minute drafting flow.
|
||
|
||
Smoke-test: every emitted draft `.sh` MUST `bash -n <file>`
|
||
cleanly (syntax check, no execution). Any draft failing this
|
||
check is removed before commit + listed in the plan markdown
|
||
under "draft generation failures" so user knows the proposed
|
||
rule needs manual drafting.
|
||
|
||
## Zero-conflict guarantee
|
||
|
||
Any conflict the refactor-engine marks `requires_human_decision` is
|
||
EXCLUDED from the generated patch and listed plainly in the plan
|
||
under the matching section. The user sees every such item explicitly.
|
||
No silent auto-apply of ambiguous changes.
|
||
|
||
## Invariants
|
||
|
||
- Plan is ALWAYS written, even if the engine finds nothing (body reads
|
||
"no conflicts this cycle").
|
||
- Fork branch never auto-merges to main.
|
||
- Phase C never touches `traces/*.jsonl` (append-only, inherited).
|
||
- Store backend is whatever `kei-store status` reports — the Phase C
|
||
pipeline is store-agnostic.
|
||
|
||
## Failure handling
|
||
|
||
- `kei-conflict-scan` fails → record the error in the plan body and
|
||
skip fork.
|
||
- `kei-refactor-engine` fails → same; keep any partial plan markdown.
|
||
- Manual edits in step 3b produce merge conflicts → delete fork branch;
|
||
append the conflict summary to the plan.
|
||
- Push fails → retry once; on second failure leave local commit and
|
||
exit 1. Local state is recoverable on next run.
|