feat(skills): /self-audit 5-phase triage pipeline

2026-04-22 00:50:04 +08:00 · 2026-04-22 00:50:04 +08:00 · 139d2f0325
commit 139d2f0325
parent 994c310fc4
6 changed files with 375 additions and 0 deletions
--- a/skills/self-audit/SKILL.md
+++ b/skills/self-audit/SKILL.md
@ -0,0 +1,96 @@
+---
+name: self-audit
+description: Session retrospective triage (RULE 0.14). Runs `kei-memory analyze + patterns`, classifies findings, presents them via click-only AskUserQuestion, routes each selected item to `/escalate-recurrence` (rule+hook), `/debug-deep` (bug RCA), or the audit-backlog (log-only). Self-audit is triage, not implementation.
+argument-hint: <optional session id; defaults to last session>
+---
+
+# Self-Audit — Session Retrospective Triage (index)
+
+You are running the RULE 0.14 self-audit on the last (or named) session.
+You convert the session's trace into a short list of findings, classify
+each, present them as a multi-select click batch, and route each selection
+to the appropriate existing skill. You NEVER write fixes yourself.
+
+This `SKILL.md` is the INDEX. Each phase lives in its own file and is
+executed in order. Never skip a phase. Never re-order phases.
+
+---
+
+## Pipeline overview (5 phases)
+
+| Phase | File | Purpose | AskUserQuestion |
+|---|---|---|---|
+| 1 | [phase-1-analyze.md](phase-1-analyze.md) | Run `kei-memory analyze` + `kei-memory patterns`; collect findings | 0 |
+| 2 | [phase-2-classify.md](phase-2-classify.md) | Categorise each finding as recurring / one-off / unknown + severity | 1× AskUserQuestion (severity confirm) |
+| 3 | [phase-3-present.md](phase-3-present.md) | Multi-select click: which findings to address | 1× AskUserQuestion |
+| 4 | [phase-4-route.md](phase-4-route.md) | For each selected finding → pick action route | 1× AskUserQuestion per selected finding |
+| 5 | [phase-5-backlog.md](phase-5-backlog.md) | Update `~/.claude/memory/audit-backlog.md`; clear processed items | 1× AskUserQuestion (confirm backlog clear) |
+
+Minimum AskUserQuestion count: **4** (Phase 2, 3, at least one Phase 4,
+Phase 5). This is the pure-click contract.
+
+---
+
+## Variables the pipeline produces
+
+| Name | Set in | Meaning |
+|---|---|---|
+| `SESSION` | Phase 1 | Session id under audit (CLI arg or `--last 1`) |
+| `FINDINGS` | Phase 1 | List of `{class, count, severity_hint, scope}` dicts |
+| `CLASSIFIED` | Phase 2 | Same list, with `category ∈ {recurring, one-off, unknown}` + `severity ∈ {critical, high, medium, low}` |
+| `SELECTED` | Phase 3 | User-picked subset of `CLASSIFIED` to address |
+| `ROUTES` | Phase 4 | Per-finding chosen action ∈ {codify, deep-dive, hook-only, log-only, postpone} |
+| `BACKLOG_ACTIONS` | Phase 5 | Which backlog entries to mark processed |
+
+---
+
+## Final report (emit after Phase 5)
+
+```
+=== SELF-AUDIT REPORT ===
+Session:       <SESSION>
+Findings:      <N total>  (recurring: <R>, one-off: <O>, unknown: <U>)
+Routed:        <K>
+  → codify:    <count>  (handed off to /escalate-recurrence)
+  → deep-dive: <count>  (handed off to /debug-deep)
+  → hook-only: <count>  (created hook stub — NOT registered)
+  → log-only:  <count>  (appended to audit-backlog.md)
+  → postpone:  <count>  (kept open, will resurface next session)
+Backlog:       <before_count> → <after_count> unprocessed items
+```
+
+---
+
+## Rules (apply throughout — enforced at every phase)
+
+- **Triage, not implementation.** This skill NEVER writes production
+  code. It hands off to `/escalate-recurrence` (rule + wiki + hook) or
+  `/debug-deep` (5-phase RCA) or logs to backlog. Any edit in this skill
+  is limited to `~/.claude/memory/audit-backlog.md`.
+- **Pure-click contract.** Only the handoff targets may ask for
+  free-text; every decision in self-audit itself is `AskUserQuestion`.
+- **NO DOWNGRADE (RULE -1).** If `kei-memory` is not installed, return
+  2-3 constructive paths (install the primitive, run the analysis by
+  hand on the JSONL, skip this session) — never "cannot audit".
+- **NO HALLUCINATION (RULE 0.4).** Every finding cited in Phase 3 must
+  come from the `kei-memory patterns` output captured in Phase 1.
+  Never invent a class that wasn't emitted.
+- **Silent-first (RULE 0.14).** If `<!-- session_count: N -->` in
+  `~/.claude/memory/audit-backlog.md` is less than 10, Phase 3 MUST
+  short-circuit to "log only" — do not prompt the user.
+- **Patent-IP exception.** If CWD sits under a banned-list project
+  (see `~/.claude/rules/security.md`) OR `CLAUDE.md` in CWD contains a
+  banned marker, run Phase 1 ONLY and stop: do not inject transcript
+  excerpts back into chat.
+- **Constructor Pattern (RULE ZERO).** Every phase file ≤ 60 LOC.
+
+---
+
+## References
+
+- `~/.claude/rules/session-self-audit.md` — RULE 0.14 full text
+- `~/.claude/skills/escalate-recurrence/SKILL.md` — codify route target
+- `skills/debug-deep/SKILL.md` — deep-dive route target
+- `_primitives/_rust/kei-memory/` — analyzer primitive
+- `hooks/session-end-dump.sh`, `hooks/milestone-commit-hook.sh`,
+  `hooks/error-spike-detector.sh` — auto-triggers
--- a/skills/self-audit/phase-1-analyze.md
+++ b/skills/self-audit/phase-1-analyze.md
@ -0,0 +1,53 @@
+# Phase 1 — Analyze
+
+Run the `kei-memory` analyzer against the target session and aggregate
+its output into a `FINDINGS` list for downstream phases.
+
+## 1a — Resolve session
+
+If the caller passed an argument, use it verbatim as `SESSION`. Otherwise
+resolve via:
+
+```
+kei-memory analyze --last 1 --summary
+```
+
+Parse the `session=<id>` field from the first line. That is `SESSION`.
+
+If the command fails (exit != 0) OR returns `(no sessions ingested yet)`
+— return the 3 constructive paths from the skill's RULE -1 clause and
+stop; do not proceed to Phase 2.
+
+## 1b — Retrospective
+
+```
+kei-memory analyze --session <SESSION>
+```
+
+Capture the full report as `REPORT`. It includes: duration, tool-call
+count, error count, top tools, top files.
+
+## 1c — In-session patterns
+
+```
+kei-memory patterns --session <SESSION>
+```
+
+Capture each line as `{event_class, count, session_id: SESSION}` and
+append to `FINDINGS` with `scope = "in-session"`.
+
+## 1d — Cross-session patterns
+
+```
+kei-memory patterns --cross-session
+```
+
+Capture each line as `{event_class, count, session_id: null}` and
+append to `FINDINGS` with `scope = "cross-session"`.
+
+## Verify-criterion
+
+- `SESSION` is a non-empty session id.
+- `REPORT` is captured (even if empty).
+- `FINDINGS` is a list (possibly empty). Empty → Phase 5 short-circuit
+  (nothing to triage, nothing to route).
--- a/skills/self-audit/phase-2-classify.md
+++ b/skills/self-audit/phase-2-classify.md
@ -0,0 +1,55 @@
+# Phase 2 — Classify
+
+Tag each finding with a category and a severity; `CLASSIFIED` is produced.
+
+## 2a — Category (automatic, no click)
+
+For each finding in `FINDINGS`:
+
+- `scope == "in-session"` AND `count ≥ 2` → `category = recurring`
+- `scope == "cross-session"` AND `count ≥ 2` → `category = recurring`
+- `scope == "in-session"` AND `count == 1` → `category = one-off`
+- otherwise → `category = unknown`
+
+## 2b — Severity hint (automatic heuristic)
+
+Grep the finding's `event_class`:
+
+| Contains                             | Severity   |
+|--------------------------------------|------------|
+| `permission_denied`, `panic`, `security` | `critical` |
+| `error`, `failed`, `timeout`, `worktree_error` | `high`     |
+| `cargo_workspace`, `tool_use:*`      | `medium`   |
+| anything else                        | `low`      |
+
+## 2c — Severity confirm click (single AskUserQuestion)
+
+Emit ONE `AskUserQuestion` batch grouping the severity confirm into a
+single question:
+
+```json
+{
+  "questions": [
+    {
+      "question": "Confirm severity for top finding?",
+      "header": "Severity",
+      "multiSelect": false,
+      "options": [
+        {"label": "critical", "description": "Security / data loss / irreversible"},
+        {"label": "high",     "description": "Blocks work or leaks to production"},
+        {"label": "medium",   "description": "Slows work; fix this week"},
+        {"label": "low",      "description": "Nice to fix; not urgent"}
+      ]
+    }
+  ]
+}
+```
+
+Apply the user's pick only to the TOP finding (highest `count`). All
+other findings keep their heuristic severity.
+
+## Verify-criterion
+
+- Every finding has a `category` and a `severity`.
+- `CLASSIFIED` is the full list with those two fields added.
+- Exactly one `AskUserQuestion` call was emitted.
--- a/skills/self-audit/phase-3-present.md
+++ b/skills/self-audit/phase-3-present.md
@ -0,0 +1,58 @@
+# Phase 3 — Present
+
+Show `CLASSIFIED` to the user as a multi-select click batch. User picks
+which findings to address; selection becomes `SELECTED`.
+
+## 3a — Silent-first guard
+
+Read `~/.claude/memory/audit-backlog.md`. Parse the
+`<!-- session_count: N -->` header. If `N < 10`:
+
+- Log every finding to the backlog with a `[SELF-AUDIT SILENT]` prefix.
+- Set `SELECTED = []` and SKIP to Phase 5.
+
+This is the RULE 0.14 silent-first contract. Do NOT prompt the user.
+
+## 3b — Patent-IP guard
+
+If CWD sits under a banned project (`~/Projects/KeiLab`, `~/Projects/keinet`,
+`~/Projects/keidog`, `~/Projects/vortex`, `~/Projects/neuralcloak`,
+`~/Projects/KGL`) OR a `CLAUDE.md` in CWD contains a banned-marker line
+matching `/banned-project|patent-ip/i`:
+
+- Log every finding to backlog with `[SELF-AUDIT OFFLINE]` prefix.
+- Set `SELECTED = []` and SKIP to Phase 5.
+
+Do NOT render transcript excerpts back to chat.
+
+## 3c — Multi-select click
+
+Emit ONE `AskUserQuestion`:
+
+```json
+{
+  "questions": [
+    {
+      "question": "Which findings do you want to address this session?",
+      "header": "Findings",
+      "multiSelect": true,
+      "options": [
+        {"label": "<class-1>  ×<count> [severity]", "description": "<scope>"},
+        {"label": "<class-2>  ×<count> [severity]", "description": "<scope>"},
+        ...
+        {"label": "None — just log to backlog",      "description": "Append all to backlog, pick up later"}
+      ]
+    }
+  ]
+}
+```
+
+Cap the option list at 8 findings (highest `count` first). If more exist,
+add a trailing option `"Show full list"` that dumps all of them to stdout
+and re-emits the click batch on the next turn.
+
+## Verify-criterion
+
+- Exactly one `AskUserQuestion` call was emitted (unless guard fired).
+- `SELECTED` is a list of finding dicts (possibly empty).
+- "None — just log to backlog" treated as `SELECTED = []`.
--- a/skills/self-audit/phase-4-route.md
+++ b/skills/self-audit/phase-4-route.md
@ -0,0 +1,58 @@
+# Phase 4 — Route
+
+For each finding in `SELECTED`, ask the user which action route to take.
+Each selection becomes one entry in `ROUTES`.
+
+## 4a — Per-finding click
+
+For EACH finding in `SELECTED`, emit ONE `AskUserQuestion`:
+
+```json
+{
+  "questions": [
+    {
+      "question": "Action for: <class>  ×<count> [<severity>]?",
+      "header": "Route",
+      "multiSelect": false,
+      "options": [
+        {"label": "codify via /escalate-recurrence", "description": "Make this a permanent rule + wiki entry + optional hook (recurring patterns)"},
+        {"label": "deep-dive via /debug-deep",       "description": "Run the 5-phase RCA skill (one-off or unknown bugs)"},
+        {"label": "create hook",                     "description": "Hand off to /escalate-recurrence but force the hook-only branch (mechanical blocks/enforce/warn)"},
+        {"label": "skip (just log)",                 "description": "Append to audit-backlog with a note; no further action this session"},
+        {"label": "postpone (remind next session)",  "description": "Keep open; re-surface at the start of the next self-audit"}
+      ]
+    }
+  ]
+}
+```
+
+## 4b — Handoff rules
+
+Based on the click, append to `ROUTES` one of:
+
+| Click label             | Action                                                            |
+|-------------------------|-------------------------------------------------------------------|
+| codify                  | Run `/escalate-recurrence` with `CLASS=<event_class>` prefilled   |
+| deep-dive               | Run `/debug-deep` with the class as the error description         |
+| create hook             | Run `/escalate-recurrence` and select "hook-only" in its phase 2  |
+| skip (just log)         | Append `[LOGGED yyyy-mm-dd] <class>` to backlog; no skill handoff |
+| postpone                | Append `[POSTPONE yyyy-mm-dd] <class>` to backlog; no handoff     |
+
+Self-audit itself does not perform the handoff action — it emits the
+`/<slash-skill>` invocation as a suggested next step. The user runs it.
+
+## 4c — Severity gate
+
+If a finding has `severity == critical` AND the user selected
+"postpone" or "skip" — echo one reminder line:
+
+> "Critical finding <class> postponed/skipped. It will resurface next
+> session but consider addressing before close."
+
+Do not block; this is advisory only.
+
+## Verify-criterion
+
+- `ROUTES` has exactly one entry per `SELECTED` finding.
+- One `AskUserQuestion` call per finding, no batching.
+- No actual fixes written — only suggested handoffs printed.
--- a/skills/self-audit/phase-5-backlog.md
+++ b/skills/self-audit/phase-5-backlog.md
@ -0,0 +1,55 @@
+# Phase 5 — Backlog
+
+Update `~/.claude/memory/audit-backlog.md`: increment the session counter,
+mark processed findings as done, leave postponed ones open.
+
+## 5a — Increment session counter
+
+Read the `<!-- session_count: N -->` header. Rewrite the same line with
+`N + 1`. This advances the silent-first threshold by one session.
+
+## 5b — Append per-finding notes
+
+For each finding, append one line based on its `ROUTES` entry:
+
+- `codify` → `- [CODIFIED yyyy-mm-dd] <class> ×<count>  → /escalate-recurrence`
+- `deep-dive` → `- [DEEP-DIVE yyyy-mm-dd] <class>  → /debug-deep`
+- `create hook` → `- [HOOK-ONLY yyyy-mm-dd] <class>  → /escalate-recurrence (hook branch)`
+- `skip` → `- [LOGGED yyyy-mm-dd] <class> ×<count>`
+- `postpone` → `- [POSTPONE yyyy-mm-dd] <class> ×<count>  (resurface next session)`
+
+If Phase 3 short-circuited (silent-first OR patent-IP guard), append all
+findings with the `[SELF-AUDIT SILENT]` or `[SELF-AUDIT OFFLINE]` prefix.
+
+## 5c — Clear processed items click
+
+Emit ONE `AskUserQuestion`:
+
+```json
+{
+  "questions": [
+    {
+      "question": "Clear processed (codified / deep-dived / logged) items from backlog?",
+      "header": "Clear",
+      "multiSelect": false,
+      "options": [
+        {"label": "Yes — clear now",      "description": "Mark processed=1 in kei-memory backlog table"},
+        {"label": "No — keep for review", "description": "Review later before clearing"}
+      ]
+    }
+  ]
+}
+```
+
+On "Yes" — run `kei-memory backlog --clear`. On "No" — no-op.
+
+## 5d — Emit final report
+
+Print the final report (format from `SKILL.md`).
+
+## Verify-criterion
+
+- `<!-- session_count: N -->` header incremented by exactly 1.
+- Every finding has a backlog line (appended in 5b).
+- Exactly one `AskUserQuestion` call in 5c.
+- Final report printed after backlog write.