From 139d2f03250825cc18d8c0663f844053228a3edf Mon Sep 17 00:00:00 2001 From: Parfii-bot Date: Wed, 22 Apr 2026 00:50:04 +0800 Subject: [PATCH] feat(skills): /self-audit 5-phase triage pipeline --- skills/self-audit/SKILL.md | 96 +++++++++++++++++++++++++++ skills/self-audit/phase-1-analyze.md | 53 +++++++++++++++ skills/self-audit/phase-2-classify.md | 55 +++++++++++++++ skills/self-audit/phase-3-present.md | 58 ++++++++++++++++ skills/self-audit/phase-4-route.md | 58 ++++++++++++++++ skills/self-audit/phase-5-backlog.md | 55 +++++++++++++++ 6 files changed, 375 insertions(+) create mode 100644 skills/self-audit/SKILL.md create mode 100644 skills/self-audit/phase-1-analyze.md create mode 100644 skills/self-audit/phase-2-classify.md create mode 100644 skills/self-audit/phase-3-present.md create mode 100644 skills/self-audit/phase-4-route.md create mode 100644 skills/self-audit/phase-5-backlog.md diff --git a/skills/self-audit/SKILL.md b/skills/self-audit/SKILL.md new file mode 100644 index 0000000..4006a6a --- /dev/null +++ b/skills/self-audit/SKILL.md @@ -0,0 +1,96 @@ +--- +name: self-audit +description: Session retrospective triage (RULE 0.14). Runs `kei-memory analyze + patterns`, classifies findings, presents them via click-only AskUserQuestion, routes each selected item to `/escalate-recurrence` (rule+hook), `/debug-deep` (bug RCA), or the audit-backlog (log-only). Self-audit is triage, not implementation. +argument-hint: +--- + +# Self-Audit — Session Retrospective Triage (index) + +You are running the RULE 0.14 self-audit on the last (or named) session. +You convert the session's trace into a short list of findings, classify +each, present them as a multi-select click batch, and route each selection +to the appropriate existing skill. You NEVER write fixes yourself. + +This `SKILL.md` is the INDEX. Each phase lives in its own file and is +executed in order. Never skip a phase. Never re-order phases. + +--- + +## Pipeline overview (5 phases) + +| Phase | File | Purpose | AskUserQuestion | +|---|---|---|---| +| 1 | [phase-1-analyze.md](phase-1-analyze.md) | Run `kei-memory analyze` + `kei-memory patterns`; collect findings | 0 | +| 2 | [phase-2-classify.md](phase-2-classify.md) | Categorise each finding as recurring / one-off / unknown + severity | 1× AskUserQuestion (severity confirm) | +| 3 | [phase-3-present.md](phase-3-present.md) | Multi-select click: which findings to address | 1× AskUserQuestion | +| 4 | [phase-4-route.md](phase-4-route.md) | For each selected finding → pick action route | 1× AskUserQuestion per selected finding | +| 5 | [phase-5-backlog.md](phase-5-backlog.md) | Update `~/.claude/memory/audit-backlog.md`; clear processed items | 1× AskUserQuestion (confirm backlog clear) | + +Minimum AskUserQuestion count: **4** (Phase 2, 3, at least one Phase 4, +Phase 5). This is the pure-click contract. + +--- + +## Variables the pipeline produces + +| Name | Set in | Meaning | +|---|---|---| +| `SESSION` | Phase 1 | Session id under audit (CLI arg or `--last 1`) | +| `FINDINGS` | Phase 1 | List of `{class, count, severity_hint, scope}` dicts | +| `CLASSIFIED` | Phase 2 | Same list, with `category ∈ {recurring, one-off, unknown}` + `severity ∈ {critical, high, medium, low}` | +| `SELECTED` | Phase 3 | User-picked subset of `CLASSIFIED` to address | +| `ROUTES` | Phase 4 | Per-finding chosen action ∈ {codify, deep-dive, hook-only, log-only, postpone} | +| `BACKLOG_ACTIONS` | Phase 5 | Which backlog entries to mark processed | + +--- + +## Final report (emit after Phase 5) + +``` +=== SELF-AUDIT REPORT === +Session: +Findings: (recurring: , one-off: , unknown: ) +Routed: + → codify: (handed off to /escalate-recurrence) + → deep-dive: (handed off to /debug-deep) + → hook-only: (created hook stub — NOT registered) + → log-only: (appended to audit-backlog.md) + → postpone: (kept open, will resurface next session) +Backlog: → unprocessed items +``` + +--- + +## Rules (apply throughout — enforced at every phase) + +- **Triage, not implementation.** This skill NEVER writes production + code. It hands off to `/escalate-recurrence` (rule + wiki + hook) or + `/debug-deep` (5-phase RCA) or logs to backlog. Any edit in this skill + is limited to `~/.claude/memory/audit-backlog.md`. +- **Pure-click contract.** Only the handoff targets may ask for + free-text; every decision in self-audit itself is `AskUserQuestion`. +- **NO DOWNGRADE (RULE -1).** If `kei-memory` is not installed, return + 2-3 constructive paths (install the primitive, run the analysis by + hand on the JSONL, skip this session) — never "cannot audit". +- **NO HALLUCINATION (RULE 0.4).** Every finding cited in Phase 3 must + come from the `kei-memory patterns` output captured in Phase 1. + Never invent a class that wasn't emitted. +- **Silent-first (RULE 0.14).** If `` in + `~/.claude/memory/audit-backlog.md` is less than 10, Phase 3 MUST + short-circuit to "log only" — do not prompt the user. +- **Patent-IP exception.** If CWD sits under a banned-list project + (see `~/.claude/rules/security.md`) OR `CLAUDE.md` in CWD contains a + banned marker, run Phase 1 ONLY and stop: do not inject transcript + excerpts back into chat. +- **Constructor Pattern (RULE ZERO).** Every phase file ≤ 60 LOC. + +--- + +## References + +- `~/.claude/rules/session-self-audit.md` — RULE 0.14 full text +- `~/.claude/skills/escalate-recurrence/SKILL.md` — codify route target +- `skills/debug-deep/SKILL.md` — deep-dive route target +- `_primitives/_rust/kei-memory/` — analyzer primitive +- `hooks/session-end-dump.sh`, `hooks/milestone-commit-hook.sh`, + `hooks/error-spike-detector.sh` — auto-triggers diff --git a/skills/self-audit/phase-1-analyze.md b/skills/self-audit/phase-1-analyze.md new file mode 100644 index 0000000..2dac19e --- /dev/null +++ b/skills/self-audit/phase-1-analyze.md @@ -0,0 +1,53 @@ +# Phase 1 — Analyze + +Run the `kei-memory` analyzer against the target session and aggregate +its output into a `FINDINGS` list for downstream phases. + +## 1a — Resolve session + +If the caller passed an argument, use it verbatim as `SESSION`. Otherwise +resolve via: + +``` +kei-memory analyze --last 1 --summary +``` + +Parse the `session=` field from the first line. That is `SESSION`. + +If the command fails (exit != 0) OR returns `(no sessions ingested yet)` +— return the 3 constructive paths from the skill's RULE -1 clause and +stop; do not proceed to Phase 2. + +## 1b — Retrospective + +``` +kei-memory analyze --session +``` + +Capture the full report as `REPORT`. It includes: duration, tool-call +count, error count, top tools, top files. + +## 1c — In-session patterns + +``` +kei-memory patterns --session +``` + +Capture each line as `{event_class, count, session_id: SESSION}` and +append to `FINDINGS` with `scope = "in-session"`. + +## 1d — Cross-session patterns + +``` +kei-memory patterns --cross-session +``` + +Capture each line as `{event_class, count, session_id: null}` and +append to `FINDINGS` with `scope = "cross-session"`. + +## Verify-criterion + +- `SESSION` is a non-empty session id. +- `REPORT` is captured (even if empty). +- `FINDINGS` is a list (possibly empty). Empty → Phase 5 short-circuit + (nothing to triage, nothing to route). diff --git a/skills/self-audit/phase-2-classify.md b/skills/self-audit/phase-2-classify.md new file mode 100644 index 0000000..0c32660 --- /dev/null +++ b/skills/self-audit/phase-2-classify.md @@ -0,0 +1,55 @@ +# Phase 2 — Classify + +Tag each finding with a category and a severity; `CLASSIFIED` is produced. + +## 2a — Category (automatic, no click) + +For each finding in `FINDINGS`: + +- `scope == "in-session"` AND `count ≥ 2` → `category = recurring` +- `scope == "cross-session"` AND `count ≥ 2` → `category = recurring` +- `scope == "in-session"` AND `count == 1` → `category = one-off` +- otherwise → `category = unknown` + +## 2b — Severity hint (automatic heuristic) + +Grep the finding's `event_class`: + +| Contains | Severity | +|--------------------------------------|------------| +| `permission_denied`, `panic`, `security` | `critical` | +| `error`, `failed`, `timeout`, `worktree_error` | `high` | +| `cargo_workspace`, `tool_use:*` | `medium` | +| anything else | `low` | + +## 2c — Severity confirm click (single AskUserQuestion) + +Emit ONE `AskUserQuestion` batch grouping the severity confirm into a +single question: + +```json +{ + "questions": [ + { + "question": "Confirm severity for top finding?", + "header": "Severity", + "multiSelect": false, + "options": [ + {"label": "critical", "description": "Security / data loss / irreversible"}, + {"label": "high", "description": "Blocks work or leaks to production"}, + {"label": "medium", "description": "Slows work; fix this week"}, + {"label": "low", "description": "Nice to fix; not urgent"} + ] + } + ] +} +``` + +Apply the user's pick only to the TOP finding (highest `count`). All +other findings keep their heuristic severity. + +## Verify-criterion + +- Every finding has a `category` and a `severity`. +- `CLASSIFIED` is the full list with those two fields added. +- Exactly one `AskUserQuestion` call was emitted. diff --git a/skills/self-audit/phase-3-present.md b/skills/self-audit/phase-3-present.md new file mode 100644 index 0000000..45e65b2 --- /dev/null +++ b/skills/self-audit/phase-3-present.md @@ -0,0 +1,58 @@ +# Phase 3 — Present + +Show `CLASSIFIED` to the user as a multi-select click batch. User picks +which findings to address; selection becomes `SELECTED`. + +## 3a — Silent-first guard + +Read `~/.claude/memory/audit-backlog.md`. Parse the +`` header. If `N < 10`: + +- Log every finding to the backlog with a `[SELF-AUDIT SILENT]` prefix. +- Set `SELECTED = []` and SKIP to Phase 5. + +This is the RULE 0.14 silent-first contract. Do NOT prompt the user. + +## 3b — Patent-IP guard + +If CWD sits under a banned project (`~/Projects/KeiLab`, `~/Projects/keinet`, +`~/Projects/keidog`, `~/Projects/vortex`, `~/Projects/neuralcloak`, +`~/Projects/KGL`) OR a `CLAUDE.md` in CWD contains a banned-marker line +matching `/banned-project|patent-ip/i`: + +- Log every finding to backlog with `[SELF-AUDIT OFFLINE]` prefix. +- Set `SELECTED = []` and SKIP to Phase 5. + +Do NOT render transcript excerpts back to chat. + +## 3c — Multi-select click + +Emit ONE `AskUserQuestion`: + +```json +{ + "questions": [ + { + "question": "Which findings do you want to address this session?", + "header": "Findings", + "multiSelect": true, + "options": [ + {"label": " × [severity]", "description": ""}, + {"label": " × [severity]", "description": ""}, + ... + {"label": "None — just log to backlog", "description": "Append all to backlog, pick up later"} + ] + } + ] +} +``` + +Cap the option list at 8 findings (highest `count` first). If more exist, +add a trailing option `"Show full list"` that dumps all of them to stdout +and re-emits the click batch on the next turn. + +## Verify-criterion + +- Exactly one `AskUserQuestion` call was emitted (unless guard fired). +- `SELECTED` is a list of finding dicts (possibly empty). +- "None — just log to backlog" treated as `SELECTED = []`. diff --git a/skills/self-audit/phase-4-route.md b/skills/self-audit/phase-4-route.md new file mode 100644 index 0000000..8dc874f --- /dev/null +++ b/skills/self-audit/phase-4-route.md @@ -0,0 +1,58 @@ +# Phase 4 — Route + +For each finding in `SELECTED`, ask the user which action route to take. +Each selection becomes one entry in `ROUTES`. + +## 4a — Per-finding click + +For EACH finding in `SELECTED`, emit ONE `AskUserQuestion`: + +```json +{ + "questions": [ + { + "question": "Action for: × []?", + "header": "Route", + "multiSelect": false, + "options": [ + {"label": "codify via /escalate-recurrence", "description": "Make this a permanent rule + wiki entry + optional hook (recurring patterns)"}, + {"label": "deep-dive via /debug-deep", "description": "Run the 5-phase RCA skill (one-off or unknown bugs)"}, + {"label": "create hook", "description": "Hand off to /escalate-recurrence but force the hook-only branch (mechanical blocks/enforce/warn)"}, + {"label": "skip (just log)", "description": "Append to audit-backlog with a note; no further action this session"}, + {"label": "postpone (remind next session)", "description": "Keep open; re-surface at the start of the next self-audit"} + ] + } + ] +} +``` + +## 4b — Handoff rules + +Based on the click, append to `ROUTES` one of: + +| Click label | Action | +|-------------------------|-------------------------------------------------------------------| +| codify | Run `/escalate-recurrence` with `CLASS=` prefilled | +| deep-dive | Run `/debug-deep` with the class as the error description | +| create hook | Run `/escalate-recurrence` and select "hook-only" in its phase 2 | +| skip (just log) | Append `[LOGGED yyyy-mm-dd] ` to backlog; no skill handoff | +| postpone | Append `[POSTPONE yyyy-mm-dd] ` to backlog; no handoff | + +Self-audit itself does not perform the handoff action — it emits the +`/` invocation as a suggested next step. The user runs it. + +## 4c — Severity gate + +If a finding has `severity == critical` AND the user selected +"postpone" or "skip" — echo one reminder line: + +> "Critical finding postponed/skipped. It will resurface next +> session but consider addressing before close." + +Do not block; this is advisory only. + +## Verify-criterion + +- `ROUTES` has exactly one entry per `SELECTED` finding. +- One `AskUserQuestion` call per finding, no batching. +- No actual fixes written — only suggested handoffs printed. diff --git a/skills/self-audit/phase-5-backlog.md b/skills/self-audit/phase-5-backlog.md new file mode 100644 index 0000000..da4c334 --- /dev/null +++ b/skills/self-audit/phase-5-backlog.md @@ -0,0 +1,55 @@ +# Phase 5 — Backlog + +Update `~/.claude/memory/audit-backlog.md`: increment the session counter, +mark processed findings as done, leave postponed ones open. + +## 5a — Increment session counter + +Read the `` header. Rewrite the same line with +`N + 1`. This advances the silent-first threshold by one session. + +## 5b — Append per-finding notes + +For each finding, append one line based on its `ROUTES` entry: + +- `codify` → `- [CODIFIED yyyy-mm-dd] × → /escalate-recurrence` +- `deep-dive` → `- [DEEP-DIVE yyyy-mm-dd] → /debug-deep` +- `create hook` → `- [HOOK-ONLY yyyy-mm-dd] → /escalate-recurrence (hook branch)` +- `skip` → `- [LOGGED yyyy-mm-dd] ×` +- `postpone` → `- [POSTPONE yyyy-mm-dd] × (resurface next session)` + +If Phase 3 short-circuited (silent-first OR patent-IP guard), append all +findings with the `[SELF-AUDIT SILENT]` or `[SELF-AUDIT OFFLINE]` prefix. + +## 5c — Clear processed items click + +Emit ONE `AskUserQuestion`: + +```json +{ + "questions": [ + { + "question": "Clear processed (codified / deep-dived / logged) items from backlog?", + "header": "Clear", + "multiSelect": false, + "options": [ + {"label": "Yes — clear now", "description": "Mark processed=1 in kei-memory backlog table"}, + {"label": "No — keep for review", "description": "Review later before clearing"} + ] + } + ] +} +``` + +On "Yes" — run `kei-memory backlog --clear`. On "No" — no-op. + +## 5d — Emit final report + +Print the final report (format from `SKILL.md`). + +## Verify-criterion + +- `` header incremented by exactly 1. +- Every finding has a backlog line (appended in 5b). +- Exactly one `AskUserQuestion` call in 5c. +- Final report printed after backlog write.