Wave A — Functional ingest fix (root cause of empty Sleep reports):
- Rewrote TraceLine struct to match real Claude Code trace JSONL:
type (was kind), timestamp ISO8601 (was epoch ts), message Object,
cwd / gitBranch / parentUuid / uuid / subtype / toolUseID / toolUseResult
- New src/extract.rs: extract_tool_uses + extract_tool_result walks
message.content[] for nested tool_use / tool_result blocks
- New src/classifier.rs: explicit table classifier (tool_error, user_correction,
retry_loop, permission_denied, tool_use:<name>, ...) replaces shallow heuristic
- New src/error.rs: KeiMemoryError enum (IO/Parse/Db) replaces semantic
mismatch where IO error was wrapped as rusqlite::InvalidParameterName
- New src/trace_line.rs: TraceLine + helpers (cube extraction)
- Schema migration v3: events.cwd column + 3 hot-query indices
(events.tool, events.file_path, events.ts) + UNIQUE on patterns
- New tests/ingest_real_trace.rs: synth-fixture asserts tool/file/cwd/class extraction
Wave B — Lib crate split:
- Cargo.toml: [lib] target added alongside existing [[bin]]
- src/lib.rs: pub re-export of all 18 modules
- src/main.rs: 11 mod declarations replaced by single use kei_memory::{…}
- tests/integration.rs: #[path] hack replaced by use kei_memory::{…}
Wave C — TF-IDF dedup + single-JOIN + filter_map fix:
- Schema migration v2: tokens.idf_dirty column + flag-based dedup
- index_document no longer triggers per-call recompute_idf rebuild
- top_similar uses single JOIN via vectors_for_overlapping_sessions helper
(was N round-trips, one session_vector per candidate)
- All filter_map(|r| r.ok()) row-error swallowing replaced with ? propagation
- New tests/tfidf_idf_dedup.rs: 4 tests covering dedup behaviour, IDF emptiness,
JOIN-pruning, empty-query safety
Wave D — Commands split + nits:
- New src/dump.rs (43 LOC) + src/stats.rs (33 LOC):
CLI renderers extracted from commands.rs (was inline SQL + format)
- src/commands.rs: thin wrappers, -42 LOC
- src/injection_guard.rs: inline tests removed (-26 LOC), file under 200 LOC threshold
- tests/injection_guard_unit.rs (new): 4 tests in proper integration crate
- src/patterns.rs: INSERT replaced with INSERT...ON CONFLICT...DO UPDATE
(idempotent re-ingest, uses Wave A's UNIQUE index)
- src/analyze.rs + src/coaccess.rs: filter_map row-error fixes
- src/coaccess.rs: misleading PK comment rewritten
Verify-before-commit (RULE 0.13 §"Verify-before-commit"):
- cargo check --all-targets: PASS (1 unrelated dead-code warning)
- cargo test: 42 passed, 0 failed across 9 test binaries
- STATUS-TRUTH markers aggregated at .claude/agents/_merge/kei-memory-2026-05-01/
Architect-spotted ARCH-MAJOR + ARCH-MINOR + ARCH-NIT findings addressed:
- ARCH-MAJOR Cargo.toml binary-only (Wave B)
- ARCH-MAJOR schema missing indices (Wave A v3)
- ARCH-MAJOR ingest_jsonl choke point (Wave A — extract.rs + classifier.rs)
- ARCH-MAJOR idf O(N·V) per-call rebuild (Wave C)
- ARCH-MINOR patterns no UPSERT (Wave D)
- ARCH-MINOR commands.rs houses dump+stats (Wave D)
- ARCH-MINOR classifier silent contract (Wave A)
- ARCH-MINOR IO error wrapped as rusqlite (Wave A)
- ARCH-MINOR injection_guard inline tests (Wave D)
- ARCH-MINOR tfidf top_similar N round-trips (Wave C)
- ARCH-NIT 3× filter_map(|r| r.ok()) sites (Wave C + D)
- ARCH-NIT coaccess misleading comment (Wave D)
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
cargo-test: PASS (42 tests, 0 failures)
behaviour-verified: yes
follow-up-required:
- tests/ingest_guard_tests.rs + tests/guard_test_corpus.rs still on #[path] hack (Wave B follow-up note, ~5 LOC)
- dead_code warning Severity::Warn unused (pre-existing, not blocking)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
101 lines
3.6 KiB
Rust
101 lines
3.6 KiB
Rust
//! Pattern detector — recurring event-classes.
|
|
//!
|
|
//! Constructor Pattern: one cube, one read/write responsibility.
|
|
//! A "pattern" is an event_class that occurred ≥2 times in ONE session
|
|
//! (in-session recurrence) or ≥2 times across DIFFERENT sessions
|
|
//! (cross-session recurrence). Results are persisted into `patterns` and
|
|
//! also returned to the caller for display.
|
|
|
|
use rusqlite::{params, Connection, Result};
|
|
|
|
#[derive(Debug)]
|
|
#[allow(dead_code)]
|
|
pub struct PatternHit {
|
|
pub event_class: String,
|
|
pub session_id: Option<String>,
|
|
pub count: i64,
|
|
}
|
|
|
|
/// Detect in-session recurrences for `session_id`. Persists rows.
|
|
pub fn detect_in_session(conn: &Connection, session_id: &str) -> Result<Vec<PatternHit>> {
|
|
let mut stmt = conn.prepare(
|
|
"SELECT event_class, COUNT(*), MIN(ts), MAX(ts)
|
|
FROM events
|
|
WHERE session_id = ?1 AND event_class IS NOT NULL
|
|
GROUP BY event_class HAVING COUNT(*) >= 2
|
|
ORDER BY COUNT(*) DESC",
|
|
)?;
|
|
let rows = stmt
|
|
.query_map(params![session_id], |r| {
|
|
Ok((
|
|
r.get::<_, String>(0)?,
|
|
r.get::<_, i64>(1)?,
|
|
r.get::<_, i64>(2)?,
|
|
r.get::<_, i64>(3)?,
|
|
))
|
|
})?
|
|
.collect::<Result<Vec<_>>>()?;
|
|
let mut out = Vec::new();
|
|
for (class, count, first, last) in rows {
|
|
// UPSERT: schema v3 added UNIQUE(event_class, COALESCE(session_id,'')).
|
|
// Re-ingest of the same session no longer duplicates rows; counts
|
|
// accumulate, last_seen_ts moves forward, first_seen_ts stays put.
|
|
conn.execute(
|
|
"INSERT INTO patterns (event_class, session_id, count, first_seen_ts, last_seen_ts)
|
|
VALUES (?1, ?2, ?3, ?4, ?5)
|
|
ON CONFLICT(event_class, COALESCE(session_id, '')) DO UPDATE SET
|
|
count = patterns.count + excluded.count,
|
|
last_seen_ts = MAX(patterns.last_seen_ts, excluded.last_seen_ts)",
|
|
params![class, session_id, count, first, last],
|
|
)?;
|
|
out.push(PatternHit {
|
|
event_class: class,
|
|
session_id: Some(session_id.to_string()),
|
|
count,
|
|
});
|
|
}
|
|
Ok(out)
|
|
}
|
|
|
|
/// Detect cross-session recurrences. Does NOT persist (history aggregate).
|
|
pub fn detect_cross_session(conn: &Connection) -> Result<Vec<PatternHit>> {
|
|
let mut stmt = conn.prepare(
|
|
"SELECT event_class, COUNT(DISTINCT session_id)
|
|
FROM events
|
|
WHERE event_class IS NOT NULL
|
|
GROUP BY event_class HAVING COUNT(DISTINCT session_id) >= 2
|
|
ORDER BY COUNT(DISTINCT session_id) DESC",
|
|
)?;
|
|
let rows = stmt
|
|
.query_map([], |r| {
|
|
Ok((r.get::<_, String>(0)?, r.get::<_, i64>(1)?))
|
|
})?
|
|
.collect::<Result<Vec<_>>>()?;
|
|
Ok(rows
|
|
.into_iter()
|
|
.map(|(class, count)| PatternHit {
|
|
event_class: class,
|
|
session_id: None,
|
|
count,
|
|
})
|
|
.collect())
|
|
}
|
|
|
|
/// List all patterns in the persistent table (newest first).
|
|
#[allow(dead_code)]
|
|
pub fn list_all(conn: &Connection, limit: usize) -> Result<Vec<PatternHit>> {
|
|
let mut stmt = conn.prepare(
|
|
"SELECT event_class, session_id, count FROM patterns
|
|
ORDER BY last_seen_ts DESC LIMIT ?1",
|
|
)?;
|
|
let rows = stmt
|
|
.query_map(params![limit as i64], |r| {
|
|
Ok(PatternHit {
|
|
event_class: r.get::<_, String>(0)?,
|
|
session_id: Some(r.get::<_, String>(1)?),
|
|
count: r.get::<_, i64>(2)?,
|
|
})
|
|
})?
|
|
.collect::<Result<Vec<_>>>()?;
|
|
Ok(rows)
|
|
}
|