Wave A — Functional ingest fix (root cause of empty Sleep reports):
- Rewrote TraceLine struct to match real Claude Code trace JSONL:
type (was kind), timestamp ISO8601 (was epoch ts), message Object,
cwd / gitBranch / parentUuid / uuid / subtype / toolUseID / toolUseResult
- New src/extract.rs: extract_tool_uses + extract_tool_result walks
message.content[] for nested tool_use / tool_result blocks
- New src/classifier.rs: explicit table classifier (tool_error, user_correction,
retry_loop, permission_denied, tool_use:<name>, ...) replaces shallow heuristic
- New src/error.rs: KeiMemoryError enum (IO/Parse/Db) replaces semantic
mismatch where IO error was wrapped as rusqlite::InvalidParameterName
- New src/trace_line.rs: TraceLine + helpers (cube extraction)
- Schema migration v3: events.cwd column + 3 hot-query indices
(events.tool, events.file_path, events.ts) + UNIQUE on patterns
- New tests/ingest_real_trace.rs: synth-fixture asserts tool/file/cwd/class extraction
Wave B — Lib crate split:
- Cargo.toml: [lib] target added alongside existing [[bin]]
- src/lib.rs: pub re-export of all 18 modules
- src/main.rs: 11 mod declarations replaced by single use kei_memory::{…}
- tests/integration.rs: #[path] hack replaced by use kei_memory::{…}
Wave C — TF-IDF dedup + single-JOIN + filter_map fix:
- Schema migration v2: tokens.idf_dirty column + flag-based dedup
- index_document no longer triggers per-call recompute_idf rebuild
- top_similar uses single JOIN via vectors_for_overlapping_sessions helper
(was N round-trips, one session_vector per candidate)
- All filter_map(|r| r.ok()) row-error swallowing replaced with ? propagation
- New tests/tfidf_idf_dedup.rs: 4 tests covering dedup behaviour, IDF emptiness,
JOIN-pruning, empty-query safety
Wave D — Commands split + nits:
- New src/dump.rs (43 LOC) + src/stats.rs (33 LOC):
CLI renderers extracted from commands.rs (was inline SQL + format)
- src/commands.rs: thin wrappers, -42 LOC
- src/injection_guard.rs: inline tests removed (-26 LOC), file under 200 LOC threshold
- tests/injection_guard_unit.rs (new): 4 tests in proper integration crate
- src/patterns.rs: INSERT replaced with INSERT...ON CONFLICT...DO UPDATE
(idempotent re-ingest, uses Wave A's UNIQUE index)
- src/analyze.rs + src/coaccess.rs: filter_map row-error fixes
- src/coaccess.rs: misleading PK comment rewritten
Verify-before-commit (RULE 0.13 §"Verify-before-commit"):
- cargo check --all-targets: PASS (1 unrelated dead-code warning)
- cargo test: 42 passed, 0 failed across 9 test binaries
- STATUS-TRUTH markers aggregated at .claude/agents/_merge/kei-memory-2026-05-01/
Architect-spotted ARCH-MAJOR + ARCH-MINOR + ARCH-NIT findings addressed:
- ARCH-MAJOR Cargo.toml binary-only (Wave B)
- ARCH-MAJOR schema missing indices (Wave A v3)
- ARCH-MAJOR ingest_jsonl choke point (Wave A — extract.rs + classifier.rs)
- ARCH-MAJOR idf O(N·V) per-call rebuild (Wave C)
- ARCH-MINOR patterns no UPSERT (Wave D)
- ARCH-MINOR commands.rs houses dump+stats (Wave D)
- ARCH-MINOR classifier silent contract (Wave A)
- ARCH-MINOR IO error wrapped as rusqlite (Wave A)
- ARCH-MINOR injection_guard inline tests (Wave D)
- ARCH-MINOR tfidf top_similar N round-trips (Wave C)
- ARCH-NIT 3× filter_map(|r| r.ok()) sites (Wave C + D)
- ARCH-NIT coaccess misleading comment (Wave D)
=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
cargo-test: PASS (42 tests, 0 failures)
behaviour-verified: yes
follow-up-required:
- tests/ingest_guard_tests.rs + tests/guard_test_corpus.rs still on #[path] hack (Wave B follow-up note, ~5 LOC)
- dead_code warning Severity::Warn unused (pre-existing, not blocking)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
171 lines
5.8 KiB
Rust
171 lines
5.8 KiB
Rust
//! Injection / exfiltration guard for memory entries.
|
|
//!
|
|
//! Constructor Pattern: scan logic only; pattern definitions live in
|
|
//! `injection_patterns.rs`.
|
|
//!
|
|
//! ## Wire-points (3 paths protected, P2.1.b lock 2026-04-28)
|
|
//!
|
|
//! 1. `ingest::insert_event` — REAL memory writes from agent JSONL
|
|
//! transcripts. Each event message is scanned before it is persisted
|
|
//! into the `events` table. Block-tier hits short-circuit insertion.
|
|
//! 2. `kei-pet::memory::record_interaction` — user-facing pet
|
|
//! conversation memory. Uses a substring/char-only sibling guard
|
|
//! (`kei_pet::injection_check`) to avoid a regex dep bump on the pet
|
|
//! crate. Block-tier coverage mirrors this module's prompt-override
|
|
//! + invisible-unicode + PEM-marker rules.
|
|
//! 3. `cmd_backlog --add` — RULE 0.14 audit-CRUD. Backlog items are
|
|
//! rendered into self-audit reports verbatim; malicious content
|
|
//! survives that path the same way it would survive insert_event.
|
|
//!
|
|
//! All three paths use the same `Severity::Block` semantics: a hit
|
|
//! results in early-return / persistence-skip, with the finding logged.
|
|
//!
|
|
//! ## Rationale
|
|
//!
|
|
//! Memory entries are injected verbatim into the system prompt. Any
|
|
//! prompt-override fragment, role-prefix, ChatML tag, invisible bidi
|
|
//! codepoint, hardcoded credential, or large base64 attestation blob
|
|
//! survives that injection and becomes effective text the model reads.
|
|
//! The scan treats these as untrusted input and rejects them.
|
|
//!
|
|
//! ## Bypass
|
|
//!
|
|
//! `KEI_MEMORY_SKIP_GUARD=1` skips the scan after logging an explicit
|
|
//! warning to stderr. Intended for one-off recovery — never the default.
|
|
|
|
use crate::injection_patterns::{regex_patterns, substring_patterns, Severity, INVISIBLE_CHARS};
|
|
|
|
/// One pattern hit. Severity drives whether the call site rejects.
|
|
#[derive(Debug, Clone)]
|
|
pub struct InjectionFinding {
|
|
pub pattern: String,
|
|
pub line: usize,
|
|
pub severity: Severity,
|
|
pub source: String,
|
|
pub snippet: String,
|
|
}
|
|
|
|
impl std::fmt::Display for InjectionFinding {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
|
write!(
|
|
f,
|
|
"[{:?}] {} ({}) at line {}: {}",
|
|
self.severity, self.pattern, self.source, self.line, self.snippet
|
|
)
|
|
}
|
|
}
|
|
|
|
impl std::error::Error for InjectionFinding {}
|
|
|
|
/// Scan `content` for prompt-injection / secret-leak patterns.
|
|
///
|
|
/// Returns `Ok(())` when the content is clean OR contains only
|
|
/// `Warn`-level findings. Returns `Err(InjectionFinding)` on the first
|
|
/// `Block`-level hit. Use `scan_all` if all findings are wanted.
|
|
pub fn scan(content: &str) -> Result<(), InjectionFinding> {
|
|
if std::env::var("KEI_MEMORY_SKIP_GUARD").as_deref() == Ok("1") {
|
|
eprintln!(
|
|
"kei-memory: WARNING — injection guard bypassed via \
|
|
KEI_MEMORY_SKIP_GUARD=1 (RULE 0.4 audit-trail)"
|
|
);
|
|
return Ok(());
|
|
}
|
|
if let Some(f) = check_invisible(content) {
|
|
return Err(f);
|
|
}
|
|
for f in scan_regex(content) {
|
|
if f.severity == Severity::Block {
|
|
return Err(f);
|
|
}
|
|
}
|
|
for f in scan_substring(content) {
|
|
if f.severity == Severity::Block {
|
|
return Err(f);
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Like `scan` but returns every finding (Block + Warn). Useful for
|
|
/// diagnostics / triage tools that want the full picture.
|
|
#[allow(dead_code)]
|
|
pub fn scan_all(content: &str) -> Vec<InjectionFinding> {
|
|
let mut out = Vec::new();
|
|
if let Some(f) = check_invisible(content) {
|
|
out.push(f);
|
|
}
|
|
out.extend(scan_regex(content));
|
|
out.extend(scan_substring(content));
|
|
out
|
|
}
|
|
|
|
fn check_invisible(content: &str) -> Option<InjectionFinding> {
|
|
for (idx, line) in content.lines().enumerate() {
|
|
for ch in line.chars() {
|
|
if INVISIBLE_CHARS.contains(&ch) {
|
|
return Some(InjectionFinding {
|
|
pattern: "invisible_unicode".to_string(),
|
|
line: idx + 1,
|
|
severity: Severity::Block,
|
|
source: "unicode:bidi".to_string(),
|
|
snippet: format!("U+{:04X}", ch as u32),
|
|
});
|
|
}
|
|
}
|
|
}
|
|
None
|
|
}
|
|
|
|
fn scan_regex(content: &str) -> Vec<InjectionFinding> {
|
|
let mut out = Vec::new();
|
|
for p in regex_patterns() {
|
|
if let Some(m) = p.re.find(content) {
|
|
out.push(InjectionFinding {
|
|
pattern: p.id.to_string(),
|
|
line: line_of_offset(content, m.start()),
|
|
severity: p.severity,
|
|
source: p.source.to_string(),
|
|
snippet: truncate(m.as_str(), 60),
|
|
});
|
|
}
|
|
}
|
|
out
|
|
}
|
|
|
|
fn scan_substring(content: &str) -> Vec<InjectionFinding> {
|
|
let lower = content.to_lowercase();
|
|
let mut out = Vec::new();
|
|
for p in substring_patterns() {
|
|
if p.needles.iter().all(|n| lower.contains(n)) {
|
|
let needle = p.needles[0];
|
|
let line = lower
|
|
.find(needle)
|
|
.map(|off| line_of_offset(&lower, off))
|
|
.unwrap_or(1);
|
|
out.push(InjectionFinding {
|
|
pattern: p.id.to_string(),
|
|
line,
|
|
severity: p.severity,
|
|
source: p.source.to_string(),
|
|
snippet: needle.to_string(),
|
|
});
|
|
}
|
|
}
|
|
out
|
|
}
|
|
|
|
fn line_of_offset(content: &str, byte_off: usize) -> usize {
|
|
content[..byte_off.min(content.len())].matches('\n').count() + 1
|
|
}
|
|
|
|
fn truncate(s: &str, max: usize) -> String {
|
|
if s.chars().count() <= max {
|
|
return s.to_string();
|
|
}
|
|
let mut out: String = s.chars().take(max).collect();
|
|
out.push('…');
|
|
out
|
|
}
|
|
|
|
// Tests moved to tests/injection_guard_unit.rs (Constructor Pattern: src
|
|
// stays under 200 LOC; integration tests reach via kei_memory::injection_guard).
|