KeiSeiKit-1.0/_primitives/_rust/kei-memory/src/injection_guard.rs
Parfii-bot 902fb3e81a feat(kei-memory): functional schema fix + 4-wave architecture refactor
Wave A — Functional ingest fix (root cause of empty Sleep reports):
- Rewrote TraceLine struct to match real Claude Code trace JSONL:
  type (was kind), timestamp ISO8601 (was epoch ts), message Object,
  cwd / gitBranch / parentUuid / uuid / subtype / toolUseID / toolUseResult
- New src/extract.rs: extract_tool_uses + extract_tool_result walks
  message.content[] for nested tool_use / tool_result blocks
- New src/classifier.rs: explicit table classifier (tool_error, user_correction,
  retry_loop, permission_denied, tool_use:<name>, ...) replaces shallow heuristic
- New src/error.rs: KeiMemoryError enum (IO/Parse/Db) replaces semantic
  mismatch where IO error was wrapped as rusqlite::InvalidParameterName
- New src/trace_line.rs: TraceLine + helpers (cube extraction)
- Schema migration v3: events.cwd column + 3 hot-query indices
  (events.tool, events.file_path, events.ts) + UNIQUE on patterns
- New tests/ingest_real_trace.rs: synth-fixture asserts tool/file/cwd/class extraction

Wave B — Lib crate split:
- Cargo.toml: [lib] target added alongside existing [[bin]]
- src/lib.rs: pub re-export of all 18 modules
- src/main.rs: 11 mod declarations replaced by single use kei_memory::{…}
- tests/integration.rs: #[path] hack replaced by use kei_memory::{…}

Wave C — TF-IDF dedup + single-JOIN + filter_map fix:
- Schema migration v2: tokens.idf_dirty column + flag-based dedup
- index_document no longer triggers per-call recompute_idf rebuild
- top_similar uses single JOIN via vectors_for_overlapping_sessions helper
  (was N round-trips, one session_vector per candidate)
- All filter_map(|r| r.ok()) row-error swallowing replaced with ? propagation
- New tests/tfidf_idf_dedup.rs: 4 tests covering dedup behaviour, IDF emptiness,
  JOIN-pruning, empty-query safety

Wave D — Commands split + nits:
- New src/dump.rs (43 LOC) + src/stats.rs (33 LOC):
  CLI renderers extracted from commands.rs (was inline SQL + format)
- src/commands.rs: thin wrappers, -42 LOC
- src/injection_guard.rs: inline tests removed (-26 LOC), file under 200 LOC threshold
- tests/injection_guard_unit.rs (new): 4 tests in proper integration crate
- src/patterns.rs: INSERT replaced with INSERT...ON CONFLICT...DO UPDATE
  (idempotent re-ingest, uses Wave A's UNIQUE index)
- src/analyze.rs + src/coaccess.rs: filter_map row-error fixes
- src/coaccess.rs: misleading PK comment rewritten

Verify-before-commit (RULE 0.13 §"Verify-before-commit"):
- cargo check --all-targets: PASS (1 unrelated dead-code warning)
- cargo test: 42 passed, 0 failed across 9 test binaries
- STATUS-TRUTH markers aggregated at .claude/agents/_merge/kei-memory-2026-05-01/

Architect-spotted ARCH-MAJOR + ARCH-MINOR + ARCH-NIT findings addressed:
- ARCH-MAJOR Cargo.toml binary-only (Wave B)
- ARCH-MAJOR schema missing indices (Wave A v3)
- ARCH-MAJOR ingest_jsonl choke point (Wave A — extract.rs + classifier.rs)
- ARCH-MAJOR idf O(N·V) per-call rebuild (Wave C)
- ARCH-MINOR patterns no UPSERT (Wave D)
- ARCH-MINOR commands.rs houses dump+stats (Wave D)
- ARCH-MINOR classifier silent contract (Wave A)
- ARCH-MINOR IO error wrapped as rusqlite (Wave A)
- ARCH-MINOR injection_guard inline tests (Wave D)
- ARCH-MINOR tfidf top_similar N round-trips (Wave C)
- ARCH-NIT 3× filter_map(|r| r.ok()) sites (Wave C + D)
- ARCH-NIT coaccess misleading comment (Wave D)

=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
cargo-test: PASS (42 tests, 0 failures)
behaviour-verified: yes
follow-up-required:
  - tests/ingest_guard_tests.rs + tests/guard_test_corpus.rs still on #[path] hack (Wave B follow-up note, ~5 LOC)
  - dead_code warning Severity::Warn unused (pre-existing, not blocking)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:10:06 +08:00

171 lines
5.8 KiB
Rust

//! Injection / exfiltration guard for memory entries.
//!
//! Constructor Pattern: scan logic only; pattern definitions live in
//! `injection_patterns.rs`.
//!
//! ## Wire-points (3 paths protected, P2.1.b lock 2026-04-28)
//!
//! 1. `ingest::insert_event` — REAL memory writes from agent JSONL
//! transcripts. Each event message is scanned before it is persisted
//! into the `events` table. Block-tier hits short-circuit insertion.
//! 2. `kei-pet::memory::record_interaction` — user-facing pet
//! conversation memory. Uses a substring/char-only sibling guard
//! (`kei_pet::injection_check`) to avoid a regex dep bump on the pet
//! crate. Block-tier coverage mirrors this module's prompt-override
//! + invisible-unicode + PEM-marker rules.
//! 3. `cmd_backlog --add` — RULE 0.14 audit-CRUD. Backlog items are
//! rendered into self-audit reports verbatim; malicious content
//! survives that path the same way it would survive insert_event.
//!
//! All three paths use the same `Severity::Block` semantics: a hit
//! results in early-return / persistence-skip, with the finding logged.
//!
//! ## Rationale
//!
//! Memory entries are injected verbatim into the system prompt. Any
//! prompt-override fragment, role-prefix, ChatML tag, invisible bidi
//! codepoint, hardcoded credential, or large base64 attestation blob
//! survives that injection and becomes effective text the model reads.
//! The scan treats these as untrusted input and rejects them.
//!
//! ## Bypass
//!
//! `KEI_MEMORY_SKIP_GUARD=1` skips the scan after logging an explicit
//! warning to stderr. Intended for one-off recovery — never the default.
use crate::injection_patterns::{regex_patterns, substring_patterns, Severity, INVISIBLE_CHARS};
/// One pattern hit. Severity drives whether the call site rejects.
#[derive(Debug, Clone)]
pub struct InjectionFinding {
pub pattern: String,
pub line: usize,
pub severity: Severity,
pub source: String,
pub snippet: String,
}
impl std::fmt::Display for InjectionFinding {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"[{:?}] {} ({}) at line {}: {}",
self.severity, self.pattern, self.source, self.line, self.snippet
)
}
}
impl std::error::Error for InjectionFinding {}
/// Scan `content` for prompt-injection / secret-leak patterns.
///
/// Returns `Ok(())` when the content is clean OR contains only
/// `Warn`-level findings. Returns `Err(InjectionFinding)` on the first
/// `Block`-level hit. Use `scan_all` if all findings are wanted.
pub fn scan(content: &str) -> Result<(), InjectionFinding> {
if std::env::var("KEI_MEMORY_SKIP_GUARD").as_deref() == Ok("1") {
eprintln!(
"kei-memory: WARNING — injection guard bypassed via \
KEI_MEMORY_SKIP_GUARD=1 (RULE 0.4 audit-trail)"
);
return Ok(());
}
if let Some(f) = check_invisible(content) {
return Err(f);
}
for f in scan_regex(content) {
if f.severity == Severity::Block {
return Err(f);
}
}
for f in scan_substring(content) {
if f.severity == Severity::Block {
return Err(f);
}
}
Ok(())
}
/// Like `scan` but returns every finding (Block + Warn). Useful for
/// diagnostics / triage tools that want the full picture.
#[allow(dead_code)]
pub fn scan_all(content: &str) -> Vec<InjectionFinding> {
let mut out = Vec::new();
if let Some(f) = check_invisible(content) {
out.push(f);
}
out.extend(scan_regex(content));
out.extend(scan_substring(content));
out
}
fn check_invisible(content: &str) -> Option<InjectionFinding> {
for (idx, line) in content.lines().enumerate() {
for ch in line.chars() {
if INVISIBLE_CHARS.contains(&ch) {
return Some(InjectionFinding {
pattern: "invisible_unicode".to_string(),
line: idx + 1,
severity: Severity::Block,
source: "unicode:bidi".to_string(),
snippet: format!("U+{:04X}", ch as u32),
});
}
}
}
None
}
fn scan_regex(content: &str) -> Vec<InjectionFinding> {
let mut out = Vec::new();
for p in regex_patterns() {
if let Some(m) = p.re.find(content) {
out.push(InjectionFinding {
pattern: p.id.to_string(),
line: line_of_offset(content, m.start()),
severity: p.severity,
source: p.source.to_string(),
snippet: truncate(m.as_str(), 60),
});
}
}
out
}
fn scan_substring(content: &str) -> Vec<InjectionFinding> {
let lower = content.to_lowercase();
let mut out = Vec::new();
for p in substring_patterns() {
if p.needles.iter().all(|n| lower.contains(n)) {
let needle = p.needles[0];
let line = lower
.find(needle)
.map(|off| line_of_offset(&lower, off))
.unwrap_or(1);
out.push(InjectionFinding {
pattern: p.id.to_string(),
line,
severity: p.severity,
source: p.source.to_string(),
snippet: needle.to_string(),
});
}
}
out
}
fn line_of_offset(content: &str, byte_off: usize) -> usize {
content[..byte_off.min(content.len())].matches('\n').count() + 1
}
fn truncate(s: &str, max: usize) -> String {
if s.chars().count() <= max {
return s.to_string();
}
let mut out: String = s.chars().take(max).collect();
out.push('…');
out
}
// Tests moved to tests/injection_guard_unit.rs (Constructor Pattern: src
// stays under 200 LOC; integration tests reach via kei_memory::injection_guard).