keisei/KeiSeiKit-1.0
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
388 commits 3 branches 11 tags 12 MiB
Commit graph

12 commits

Author SHA1 Message Date
Parfii-bot
9622d41bb6 refactor(wave17): cleanup — kei-shared SSoT + MEDIUM audit residuals + docs drift 
47 crates, 771 tests green (up from 753 at v0.33.0). Zero new
features — pure hygiene.

## kei-shared extract (SSoT for DNA format)

New crate `kei-shared` consolidates DNA-parse logic that was duplicated
across kei-agent-runtime + kei-dna-index. Both consumers migrated to
import ParsedDna / parse_dna / is_hex8 from kei_shared.

- 12 tests (10 integration + 2 unit)
- kei-dna-index LOC reduction: -60 in parsed.rs (body replaced by wrapper)
- kei-agent-runtime preserves lenient DnaError (legacy 4-hex parse path)
- Format-string SSoT: kei_shared::compose_dna is sole source

## MEDIUM audit residuals closed (kei-entity-store)

A. DDL panic coverage — verified exhaustive match across all 12
   FieldKind variants; new test ddl_never_panics_on_any_fieldkind
   compile-time-breaks if a variant added without test update.
B. Update FTS reindex invariant — doc + new update_invariant.rs module
   with debug_assert validating non-input FTS columns don't drift
   pre/post UPDATE. Zero release-mode cost (cfg-gated).
C. WAL fallback — wal_pragma_fallback_keeps_store_usable test (cfg(unix))
   verifies read-only-parent dir doesn't brick Store::open.
D. Search Unicode edge cases — 4 new tests (punctuation, emoji,
   zero-width, mixed RTL). has_searchable_token already correct, no
   source change needed; tests pin current behavior.

Added: residual_audit_smoke.rs (8 tests), update_invariant.rs module.
kei-entity-store: 57 → 65 tests.

## Docs drift fixed (count claims → reality)

- README.md: "36 crates → 47 crates", "500+ tests → 800+ tests"
- PLUGIN.md, docs/INSTALL.md, docs/REFERENCE.md, docs/SUBSTRATE-SCHEMA.md
  all synced to real counts.
- CHANGELOG.md: 6 new version blocks (v0.28 → v0.33) consolidated
  in existing style.
- Historical snapshots (HANDOFF-WAKE v0.29, CONVERGENCE-PLAN, etc)
  deliberately preserved — they're version-scoped, not drift.

## Known deviation from task spec

kei-shared's [workspace] table was dropped (Cargo rejected "multiple
workspace roots" when parent workspace pulls via path dep). Crate
registered in workspace.members instead. Verified cargo check + test
clean in both modes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 20:34:43 +08:00
Parfii-bot
0ea429054f feat(wave14): 5 bio-inspired primitives + phase2 cleanup + substrate dogfood 
## Wave 14 — 5 new primitives (44 crates total, 713 tests green)

All specs written as task.toml → passed through kei-agent-runtime prepare
→ composed prompts via capability fragments → Agent tool invocation.
First fully-dogfooded wave.

- kei-prune (9 tests): biological pruning. `candidates(idle_days)` +
  `mark_retired(id)` on sidecar `prune_retirements` table (agents.status
  CHECK precluded 'retired' value).
- kei-discover (8 tests): federated marketplace discovery stub. UNIQUE
  slug via custom migration + FTS5 on slug+description. Engine-native
  via kei-entity-store. Typed DuplicateSlug error.
- kei-brain-view (6-8 tests): stdout visualizer for ledger taxonomy
  graph + agent lineage. Tree / stats / lineage subcommands. NO_COLOR
  env respected. No kei-entity-store dep (direct rusqlite).
- kei-hibernate (6 tests): whole-brain tar.zst export/import. Manifest
  with sha256 per-file, version gate, safe_join on extract, dry-run
  mode. tar 0.4 + zstd 0.13.
- kei-ledger-sign (7 tests): ed25519 creator attestation. keygen / sign /
  verify CLI. Canonical message `dna|spec_sha|creator_id` with pipe
  rejection. chmod 600 on key storage (unix). Tamper-detection on load
  via pubkey re-derivation.

## Phase 2 cleanup shipped in same commit

- LOC splits: walk.rs 221→91 (path_safety.rs + wikilink.rs extracted),
  prepare.rs 228→199 (dead build_ledger_row removed, fn helpers split).
- Clippy pass: 6 warnings fixed (derivable_impls, manual_contains,
  type_complexity x2, doc_overindented_list_items x2) in
  kei-entity-store, kei-ledger, kei-spawn.
- DNA eprintln removed from kei-agent-runtime/src/dna.rs (stderr
  pollution from library parse).
- kei-pipe integrations: hot_reload.rs (kei-watch wrapper, sync API,
  50ms debounce) + scheduler_bridge.rs (kei-scheduler executor, shell
  exec documented). +6 tests.
- Workspace [workspace.dependencies] centralised: rusqlite/chrono/
  anyhow/thiserror/tempfile/toml — future crates opt in via
  `.workspace = true`. Existing pins preserved.

## Substrate dogfood verified

task.toml → `kei-agent-runtime prepare` → DNA + composed prompt from
capability fragments → Agent tool invocation. kei-spawn also tested
end-to-end (prompt.md written to tasks/<agent-id>/, ledger row created).

Verified: cargo check --workspace clean, 713 tests passing,
substrate_integration.sh ✓, hook_wiring_integration.sh ✓.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 17:19:25 +08:00
Parfii-bot
c1556f505a fix: Wave 13 cleanup — HttpDriver + agent_id validator + safe_join + 4 MEDIUM 
Closes the remaining v0.29.0 follow-ups + post-audit MEDIUMs.

## HttpDriver (kei-spawn http-driver feature)
- Real reqwest::blocking POST to api.anthropic.com/v1/messages
- Feature flag `http-driver = ["dep:reqwest"]` (default off, zero breaking)
- KEI_ANTHROPIC_KEY read at invoke time (rotation-friendly)
- 5 httpmock tests (missing key, 200, 4xx, 5xx, malformed json)
- Endpoint override via KEI_ANTHROPIC_ENDPOINT env for tests
- Files: drive.rs, drive_http.rs (new), drive_http_parse.rs (new), tests/http_driver.rs

## agent_id path-traversal validator (HIGH)
- New validate.rs with validate_agent_id() — whitelist grammar, 64-char cap,
  rejects /, \, .., leading dot/dash, NUL, :, whitespace, non-ASCII,
  Windows-reserved (CON/PRN/AUX/NUL/COM1-9/LPT1-9)
- Wired into all 5 agent_id→path sinks: load_task, resolve_agent_id,
  prepare, simulated_merge, verify_task
- autogen_agent_id moved to validate.rs with slugify_role helper —
  output passes validator by construction (100-draw property test)
- 33 new tests in agent_id_validator.rs

## safe_join symlink escape (MEDIUM)
- Base must canonicalize (nonexistent → Canonicalize error)
- Joined must start_with base_canon OR joined.parent() must start_with base_canon
- Blocks symlink-to-outside-base with non-existent tail file
- walk.rs refactored into 5 ≤17-LOC helpers
- 7 new tests in safe_join_hardening.rs

## entity-store 4 MEDIUM fixes
- ddl.rs: panic on unsupported FieldKind → typed DdlError::UnsupportedExtraColumn
  propagated through Store::open as VerbError::InvalidInput (exit 2).
  Extracted ddl_edge.rs + ddl_error.rs modules. Backward-compat shim preserved.
- search.rs: FTS5 empty-tokenization → typed InvalidInput on queries with
  no alphanumeric tokens (was opaque rusqlite error). Unicode-aware via
  char::is_alphanumeric.
- engine.rs: WAL pragma failure now logged to stderr with path + rusqlite
  source; fallback to rollback journal preserved (exit-code contract intact).
- bug_fixes_smoke: added fts5_phrase_quoting_preserves_legitimate_queries —
  catches over-broad sanitizer that passes injection test alone.

## Verified
- cargo check --workspace clean (both with and without http-driver feature)
- cargo test --workspace: 668 tests green (up from 620)
- substrate_integration.sh ✓, hook_wiring_integration.sh ✓

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 16:16:24 +08:00
Parfii-bot
0706733abf fix(entity-store): FTS delete-tx + archive FTS desync (HIGH audit) 
2 HIGH security/correctness fixes from post-v0.28 audit:

- delete.rs: hard-delete now wraps FTS DELETE + table DELETE in
  unchecked_transaction. Prior: two separate execute calls could
  leave FTS orphan on second-stmt failure. Same pattern as C2 fix.

- archive.rs: archive-UPDATE + FTS DELETE now atomic in one tx.
  Prior: archived rows remained searchable (UX/privacy leak).
  Semantics documented: archive = hidden from FTS; unarchive must
  re-insert (caller responsibility).

+4 regression tests: delete_rollback_on_fts_sabotage,
archive_removes_from_fts, archive_rollback_on_sabotage,
delete_succeeds_when_no_fts_configured. All green (49/49 kes tests).

Constructor Pattern: all files <200 LOC, all functions <30 LOC.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 15:45:23 +08:00
Parfii-bot
c439a01961 feat(a): Store::open multi-schema — kei-chat-store sessions fully engine-owned 
Breaking API change: Store::open(path, schemas: &[&EntitySchema])
replaces single-schema variant. All 6 callers migrated in same commit.

Engine changes:
- engine::run_migrations wraps ALL schemas in one unchecked_transaction
- Cross-schema atomicity: failing migration in schema[N] rolls back
  schema[0..N-1] too. Verified via new failing_migration_rolls_back_
  prior_schemas test.

Sister crates (5): trivial slice wrap in store.rs — 6 sites each,
~2 LOC delta.

kei-chat-store MAJOR change:
- Split CHAT_SCHEMA → MESSAGES_SCHEMA (INTEGER PK, FTS) + SESSIONS_SCHEMA
  (TextPk UUID, TextArchiveEnum status 'active'|'archived', status_at +
  message_count/total_tokens/total_cost aggregate cols)
- Drop chat_sessions DDL from custom_migrations (engine owns now)
- start_session → v_create::run(&SESSIONS_SCHEMA)
- archive_session → v_archive::run (writes 'archived' sentinel +
  stamps status_at via TextArchiveEnum mode)
- get_session → v_get::run
- bump_session_totals stays bespoke (per-message aggregate UPDATE not
  a generic CRUD op)

Substrate coverage kei-chat-store: ~60% → 100% (messages + sessions
both engine-owned).

Tests: 45 entity-store (was 42, +3 multi_schema_smoke: two-schema
creates, verbs dispatch per schema, rollback atomicity).
All sister crates preserved: task 9, chat 6, content 4, social 5,
crossdomain 5, sage 36.

Closes HANDOFF-WAKE's final architectural gap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 14:27:15 +08:00
Parfii-bot
ec205d5ee5 feat(w10a): engine TextPairWithMetadata extra_columns + kei-crossdomain re-migrated 
EdgeKeyKind::TextPairWithMetadata extended with:
- from_col / to_col (custom column names, default src_path/dst_path)
- extra_columns: &[(name, FieldKind)] for domain-specific edge metadata

kei-crossdomain fully re-migrated via engine:
- edge_table: Some('cross_edges') + TextPairWithMetadata variant with
  from_col='from_uri', to_col='to_uri', has_id/has_weight/has_created_at,
  extra_columns=[evidence, metadata]
- Custom edges DDL dropped from custom_migrations (engine owns it now)
- edges.rs query_edges SELECT uses edge_id (engine-emitted PK)

Tests: 42/42 kei-entity-store (+2), 5/5 kei-crossdomain preserved.
Sister crates (task/chat/content/social/sage) no regression.

Closes HANDOFF-WAKE deferred item #2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 13:59:06 +08:00
Parfii-bot
eac09a6354 feat(e1): engine improvements — TextPk + Real + TextArchiveEnum + TextPairWithMetadata 
4 additive FieldKind/EdgeKeyKind variants addressing M1 dogfood +
M4/M5 flags. Backward-compat — existing schemas unchanged.

FieldKind::TextPk — TEXT PRIMARY KEY (kei-chat-store UUID sessions)
FieldKind::Real + RealDefault(f64) — REAL columns (kei-chat-store cost)
FieldKind::TextArchiveEnum — archive verb writes string sentinel instead of 1
EdgeKeyKind::TextPairWithMetadata — src_path/dst_path + id/weight/created_at/extra

Archive verb now dispatches: IntegerFlag writes 1, TextArchiveEnum writes sentinel.
Link/rank handle TextPairWithMetadata via generic weight+id propagation.
PK helpers in verbs/pk.rs abstract integer vs text primary key.

Engine decomposed to stay under 200 LOC:
- schema.rs (149) + field.rs (94, new) + ddl.rs (157, new)
- verbs/pk.rs + create_defaults.rs split from create.rs

Tests: 40/40 (was 32, +8 real_text_pk_smoke).
Sister crates verified backward-compat:
- kei-task 9/9, kei-chat-store 5/5, kei-content-store 4/4
- kei-social-store 5/5, kei-crossdomain 5/5, kei-sage 28/28

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 10:22:22 +08:00
Parfii-bot
68e37ecf68 fix(a1-integration): add EdgeKeyKind + archived_field defaults to bug_fixes_smoke fixture 
A1's test fixture created before B5 merged new fields onto EntitySchema.
Cross-wave fixup — non-functional change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 05:33:21 +08:00
Parfii-bot
70867a7fd0 Merge fix/b5-textpair — EdgeKeyKind + archive verb 2026-04-23 05:31:03 +08:00
Parfii-bot
2d2c9881de feat(entity-store/b5): EdgeKeyKind::TextPair + archive verb — unblock kei-sage + kei-chat-store migration 
schema.rs: EdgeKeyKind enum (IntegerPair default, TextPair) + archived_field:
Option<&'static str> on EntitySchema. Backward-compat via Default impl.

engine.rs: ddl_edge_table_for() dispatches integer vs text edge DDL.
Existing ddl_edge_table() untouched; new ddl_edge_table_text() adds
src_path/dst_path TEXT columns.

verbs/link.rs (rewrite): dispatches on edge_key_kind. Input JSON
{from:int, to:int} for IntegerPair OR {from:str, to:str} for TextPair.

verbs/rank.rs (rewrite): generic pagerank<K: Hash + Eq + Clone> over
node key type. Same algorithm, polymorphic in key.

verbs/archive.rs (new, 64 LOC): soft-delete via UPDATE <tbl> SET
<archived_field>=1 + optional <archived_field>_at=now(). Schema
without archived_field declared returns VerbError.

Tests: 20/20 kei-entity-store (was 10, +10: 5 text_pair + 5 archive).
kei-task 9/9 preserved (IntegerPair still default, backward-compat
verified).

Enables: kei-sage migration (TextPair edges) + kei-chat-store
migration (archive).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 05:30:33 +08:00
Parfii-bot
02e4b25af2 fix(entity-store/a1): C1 type coercion + C2 FTS transaction + FTS5 injection + M3 SQL escape + TEXT cap + M2 migration version 
6 critical/high bugs from post-convergence audit fixed in one pass.

New src/verbs/validate.rs (95 LOC) — shared typed validator:
- coerce() returns VerbError::InvalidType{field,expected,got} on wrong-kind JSON
- Preserves "missing key → default" semantics (additive, not breaking)
- MAX_TEXT_BYTES 64 KiB cap enforced on all text fields (M2 audit)

verbs/create.rs + update.rs (C1 + C2):
- Typed validation replaces silent 0/""/empty coercion
- insert_tx() + update_tx() wrap INSERT + FTS DELETE+INSERT in
  conn.unchecked_transaction(); mid-flight failure rolls back both
  main row + FTS sidecar together (no orphan FTS rows)

verbs/search.rs — fts5_quote() defends FTS5 syntax injection:
- User query wrapped in "..." phrase quotes, internal " doubled
- Column-prefix `title:secret`, NEAR(), wildcards become literal
- Integration + unit tests

engine.rs:
- ddl_column() escapes ' → '' on TextDefault values (M3)
- apply_user_version() stamps CURRENT_USER_VERSION=1 via PRAGMA;
  idempotent (won't downgrade). Opens migration path (M2).

Tests: 22/22 (was 10, +12). kei-task 9/9 preserved — no regression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 05:27:59 +08:00
Parfii-bot
793b91bc43 feat(convergence/p1): kei-entity-store engine + schema fragments + kei-task pilot 
Layer A + B of convergence wave. Extract common SQLite-CRUD + graph
logic into kei-entity-store engine; introduce JSON Schema fragments;
pilot-migrate kei-task to verify parity.

New crate _primitives/_rust/kei-entity-store/ (1151 LOC):
- src/schema.rs — EntitySchema + FieldDef + enabled_verbs + fts_columns
  + edge_table + custom_migrations
- src/engine.rs — Store::open with WAL pragma + migration runner
- src/verbs/ — 8 data-driven verb modules (create/get/list/search/
  update/delete/link/rank) uniform JSON-in/JSON-out signature
- src/error.rs — typed VerbError enum
- tests/verb_smoke.rs — 10/10 green

New _schemas/fragments/ (83 LOC JSON):
- entity-base.json, titled.json, titled-content.json, edge.json

kei-task pilot migration:
- TASK_SCHEMA: EntitySchema static (67 LOC, was 58)
- store.rs becomes thin shim over engine::Store
- atoms/create.rs + atoms/search.rs delegate to engine verbs
- atoms/schemas/*.json use $ref to _schemas/fragments/ (DRY)
- Task-specific secondary tables (milestones, task_deps) stay via
  schema.custom_migrations; cycle-detection in deps.rs stays
  hand-rolled (domain logic, not generic CRUD)
- 9/9 tests green — full behavioural parity

Convergence delta:
- kei-task touched files: 342 → 389 LOC (+47 for JSON marshalling
  boundary; net wash on pilot)
- BUT each remaining 5 sibling crate can shrink ~400-500 LOC on migration
- Expected total reduction when all 6 migrated: ~2500 LOC across the cluster

Follow-ups declared:
- Migrate kei-chat-store, kei-content-store, kei-social-store to engine
- Migrate kei-sage (needs string-id edge variant; currently generic
  link/rank assume int ids)
- Migrate kei-crossdomain
- Expose list/delete atoms in kei-task (engine supports, atoms not yet)
- Fold kei-curator as engine::hygiene module (per P4 audit)
- Fold kei-search-core entities, keep workflow as thin kei-search-pipeline

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 04:46:47 +08:00