KeiSeiKit-1.0/_primitives/_rust/kei-graph-stream/src/tail.rs
Parfii-bot 52a02dfbff feat(live-graph): WebSocket activity stream — orchestrator-centric live view
User pushback: "транслирует в онлайне какие агенты создаются? основное
окно агента, а дальше при запусках появляются новые ветки, мы показываем
в онлайне как агенты собираются и работают"

Earlier `kei-graph-export` rendered the static SUBSTRATE (all 581 atoms,
catalog-style). User wanted the LIFECYCLE: orchestrator at center, every
new agent as a fading-in branch, every tool call as a pulse, every
completion as a fade-out. TTL = until done; pure online, no history
accumulation per user direction.

Three-layer architecture, all conforming to schema /tmp/agent-events-schema.md:

LAYER 1 — Event emitters (4 hooks)
  hooks/agent-event-spawn.sh   PreToolUse:Agent  → agent_spawn event
  hooks/agent-event-done.sh    PostToolUse:Agent → agent_done event
                               (parses STATUS-TRUTH MARKER for outcome,
                                computes cost_usd from token×pricing table)
  hooks/tool-use-event.sh      PreToolUse:Bash|Read|Edit|Write|Grep|Glob|NotebookEdit
                               → tool_use event
  hooks/skill-record.sh        EXTENDED — second emit step writes skill_use
                               event in addition to existing kei-ledger
                               record-skill call

  All 4 are POSIX /bin/sh, defensive (never block, exit 0), bypass via
  KEI_EVENTS_BYPASS=1. Append-only JSONL to
  ~/.claude/memory/agent-events.jsonl.

  Smoke: 4 synthetic invocations cover spawn/done/tool/filter cases.

LAYER 2 — kei-graph-stream Rust daemon
  _primitives/_rust/kei-graph-stream/  (~480 LOC, 5 files + 1 test)

  - Tails events.jsonl every 200ms (poll-based, no notify dep).
  - Parses each event, updates AliveState (insert on spawn, remove on done).
  - Broadcasts {"type":"event","data":<event>} to all WebSocket clients.
  - On client connect: sends {"type":"snapshot","alive":[...]} first.
  - Heartbeat: {"type":"ping"} every 30s.
  - axum 0.7 + ws feature (already in Cargo.lock via kei-cortex).
  - Bypass: KEI_GRAPH_STREAM_BYPASS=1.

  Bound to 127.0.0.1:8201 (loopback only). Endpoints:
    GET /stream  → WebSocket upgrade
    GET /health  → "kei-graph-stream alive"

  4 unit + 1 integration test. cargo build clean.

  Installed binary: ~/.cargo/bin/kei-graph-stream
  Launchd plist: io.keisei.graph-stream (RunAtLoad, KeepAlive)
  Loaded as PID 52678, /health 200 OK verified.

LAYER 3 — live-graph.html (single-file frontend)
  ~/Projects/lbm-graph-viz/live-graph.html  (~464 LOC, self-contained)

  - SVG full-viewport, dark #0f172a, CSS grid background.
  - Pinned center node "main" (orchestrator), gold #fbbf24, glowing.
  - Agents radiate via D3 force-simulation; color-by-model
    (sonnet=green, opus=red, haiku=blue, default=gray).
  - On agent_spawn: fade-in 300ms, edge from main to new node.
  - On tool_use: pulse on agent node (r 8→12→8 over 400ms) +
    floating tool name label fades 800ms.
  - On agent_done: outcome-color flash → fade-out 800ms → remove.
  - WebSocket client: ws://127.0.0.1:8201/stream, exponential-backoff
    reconnect (1s→30s).
  - Top-right status badge: ● connected | ○ reconnecting | ✕ disconnected.
  - Bottom counters: alive / spawned / tool calls / done / last event age.
  - No build step. D3 v7 from CDN. Pure HTML+JS+CSS.

End-to-end smoke (this machine, just now):
  - daemon health 200 OK
  - hook injected agent_spawn → daemon broadcasts → AliveState=1
  - hook injected agent_done  → daemon broadcasts → AliveState=0
  - frontend file syntax-checked clean

What this does NOT do (deferred, by user direction "это онлайн"):
  - History persistence — agents who finished are GONE from the graph.
    Per-session log remains in events.jsonl + sleep-sync if user wants
    to consult later, but the live view is RIGHT NOW only.
  - Sub-agent attribution beyond "main" — orchestrator-direct tool calls
    show on the orchestrator node. Sub-agent's internal tool calls would
    need session-id correlation; current schema has agent_id="main"
    placeholder for non-Agent tool calls.
  - Replay mode — no time-scrubber. Possible follow-up if useful.
  - Auth on WebSocket — bound to 127.0.0.1 only. Local-only by design.

=== STATUS-TRUTH MARKER ===
shipped: functional
stubs: 0
cargo-check: PASS
behaviour-verified: yes
follow-up-required:
  - Sub-agent tool-call attribution (correlate session_id chain)
  - Replay mode with time scrubber (if user finds use)
  - Tool aggregator nodes ("Bash bucket" with N) instead of per-agent pulses

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 13:30:24 +08:00

135 lines
3.8 KiB
Rust

use anyhow::Result;
use std::path::PathBuf;
use std::sync::Arc;
use tokio::io::{AsyncBufReadExt, AsyncSeekExt, BufReader};
use tokio::sync::broadcast;
use tokio::time::{Duration, sleep};
use crate::state::AliveState;
const POLL_INTERVAL: Duration = Duration::from_millis(200);
/// Continuously tail `path`, parse events, update alive state, broadcast.
pub async fn run(
path: PathBuf,
tx: Arc<broadcast::Sender<String>>,
alive: Arc<AliveState>,
) -> Result<()> {
let mut file = tokio::fs::File::open(&path).await?;
// Seek to end — no history replay.
let initial_len = file.seek(tokio::io::SeekFrom::End(0)).await?;
let mut cursor = initial_len;
loop {
sleep(POLL_INTERVAL).await;
let meta = match tokio::fs::metadata(&path).await {
Ok(m) => m,
Err(_) => continue,
};
let current_len = meta.len();
if current_len < cursor {
// File was rotated/truncated — reopen and reset.
file = tokio::fs::File::open(&path).await?;
cursor = 0;
}
if current_len == cursor {
continue;
}
// Read new bytes from cursor.
file.seek(tokio::io::SeekFrom::Start(cursor)).await?;
let mut reader = BufReader::new(&mut file);
let mut lines_read: u64 = 0;
let mut line = String::new();
loop {
line.clear();
let n = reader.read_line(&mut line).await?;
if n == 0 {
break;
}
lines_read += n as u64;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
process_line(trimmed, &tx, &alive);
}
cursor += lines_read;
}
}
fn process_line(
line: &str,
tx: &broadcast::Sender<String>,
alive: &AliveState,
) {
let Ok(event) = serde_json::from_str::<serde_json::Value>(line) else {
return;
};
match event["event"].as_str() {
Some("agent_spawn") => alive.insert(&event),
Some("agent_done") => alive.remove(&event),
_ => {}
}
let frame = match serde_json::to_string(&serde_json::json!({
"type": "event",
"data": &event,
})) {
Ok(s) => s,
Err(_) => return,
};
// Ignore send errors (no subscribers yet is fine).
let _ = tx.send(frame);
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use tempfile::NamedTempFile;
use std::io::Write;
#[tokio::test]
async fn tail_detects_new_lines() {
let mut tmp = NamedTempFile::new().unwrap();
let path = PathBuf::from(tmp.path());
let (tx, mut rx) = broadcast::channel::<String>(16);
let tx = Arc::new(tx);
let alive = Arc::new(AliveState::new());
// Spawn tail task (will seek to EOF of empty file → cursor=0).
let path2 = path.clone();
let tx2 = Arc::clone(&tx);
let alive2 = Arc::clone(&alive);
tokio::spawn(async move { run(path2, tx2, alive2).await });
// Wait for first poll cycle.
tokio::time::sleep(Duration::from_millis(50)).await;
// Append a spawn event.
let ev = json!({"ts":"2026-05-02T13:00:00Z","event":"agent_spawn","id":"t1","subagent_type":"researcher","model":"sonnet","prompt_preview":"test"});
writeln!(tmp, "{}", ev.to_string()).unwrap();
// Allow poll to pick it up.
tokio::time::sleep(Duration::from_millis(400)).await;
let msg = rx.recv().await.unwrap();
let parsed: serde_json::Value = serde_json::from_str(&msg).unwrap();
assert_eq!(parsed["type"], "event");
assert_eq!(parsed["data"]["event"], "agent_spawn");
// Alive state should contain t1.
let snap = alive.snapshot();
assert_eq!(snap.len(), 1);
assert_eq!(snap[0].id, "t1");
}
}