Merge branch 'feat/v0.6-database' — 5 blocks + kei-migrate Rust + /schema-design

This commit is contained in:
Parfii-bot 2026-04-21 21:14:50 +08:00
commit f205a12348
25 changed files with 3891 additions and 0 deletions

51
_blocks/db-drizzle.md Normal file
View file

@ -0,0 +1,51 @@
# DB — Drizzle ORM (TypeScript) patterns
Use when the project is TypeScript/Next.js/Bun/Node and needs a type-safe SQL layer without Prisma's heavyweight engine process. Pairs with `stack-nextjs`. [E4 — expert assessment]
**Core versions:** `drizzle-orm` (latest on npm) + `drizzle-kit` (migrations CLI) as of 2026-04. Peer-deps: `pg` for Postgres, `better-sqlite3` / `@libsql/client` for SQLite, `mysql2` for MySQL. [UNVERIFIED: pin exact versions from npm before shipping]
**Schema-first, not code-first:**
```ts
// db/schema.ts
import { pgTable, serial, text, timestamp, integer } from "drizzle-orm/pg-core";
export const users = pgTable("users", {
id: serial("id").primaryKey(),
email: text("email").notNull().unique(),
createdAt: timestamp("created_at").defaultNow().notNull(),
});
export const posts = pgTable("posts", {
id: serial("id").primaryKey(),
authorId: integer("author_id").references(() => users.id).notNull(),
body: text("body").notNull(),
});
```
`schema.ts` IS the source of truth. All types flow from it — `typeof users.$inferSelect` gives you the row type.
**Query with full inference:**
```ts
import { eq } from "drizzle-orm";
const rows = await db.select().from(users).where(eq(users.id, 1));
// rows: { id: number; email: string; createdAt: Date }[]
```
No codegen step, no separate `.prisma` file. Type errors surface in the IDE immediately.
**Migrations via drizzle-kit:**
```bash
drizzle-kit generate # diff schema.ts against prev snapshot → emit SQL in drizzle/
drizzle-kit migrate # apply pending migrations
drizzle-kit studio # local web UI to inspect data
```
Config in `drizzle.config.ts` — specify `dialect`, `schema`, `out`, `dbCredentials`.
**Connection / pool:**
```ts
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 20 });
export const db = drizzle(pool, { schema });
```
Serverless (Vercel / CF Workers): use `neon-serverless` or `@libsql/client` driver instead — the `pg` Pool doesn't survive cold-start boundaries.
**Forbidden:** template-string SQL with untrusted input (`sql\`SELECT * WHERE x = ${userInput}\`` — use `sql.placeholder` or the query builder); committing `drizzle/meta/_journal.json` conflicts (merge manually or regenerate); mixing drizzle-kit versions across dev machines.

View file

@ -0,0 +1,33 @@
# DB — Migration hygiene (universal)
Applies to every migration tool — `kei-migrate`, Atlas, goose, sqlx-cli, drizzle-kit, Alembic, Prisma migrate, Ecto migrations. [E4 — expert assessment]
**Numbering:** timestamp prefix, not integer. `20260421_120000_add_users_email_index.sql` sorts correctly forever and doesn't collide on parallel branches. Integer sequences (`0001_`, `0002_`) collide on merge; reject them in review.
**Up + down pairs:** every migration has a reverse. If the reverse is destructive and unsafe (e.g. dropping a column with data), write a `-- IRREVERSIBLE` comment and stop the down-script there. NEVER auto-run destructive downs on prod without a human click.
**Idempotent where possible:**
```sql
CREATE TABLE IF NOT EXISTS users (...);
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
ALTER TABLE users ADD COLUMN IF NOT EXISTS bio TEXT; -- PG 9.6+, verify per-DB
```
Re-running a partially-applied migration should be safe. A migration that crashes mid-way and can't be re-run = 2AM incident waiting to happen.
**Zero-downtime pattern (add-then-drop):**
1. Deploy migration that ADDS new column / table (old code still works).
2. Deploy app code that writes BOTH old + new.
3. Backfill old → new.
4. Deploy app code that reads new, ignores old.
5. Deploy migration that DROPS old column.
Never `DROP` + `ADD RENAME` in one migration on a live table. That's a table lock + app-downtime event.
**Backfill patterns:**
- Small table (< 1M rows): `UPDATE ... SET new = f(old)` in a single migration.
- Large table: background job with batched `UPDATE ... WHERE id BETWEEN ? AND ?` + `LIMIT`. Commit per batch. Monitor lag.
- Very large (> 100M rows): use the DB's native tooling (PG `VACUUM FULL` not needed; `pg_repack` if column-add bloats). [UNVERIFIED: verify on current PG docs]
**Tracking table (`_kei_migrations` or equivalent):** stores (version, name, checksum, applied_at). Checksum prevents silent tampering with an already-applied file. If checksum mismatches on an applied migration → hard-fail, demand human intervention.
**Forbidden:** editing a migration file after it's been applied on any environment (checksum break); `DROP TABLE` without backup + 24h cooldown; mixing DDL + large DML in one transaction (long locks); running migrations automatically on app startup in multi-replica deploys without a leader-election guard (every replica tries to apply = race condition).

28
_blocks/db-postgres.md Normal file
View file

@ -0,0 +1,28 @@
# DB — PostgreSQL (current major — 17 as of 2026-04) patterns
Use when the project needs relational integrity, concurrent writes, or server-side indexing power that SQLite can't match. Default RDBMS for new multi-user services. [E4 — expert assessment]
**Version choice:** PostgreSQL 17 for new projects (current GA line, improved vacuum, JSON_TABLE, better parallel index builds). PostgreSQL 16 acceptable if hosting provider pins it. [UNVERIFIED: exact feature matrix — verify on postgresql.org/docs before committing to a minor-version-specific feature]
**Schema migrations:** every schema change ships as a numbered `.sql` file, never `ALTER TABLE` on prod. Use `kei-migrate` (this kit) or Atlas/goose/sqlx-cli — see `db-migration-hygiene.md`. One migration per logical change; no mega-migrations.
**Indexing:**
- B-tree default for equality + range. `CREATE INDEX CONCURRENTLY` on prod to avoid table lock.
- `GIN` for `jsonb` / array / full-text (`tsvector`).
- `BRIN` only for massive append-only time-series (orders of magnitude smaller than B-tree).
- Partial indexes (`WHERE active = true`) for sparse predicates.
- **Verify with `EXPLAIN (ANALYZE, BUFFERS)`** before declaring an index necessary. No blind indexing.
**Connection pooling:** app-side connection pool is NOT enough at scale. Use:
- **PgBouncer** (transaction mode) for most services — battle-tested, low overhead.
- **Supavisor** if already on Supabase — serverless-friendly, wire-compatible. [E4]
- Native server pooling (PG 17's improved but still not a substitute). [UNVERIFIED]
Sizing rule of thumb: `max_connections` on server × 1 pool layer. Don't stack pools (pool → PgBouncer → PG = deadlock risk).
**Backup:**
- Logical: `pg_dump` nightly for schema + data portability.
- Physical: `pg_basebackup` + WAL archiving (`archive_command`) for PITR.
- Managed service (RDS / Supabase / Neon) — verify backup retention in their UI, don't assume.
**Forbidden:** `SELECT *` in hot paths (N+1 + column drift); unindexed FK columns (join explosion); `SERIAL` on new tables — prefer `GENERATED ALWAYS AS IDENTITY` (SQL standard, PG 10+); plaintext passwords in `pg_hba.conf`; committing `.env` with DB URL.

34
_blocks/db-sqlite.md Normal file
View file

@ -0,0 +1,34 @@
# DB — SQLite (prod-suitable) patterns
Use when the workload is read-heavy, single-writer-acceptable, or needs zero-ops embedded storage. SQLite is prod-suitable — Fly.io, Turso, Cloudflare D1, and countless CLI/mobile apps run it in production. [E4 — expert assessment]
**When NOT to use:** high-concurrency write workload (> ~1 writer/sec sustained), multi-region strong consistency, horizontal write scaling. Use Postgres instead.
**WAL mode is mandatory for prod:**
```sql
PRAGMA journal_mode = WAL; -- readers don't block writer, writer doesn't block readers
PRAGMA synchronous = NORMAL; -- durable across app crash, NOT across power loss (use FULL if PSU-risk)
PRAGMA busy_timeout = 5000; -- 5s wait for lock instead of instant SQLITE_BUSY
PRAGMA foreign_keys = ON; -- default OFF in SQLite (!), always enable
PRAGMA temp_store = MEMORY;
```
Apply these on every connection open — they are per-connection, not per-database (except `journal_mode` which persists).
**Distributed patterns:**
- **Turso** (libSQL fork): edge-replicated read replicas with HTTP/WebSocket wire protocol. Primary single-writer, replicas read-only. [E4]
- **LiteFS** (Fly.io): file-system replication, leader-election via Consul. Primary+replicas. [E4]
- **Cloudflare D1**: managed SQLite on edge with their own replication. [UNVERIFIED: current throughput limits]
- **Litestream**: continuous replication to S3/R2 for backup + PITR; single node, not HA.
**Full-text search (FTS5):**
```sql
CREATE VIRTUAL TABLE docs_fts USING fts5(title, body, content=docs, content_rowid=id);
CREATE TRIGGER docs_ai AFTER INSERT ON docs BEGIN
INSERT INTO docs_fts(rowid, title, body) VALUES (new.id, new.title, new.body);
END;
```
FTS5 outperforms bolt-on `LIKE '%x%'` by 100×+ on large text corpora. Native, no extension install.
**Backup:** `sqlite3 db '.backup /path/backup.db'` while app runs (safe with WAL). Or Litestream for continuous.
**Forbidden:** multiple writer processes without a coordination layer; opening the same DB over NFS (lock semantics broken); `DELETE FROM bigtable` without `VACUUM` after (doesn't shrink file); committing the `.db` / `.db-wal` / `.db-shm` files to git.

39
_blocks/db-sqlx.md Normal file
View file

@ -0,0 +1,39 @@
# DB — SQLx (Rust) patterns
Use when the project is Rust and needs a SQL-first (not ORM) query layer with compile-time checking. Pairs with `stack-rust-axum`, `stack-rust-cli`. [E4 — expert assessment]
**Core versions:** `sqlx = "0.8"` (current as of 2026-04) with features `runtime-tokio`, `tls-rustls`, and one of `postgres` / `sqlite` / `mysql`. Never mix `runtime-async-std` and `runtime-tokio` — they clash at link time. [UNVERIFIED: verify latest on crates.io before pinning]
**Compile-time checked queries:**
```rust
let row = sqlx::query!("SELECT id, name FROM users WHERE id = $1", user_id)
.fetch_one(&pool).await?;
```
Requires either:
- `DATABASE_URL` env set during `cargo build` (live DB) — convenient in dev, brittle in CI.
- **Offline mode** (recommended for CI): `cargo sqlx prepare` commits `.sqlx/query-*.json` to the repo, then CI builds with `SQLX_OFFLINE=true` and no DB access.
**Connection pool:**
```rust
let pool = sqlx::postgres::PgPoolOptions::new()
.max_connections(20) // tune to server max_connections / replica count
.acquire_timeout(Duration::from_secs(3))
.connect(&database_url).await?;
```
Single `PgPool` per process, `Arc`-cloned into handlers. Don't open per-request.
**Migrations:**
```rust
sqlx::migrate!("./migrations").run(&pool).await?;
```
Built-in runner reads `YYYYMMDDHHMMSS_<name>.sql` files. For richer UX (up/down, status, create scaffolding) use the `kei-migrate` primitive in this kit.
**Transactions:**
```rust
let mut tx = pool.begin().await?;
sqlx::query!("...").execute(&mut *tx).await?;
sqlx::query!("...").execute(&mut *tx).await?;
tx.commit().await?; // explicit; Drop = rollback
```
**Forbidden:** `sqlx::query` (non-macro) with untrusted input without `bind()` — that's string concat, i.e. SQL injection; `.unwrap()` on DB calls in prod paths; enabling both `runtime-tokio` and `runtime-async-std`; committing a live `DATABASE_URL` to `.env.example`.

View file

@ -0,0 +1 @@
/target

2344
_primitives/_rust/kei-migrate/Cargo.lock generated Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,28 @@
[package]
name = "kei-migrate"
version = "0.1.0"
edition = "2021"
description = "Universal SQL migration runner — Postgres/SQLite/MySQL autodetect from DATABASE_URL"
license = "MIT"
[[bin]]
name = "kei-migrate"
path = "src/main.rs"
[dependencies]
anyhow = "1"
chrono = { version = "0.4", default-features = false, features = ["clock"] }
clap = { version = "4", features = ["derive", "env"] }
sha2 = "0.10"
sqlx = { version = "0.8", default-features = false, features = [
"runtime-tokio",
"tls-rustls",
"any",
"postgres",
"sqlite",
"mysql",
] }
tokio = { version = "1", features = ["rt-multi-thread", "macros", "fs"] }
[dev-dependencies]
tempfile = "3"

View file

@ -0,0 +1,44 @@
//! CLI surface — clap argument parsing for `kei-migrate`.
use clap::{Parser, Subcommand};
#[derive(Parser, Debug)]
#[command(
name = "kei-migrate",
about = "Universal SQL migration runner (Postgres / SQLite / MySQL)",
version
)]
pub struct Cli {
/// Database URL. Overrides $DATABASE_URL.
/// Formats:
/// postgres://user:pass@host:port/db
/// sqlite:///absolute/path.db or sqlite::memory:
/// mysql://user:pass@host:port/db
#[arg(long, env = "DATABASE_URL")]
pub database_url: String,
/// Migrations directory (default: ./migrations)
#[arg(long, default_value = "migrations")]
pub dir: String,
#[command(subcommand)]
pub command: Command,
}
#[derive(Subcommand, Debug)]
pub enum Command {
/// Apply all pending migrations.
Up,
/// Revert the last N migrations (requires <ts>_<name>.down.sql).
Down {
#[arg(default_value_t = 1)]
n: u32,
},
/// List applied vs pending migrations.
Status,
/// Create a new timestamped migration scaffold: <ts>_<name>.sql (+ .down.sql).
Create {
/// Short migration name, e.g. "add_users_email_index".
name: String,
},
}

View file

@ -0,0 +1,44 @@
//! `kei-migrate create <name>` — scaffold a new timestamped migration pair.
use anyhow::{bail, Context, Result};
use chrono::Utc;
use std::fs;
use std::path::{Path, PathBuf};
const UP_TEMPLATE: &str = "-- up migration\n-- Write forward-direction SQL below.\n\n";
const DOWN_TEMPLATE: &str =
"-- down migration\n-- Write reverse SQL below, or add `-- IRREVERSIBLE` to block reversion.\n\n";
/// Create `<dir>/<utc-timestamp>_<sanitized-name>.sql` + `.down.sql`. Returns paths written.
pub fn run(dir: &Path, name: &str) -> Result<(PathBuf, PathBuf)> {
validate_name(name)?;
fs::create_dir_all(dir).with_context(|| format!("mkdir -p {}", dir.display()))?;
let ts = Utc::now().format("%Y%m%d%H%M%S").to_string();
let sanitized = sanitize(name);
let up = dir.join(format!("{}_{}.sql", ts, sanitized));
let down = dir.join(format!("{}_{}.down.sql", ts, sanitized));
if up.exists() || down.exists() {
bail!("collision: {} or {} already exists", up.display(), down.display());
}
fs::write(&up, UP_TEMPLATE)?;
fs::write(&down, DOWN_TEMPLATE)?;
println!("[create] {}", up.display());
println!("[create] {}", down.display());
Ok((up, down))
}
fn validate_name(name: &str) -> Result<()> {
if name.is_empty() {
bail!("migration name must not be empty");
}
if name.len() > 80 {
bail!("migration name too long ({} chars, max 80)", name.len());
}
Ok(())
}
fn sanitize(name: &str) -> String {
name.chars()
.map(|c| if c.is_ascii_alphanumeric() { c.to_ascii_lowercase() } else { '_' })
.collect()
}

View file

@ -0,0 +1,58 @@
//! `kei-migrate down [n]` — revert the last N applied migrations.
//!
//! Requires a sibling `<version>_<name>.down.sql` for each target. Missing
//! down-file = hard error — we don't guess reversals.
use crate::discover::Migration;
use crate::tracker;
use anyhow::{bail, Context, Result};
use sqlx::AnyPool;
use std::collections::HashMap;
/// Revert the last `n` applied migrations in reverse order.
pub async fn run(pool: &AnyPool, migrations: &[Migration], n: u32) -> Result<u32> {
let mut applied: Vec<i64> = tracker::applied_versions(pool).await?;
applied.sort_unstable();
applied.reverse(); // newest first
let by_version: HashMap<i64, &Migration> =
migrations.iter().map(|m| (m.version, m)).collect();
let mut reverted = 0u32;
for v in applied.into_iter().take(n as usize) {
let m = by_version.get(&v).with_context(|| {
format!("applied version {} has no matching file on disk", v)
})?;
revert_one(pool, m).await?;
reverted += 1;
println!("[down] {} {} — reverted", m.version, m.name);
}
Ok(reverted)
}
async fn revert_one(pool: &AnyPool, m: &Migration) -> Result<()> {
let down_path = m.down_path.as_ref().ok_or_else(|| {
anyhow::anyhow!(
"no down-sql for migration {} ({}) — create {}_{}.down.sql",
m.version,
m.name,
m.version,
m.name
)
})?;
let sql = std::fs::read_to_string(down_path)
.with_context(|| format!("read {}", down_path.display()))?;
if sql.contains("-- IRREVERSIBLE") {
bail!(
"migration {} ({}) is marked IRREVERSIBLE — refusing to run down-sql",
m.version,
m.name
);
}
let mut tx = pool.begin().await?;
sqlx::raw_sql(&sql)
.execute(&mut *tx)
.await
.with_context(|| format!("revert migration {} ({})", m.version, m.name))?;
tx.commit().await?;
tracker::record_down(pool, m.version).await?;
Ok(())
}

View file

@ -0,0 +1,29 @@
//! `kei-migrate status` — list applied + pending migrations.
use crate::discover::Migration;
use crate::tracker;
use anyhow::Result;
use sqlx::AnyPool;
use std::collections::HashSet;
/// Print a human-readable table. Returns (applied_count, pending_count).
pub async fn run(pool: &AnyPool, migrations: &[Migration]) -> Result<(u32, u32)> {
let applied: HashSet<i64> = tracker::applied_versions(pool).await?.into_iter().collect();
let mut a = 0u32;
let mut p = 0u32;
println!("{:>14} {:<8} name", "version", "status");
println!("{:>14} {:<8} ----", "-------", "------");
for m in migrations {
let status = if applied.contains(&m.version) {
a += 1;
"APPLIED"
} else {
p += 1;
"PENDING"
};
println!("{:>14} {:<8} {}", m.version, status, m.name);
}
println!();
println!("{} applied, {} pending", a, p);
Ok((a, p))
}

View file

@ -0,0 +1,42 @@
//! `kei-migrate up` — apply all pending migrations in version-ASC order.
use crate::discover::Migration;
use crate::tracker;
use anyhow::{Context, Result};
use chrono::Utc;
use sqlx::AnyPool;
use std::collections::HashSet;
/// Apply every migration whose version is not in the applied set.
/// Each migration runs in its own transaction; failure aborts and leaves
/// prior applied migrations committed.
pub async fn run(pool: &AnyPool, migrations: &[Migration]) -> Result<u32> {
let applied: HashSet<i64> = tracker::applied_versions(pool).await?.into_iter().collect();
let on_disk: Vec<(i64, &str, &str)> = migrations
.iter()
.map(|m| (m.version, m.name.as_str(), m.checksum.as_str()))
.collect();
tracker::verify_checksums(pool, on_disk).await?;
let mut count = 0u32;
for m in migrations {
if applied.contains(&m.version) {
continue;
}
apply_one(pool, m).await?;
count += 1;
println!("[up] {} {} — applied", m.version, m.name);
}
Ok(count)
}
async fn apply_one(pool: &AnyPool, m: &Migration) -> Result<()> {
let mut tx = pool.begin().await?;
sqlx::raw_sql(&m.up_sql)
.execute(&mut *tx)
.await
.with_context(|| format!("apply migration {} ({})", m.version, m.name))?;
tx.commit().await?;
let now = Utc::now().to_rfc3339();
tracker::record_up(pool, m.version, &m.name, &m.checksum, &now).await?;
Ok(())
}

View file

@ -0,0 +1,68 @@
//! Database backend detection + pool construction.
//!
//! Uses `sqlx::Any` so one binary covers Postgres / SQLite / MySQL.
//! Detection is purely on URL scheme — no live probe needed.
use anyhow::{bail, Result};
use sqlx::any::{install_default_drivers, AnyPoolOptions};
use sqlx::AnyPool;
/// Backend inferred from the URL scheme. Determines dialect quirks.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Backend {
Postgres,
Sqlite,
Mysql,
}
impl Backend {
/// Backend-specific CREATE TABLE for `_kei_migrations`.
pub fn create_tracker_sql(self) -> &'static str {
match self {
Backend::Postgres | Backend::Mysql => {
"CREATE TABLE IF NOT EXISTS _kei_migrations (
version BIGINT PRIMARY KEY,
name VARCHAR(255) NOT NULL,
checksum CHAR(64) NOT NULL,
applied_at VARCHAR(32) NOT NULL
)"
}
Backend::Sqlite => {
"CREATE TABLE IF NOT EXISTS _kei_migrations (
version INTEGER PRIMARY KEY,
name TEXT NOT NULL,
checksum TEXT NOT NULL,
applied_at TEXT NOT NULL
)"
}
}
}
}
/// Parse a database URL into a [`Backend`]. Never touches the network.
pub fn detect_backend(url: &str) -> Result<Backend> {
let lower = url.to_ascii_lowercase();
if lower.starts_with("postgres://") || lower.starts_with("postgresql://") {
Ok(Backend::Postgres)
} else if lower.starts_with("sqlite:") {
Ok(Backend::Sqlite)
} else if lower.starts_with("mysql://") || lower.starts_with("mariadb://") {
Ok(Backend::Mysql)
} else {
bail!(
"unsupported or unrecognised DATABASE_URL scheme: {}. \
Expected postgres://, sqlite:, or mysql://",
url
)
}
}
/// Build a sqlx `AnyPool` for the given URL (max 4 conns — migration runner is not a server).
pub async fn connect(url: &str) -> Result<AnyPool> {
install_default_drivers();
let pool = AnyPoolOptions::new()
.max_connections(4)
.connect(url)
.await?;
Ok(pool)
}

View file

@ -0,0 +1,87 @@
//! Filesystem migration discovery.
//!
//! Convention: `migrations/<version>_<name>.sql` (up) and optional
//! `migrations/<version>_<name>.down.sql` (down). Version is a monotonic
//! integer, typically a UTC timestamp like `20260421120000`.
use anyhow::{bail, Context, Result};
use sha2::{Digest, Sha256};
use std::fs;
use std::path::{Path, PathBuf};
/// One discovered migration (up-side). `down_path` is `Some` iff the sibling file exists.
#[derive(Debug, Clone)]
pub struct Migration {
pub version: i64,
pub name: String,
pub up_path: PathBuf,
pub down_path: Option<PathBuf>,
pub up_sql: String,
pub checksum: String,
}
/// Read every `<version>_<name>.sql` file (ignoring `.down.sql`), sort by version ASC.
pub fn scan(dir: &Path) -> Result<Vec<Migration>> {
if !dir.exists() {
return Ok(Vec::new());
}
let mut out = Vec::new();
for entry in fs::read_dir(dir).with_context(|| format!("read_dir {}", dir.display()))? {
let entry = entry?;
let path = entry.path();
if path.extension().and_then(|s| s.to_str()) != Some("sql") {
continue;
}
let fname = path.file_name().and_then(|s| s.to_str()).unwrap_or("");
if fname.ends_with(".down.sql") {
continue;
}
let m = parse_migration(&path)?;
out.push(m);
}
out.sort_by_key(|m| m.version);
check_unique(&out)?;
Ok(out)
}
fn parse_migration(path: &Path) -> Result<Migration> {
let stem = path
.file_stem()
.and_then(|s| s.to_str())
.context("non-utf8 filename")?;
let (ver_str, name) = stem
.split_once('_')
.with_context(|| format!("filename not <version>_<name>.sql: {}", stem))?;
let version: i64 = ver_str
.parse()
.with_context(|| format!("version must be integer, got {}", ver_str))?;
let up_sql = fs::read_to_string(path)
.with_context(|| format!("read {}", path.display()))?;
let mut hasher = Sha256::new();
hasher.update(up_sql.as_bytes());
let checksum = format!("{:x}", hasher.finalize());
let down_path = path.with_file_name(format!("{}_{}.down.sql", version, name));
let down = if down_path.exists() { Some(down_path) } else { None };
Ok(Migration {
version,
name: name.to_string(),
up_path: path.to_path_buf(),
down_path: down,
up_sql,
checksum,
})
}
fn check_unique(migs: &[Migration]) -> Result<()> {
for w in migs.windows(2) {
if w[0].version == w[1].version {
bail!(
"duplicate migration version {} ({} and {})",
w[0].version,
w[0].up_path.display(),
w[1].up_path.display()
);
}
}
Ok(())
}

View file

@ -0,0 +1,54 @@
//! kei-migrate — universal SQL migration runner.
//!
//! Single binary, three backends (Postgres / SQLite / MySQL) autodetected
//! from `DATABASE_URL`. Sequential `.sql` files in `migrations/`, tracked in
//! `_kei_migrations` with SHA-256 checksums.
//!
//! Library surface exists so integration tests can drive the primitive
//! without `process::Command` gymnastics.
pub mod cli;
pub mod cmd_create;
pub mod cmd_down;
pub mod cmd_status;
pub mod cmd_up;
pub mod db;
pub mod discover;
pub mod tracker;
use anyhow::Result;
use std::path::Path;
/// End-to-end `up` entry: connect, ensure tracker, scan dir, apply pending.
/// Returns number of migrations applied.
pub async fn do_up(database_url: &str, dir: &Path) -> Result<u32> {
let backend = db::detect_backend(database_url)?;
let pool = db::connect(database_url).await?;
tracker::ensure_table(&pool, backend).await?;
let migs = discover::scan(dir)?;
let n = cmd_up::run(&pool, &migs).await?;
pool.close().await;
Ok(n)
}
/// End-to-end `down` entry: revert last N applied.
pub async fn do_down(database_url: &str, dir: &Path, n: u32) -> Result<u32> {
let backend = db::detect_backend(database_url)?;
let pool = db::connect(database_url).await?;
tracker::ensure_table(&pool, backend).await?;
let migs = discover::scan(dir)?;
let reverted = cmd_down::run(&pool, &migs, n).await?;
pool.close().await;
Ok(reverted)
}
/// End-to-end `status` entry: returns (applied, pending) counts.
pub async fn do_status(database_url: &str, dir: &Path) -> Result<(u32, u32)> {
let backend = db::detect_backend(database_url)?;
let pool = db::connect(database_url).await?;
tracker::ensure_table(&pool, backend).await?;
let migs = discover::scan(dir)?;
let r = cmd_status::run(&pool, &migs).await?;
pool.close().await;
Ok(r)
}

View file

@ -0,0 +1,30 @@
//! kei-migrate CLI entry. Dispatches to the library surface in `lib.rs`.
use anyhow::Result;
use clap::Parser;
use kei_migrate::cli::{Cli, Command};
use kei_migrate::{cmd_create, do_down, do_status, do_up};
use std::path::Path;
#[tokio::main(flavor = "multi_thread", worker_threads = 2)]
async fn main() -> Result<()> {
let cli = Cli::parse();
let dir = Path::new(&cli.dir);
match cli.command {
Command::Up => {
let n = do_up(&cli.database_url, dir).await?;
println!("up: {} migration(s) applied", n);
}
Command::Down { n } => {
let r = do_down(&cli.database_url, dir, n).await?;
println!("down: {} migration(s) reverted", r);
}
Command::Status => {
do_status(&cli.database_url, dir).await?;
}
Command::Create { name } => {
cmd_create::run(dir, &name)?;
}
}
Ok(())
}

View file

@ -0,0 +1,86 @@
//! `_kei_migrations` tracking table operations.
//!
//! Row shape: (version, name, checksum, applied_at). Checksum guards against
//! silent edits of an applied file — mismatch = hard fail, requires human ack.
use crate::db::Backend;
use anyhow::{bail, Result};
use sqlx::{AnyPool, Row};
/// Create tracker table if missing. Idempotent.
pub async fn ensure_table(pool: &AnyPool, backend: Backend) -> Result<()> {
sqlx::query(backend.create_tracker_sql()).execute(pool).await?;
Ok(())
}
/// Versions of all applied migrations, ASC.
pub async fn applied_versions(pool: &AnyPool) -> Result<Vec<i64>> {
let rows = sqlx::query("SELECT version FROM _kei_migrations ORDER BY version ASC")
.fetch_all(pool)
.await?;
let mut out = Vec::with_capacity(rows.len());
for r in rows {
out.push(r.try_get::<i64, _>(0)?);
}
Ok(out)
}
/// Checksum of a specific applied version, or `None` if not applied.
pub async fn applied_checksum(pool: &AnyPool, version: i64) -> Result<Option<String>> {
let row = sqlx::query("SELECT checksum FROM _kei_migrations WHERE version = $1")
.bind(version)
.fetch_optional(pool)
.await?;
Ok(row.map(|r| r.try_get::<String, _>(0)).transpose()?)
}
/// Insert a tracker row after a successful up-migration.
pub async fn record_up(
pool: &AnyPool,
version: i64,
name: &str,
checksum: &str,
applied_at: &str,
) -> Result<()> {
sqlx::query(
"INSERT INTO _kei_migrations (version, name, checksum, applied_at) VALUES ($1, $2, $3, $4)",
)
.bind(version)
.bind(name)
.bind(checksum)
.bind(applied_at)
.execute(pool)
.await?;
Ok(())
}
/// Delete a tracker row after a successful down-migration.
pub async fn record_down(pool: &AnyPool, version: i64) -> Result<()> {
sqlx::query("DELETE FROM _kei_migrations WHERE version = $1")
.bind(version)
.execute(pool)
.await?;
Ok(())
}
/// Abort if any applied migration's recorded checksum doesn't match the on-disk file.
pub async fn verify_checksums<'a, I>(pool: &AnyPool, on_disk: I) -> Result<()>
where
I: IntoIterator<Item = (i64, &'a str, &'a str)>, // (version, name, checksum)
{
for (version, name, disk_sum) in on_disk {
if let Some(db_sum) = applied_checksum(pool, version).await? {
if db_sum != disk_sum {
bail!(
"checksum drift on applied migration {} ({}): db={}, disk={}. \
Refusing to proceed someone edited an already-applied file.",
version,
name,
db_sum,
disk_sum
);
}
}
}
Ok(())
}

View file

@ -0,0 +1,174 @@
//! Integration tests for kei-migrate against a SQLite file (safe, no deps).
//!
//! SQLite is chosen as the test backend because it has no server dependency
//! and the sqlx-Any path through it exercises the same code path as Postgres
//! / MySQL for everything except dialect-specific DDL (which we abstract in
//! `db::Backend::create_tracker_sql`).
use kei_migrate::{cmd_create, db, discover, do_down, do_status, do_up, tracker};
use std::fs;
use std::path::PathBuf;
use tempfile::TempDir;
struct Env {
_tmp: TempDir,
url: String,
dir: PathBuf,
}
fn setup() -> Env {
let tmp = TempDir::new().unwrap();
let db_path = tmp.path().join("test.db");
let url = format!("sqlite://{}?mode=rwc", db_path.display());
let dir = tmp.path().join("migrations");
fs::create_dir_all(&dir).unwrap();
Env { _tmp: tmp, url, dir }
}
fn write_migration(dir: &std::path::Path, version: i64, name: &str, up: &str, down: Option<&str>) {
fs::write(dir.join(format!("{}_{}.sql", version, name)), up).unwrap();
if let Some(d) = down {
fs::write(dir.join(format!("{}_{}.down.sql", version, name)), d).unwrap();
}
}
#[test]
fn detects_backend_from_url_scheme() {
assert_eq!(
db::detect_backend("postgres://u:p@h/d").unwrap(),
db::Backend::Postgres
);
assert_eq!(
db::detect_backend("sqlite:///tmp/x.db").unwrap(),
db::Backend::Sqlite
);
assert_eq!(
db::detect_backend("mysql://u:p@h/d").unwrap(),
db::Backend::Mysql
);
assert!(db::detect_backend("mongodb://h").is_err());
}
#[test]
fn scan_empty_dir_is_empty() {
let env = setup();
let migs = discover::scan(&env.dir).unwrap();
assert!(migs.is_empty());
}
#[test]
fn scan_sorts_by_version_and_skips_down_files() {
let env = setup();
write_migration(&env.dir, 2, "second", "SELECT 1;", Some("SELECT 2;"));
write_migration(&env.dir, 1, "first", "SELECT 3;", None);
let migs = discover::scan(&env.dir).unwrap();
assert_eq!(migs.len(), 2);
assert_eq!(migs[0].version, 1);
assert_eq!(migs[1].version, 2);
assert!(migs[0].down_path.is_none());
assert!(migs[1].down_path.is_some());
}
#[test]
fn scan_rejects_duplicate_versions() {
let env = setup();
write_migration(&env.dir, 1, "a", "SELECT 1;", None);
// same version, different name
fs::write(env.dir.join("1_b.sql"), "SELECT 2;").unwrap();
let err = discover::scan(&env.dir).unwrap_err();
assert!(err.to_string().contains("duplicate migration version"));
}
#[tokio::test]
async fn up_applies_pending_and_tracks_them() {
let env = setup();
write_migration(
&env.dir,
1,
"create_t",
"CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT);",
Some("DROP TABLE t;"),
);
write_migration(
&env.dir,
2,
"insert_row",
"INSERT INTO t (id, v) VALUES (1, 'a');",
Some("DELETE FROM t WHERE id = 1;"),
);
let n = do_up(&env.url, &env.dir).await.unwrap();
assert_eq!(n, 2);
// re-running is a no-op
let n2 = do_up(&env.url, &env.dir).await.unwrap();
assert_eq!(n2, 0);
// status: 2 applied, 0 pending
let (a, p) = do_status(&env.url, &env.dir).await.unwrap();
assert_eq!(a, 2);
assert_eq!(p, 0);
}
#[tokio::test]
async fn down_reverts_last_n() {
let env = setup();
write_migration(
&env.dir,
1,
"create_t",
"CREATE TABLE t (id INTEGER PRIMARY KEY);",
Some("DROP TABLE t;"),
);
write_migration(
&env.dir,
2,
"add_col",
"ALTER TABLE t ADD COLUMN v TEXT;",
Some("-- down unsupported on sqlite but we just drop the table for this test\nDROP TABLE t; CREATE TABLE t (id INTEGER PRIMARY KEY);"),
);
do_up(&env.url, &env.dir).await.unwrap();
let reverted = do_down(&env.url, &env.dir, 1).await.unwrap();
assert_eq!(reverted, 1);
let (a, p) = do_status(&env.url, &env.dir).await.unwrap();
assert_eq!(a, 1);
assert_eq!(p, 1);
}
#[tokio::test]
async fn down_refuses_irreversible_marker() {
let env = setup();
write_migration(
&env.dir,
1,
"make_t",
"CREATE TABLE t (id INTEGER PRIMARY KEY);",
Some("-- IRREVERSIBLE\n-- don't auto-revert"),
);
do_up(&env.url, &env.dir).await.unwrap();
let err = do_down(&env.url, &env.dir, 1).await.unwrap_err();
assert!(err.to_string().contains("IRREVERSIBLE"));
}
#[tokio::test]
async fn up_detects_checksum_drift() {
let env = setup();
let up_path = env.dir.join("1_seed.sql");
fs::write(&up_path, "CREATE TABLE t (id INTEGER PRIMARY KEY);").unwrap();
do_up(&env.url, &env.dir).await.unwrap();
// someone edits the already-applied file
fs::write(&up_path, "CREATE TABLE t (id INTEGER PRIMARY KEY, extra TEXT);").unwrap();
let err = do_up(&env.url, &env.dir).await.unwrap_err();
assert!(err.to_string().contains("checksum drift"));
// tracker module is exercised end-to-end here
let _ = tracker::applied_versions; // keep tracker import used
}
#[test]
fn create_scaffolds_up_and_down_files() {
let env = setup();
let (up, down) = cmd_create::run(&env.dir, "add_users_index").unwrap();
assert!(up.exists());
assert!(down.exists());
let up_txt = fs::read_to_string(&up).unwrap();
let down_txt = fs::read_to_string(&down).unwrap();
assert!(up_txt.contains("up migration"));
assert!(down_txt.contains("down migration"));
}

View file

@ -0,0 +1,115 @@
---
name: schema-design
description: Hub-and-spoke pipeline that converts "I need a database for app X" into a designed relational schema, a generated first migration, and optional seed/fixture data — via pure-click decisions across five phases. Emits SQL DDL, a kei-migrate-shaped migrations directory, and a library/ORM pick; never writes production secrets.
argument-hint: <one-line app description, e.g. "multi-tenant B2B SaaS, 6-8 entities, Postgres + Drizzle">
---
# Schema-Design — Relational Schema & Migration Pipeline (index)
You are converting "I need a database for app X" into a concrete, reviewable
design: chosen DB + ORM, entity list + relations, SQL DDL with indexes and
FKs, a scaffolded migrations directory with the first migration, and (if
asked) seed data for tests and dev. Every decision is a click; the only
typed inputs are the one-line app description in Phase 1 and the entity
list in Phase 2.
This skill does NOT run migrations or touch production. It produces files
under `db/schema.sql`, `migrations/<ts>_init.sql` (+ `.down.sql`), and
optionally `db/seed.sql`. Applying them is a separate command (`kei-migrate
up`), owned by the project's code-implementer.
The skill reads the five database blocks heavily — every phase references
at least one of them:
- `_blocks/db-postgres.md` — PG 17 patterns, indexing, pooling.
- `_blocks/db-sqlite.md` — single-node / edge pragmas.
- `_blocks/db-sqlx.md` — Rust query + migration flow.
- `_blocks/db-drizzle.md` — TS schema-first ORM.
- `_blocks/db-migration-hygiene.md` — universal up/down + checksum rules.
Primitive used for scaffolding: `_primitives/_rust/kei-migrate` (universal
Postgres / SQLite / MySQL migration runner — create + up + down + status).
---
## Pipeline overview (5 phases, ≥5 AskUserQuestion calls)
| Phase | File | Purpose | AskUserQuestion |
|---|---|---|---|
| 1 | [phase-1-intake.md](phase-1-intake.md) | DB, ORM, scale, style, migration control | 5× (batched) |
| 2 | [phase-2-entities.md](phase-2-entities.md) | Entity list + relations matrix | 1× |
| 3 | [phase-3-schema.md](phase-3-schema.md) | Generate DDL + indexes + FKs + constraints; review/revise | 1× |
| 4 | [phase-4-migrations.md](phase-4-migrations.md) | Scaffold `migrations/` + first migration + kei-migrate wiring | 1× |
| 5 | [phase-5-seed.md](phase-5-seed.md) | Optional seed + test fixtures | 1× |
Minimum AskUserQuestion count across a full session: **9** (5 in Phase 1 +
1 each in Phases 25). Exceeds the ≥5 hub-and-spoke contract.
---
## Variables the pipeline produces
| Name | Set in | Meaning |
|---|---|---|
| `INTAKE` | Phase 1 | one-paragraph app description (verbatim) |
| `DB` | Phase 1 | Postgres / SQLite / MySQL |
| `ORM` | Phase 1 | none (raw SQL) / Drizzle / SQLx / Prisma / SQLAlchemy |
| `SCALE` | Phase 1 | solo-prototype / team-dev / production-multi-replica |
| `STYLE` | Phase 1 | schema-first (SQL → types) / code-first (types → SQL) |
| `MIGCTL` | Phase 1 | manual / auto-on-deploy / hybrid (manual prod, auto dev) |
| `ENTITIES` | Phase 2 | list of entities + fields + relations matrix |
| `DDL` | Phase 3 | generated SQL (tables, indexes, FKs, constraints) |
| `MIGDIR` | Phase 4 | path of migrations dir + first migration filenames |
| `SEED` | Phase 5 | seed-data plan (or "skipped") |
---
## Final report (emit after Phase 5)
```
=== SCHEMA-DESIGN REPORT ===
App: <first 80 chars of INTAKE>...
DB / ORM: <DB> + <ORM> (style: <STYLE>)
Scale: <SCALE> migration control: <MIGCTL>
Entities: <N> tables, <M> relations
Schema: db/schema.sql (<LOC> lines, <I> indexes, <F> FKs, <C> constraints)
Migrations: migrations/<ts>_init.sql (+ .down.sql) runner: kei-migrate
Seed: <SEED summary or "skipped">
Libraries: <ORM pick + driver crate/package, one line>
Next: run `kei-migrate up` against dev DB, then hand off to code-implementer
```
---
## Rules (apply throughout)
- **Pure-click contract.** Only the Phase 1 intake paragraph and the Phase 2
entity list are typed. Every other decision is an `AskUserQuestion` call.
- **RULE 0.8 Secrets SSoT.** The skill never emits a live `DATABASE_URL`
value, never writes to `secrets/*.env`, never hard-codes credentials in
DDL or seed. It emits ENV-VAR NAMES only; storage path is
`<repo>/secrets/db.env` per `domain-has-secrets.md`.
- **NO DOWNGRADE.** If the chosen combination is unsafe (e.g.
`auto-on-deploy` + multi-replica without leader-election) the skill
returns 23 constructive alternatives, never "not supported".
- **Migration hygiene enforced.** Every migration emitted is
timestamp-prefixed, has a `.down.sql` counterpart, and uses
`IF NOT EXISTS` / `IF EXISTS` where safe. See `db-migration-hygiene.md`.
- **Test-First.** If Phase 5 is selected, seed includes at minimum a
smoke-test fixture (one row per entity) to verify schema loads.
- **Surgical scope.** Reads the five db-* blocks; writes only to
`db/schema.sql`, `migrations/<ts>_init.{sql,down.sql}`, and optionally
`db/seed.sql`. Never touches application code.
---
## References
- `_blocks/db-postgres.md`, `_blocks/db-sqlite.md`, `_blocks/db-sqlx.md`,
`_blocks/db-drizzle.md`, `_blocks/db-migration-hygiene.md`.
- `_primitives/_rust/kei-migrate` — universal migration runner
(autodetects Postgres / SQLite / MySQL from `DATABASE_URL`).
- `_blocks/domain-has-secrets.md` — DB URL storage convention.
- `_blocks/rule-pre-dev-gate.md` — check existing schema before inventing.
- Evidence grade [E4] — pipeline mirrors standard relational-modelling
practice (Codd 1NF-3NF, surrogate keys, FK-first indexing).

View file

@ -0,0 +1,92 @@
# Phase 1 — Intake (DB, ORM, scale, style, migration control)
One free-text paragraph, then ONE batched `AskUserQuestion` call with all
five click decisions. This is the only phase that accepts typed input.
## 1a — Ask for the app description
Emit a regular message (NOT AskUserQuestion):
> Describe the app in one paragraph: what is it, how many entities (rough
> count), any constraint I should know (existing DB, regulated data,
> multi-tenant, edge / serverless, expected row counts). Reply in one
> message.
Store the reply verbatim as `INTAKE`.
## 1b — Batched click (AskUserQuestion, 5 questions in ONE call)
The UI cap per `AskUserQuestion` call is 45 questions; emit all five at
once for a smooth click-through.
```json
{
"questions": [
{
"question": "Which database engine?",
"header": "DB",
"multiSelect": false,
"options": [
{"label": "PostgreSQL 17", "description": "Default for multi-user / relational integrity. See _blocks/db-postgres.md"},
{"label": "SQLite", "description": "Single-node, edge-friendly, ~100k users ceiling. See _blocks/db-sqlite.md"},
{"label": "MySQL / MariaDB", "description": "Existing stack compatibility; kei-migrate supports it"}
]
},
{
"question": "ORM / query layer?",
"header": "ORM",
"multiSelect": false,
"options": [
{"label": "None (raw SQL)", "description": "Hand-written queries; max control, no magic"},
{"label": "Drizzle (TS)", "description": "Schema-first or code-first; see _blocks/db-drizzle.md"},
{"label": "SQLx (Rust)", "description": "Compile-time checked queries; see _blocks/db-sqlx.md"},
{"label": "Prisma (TS)", "description": "Code-first; own migration engine (NOT kei-migrate)"},
{"label": "SQLAlchemy (Py)", "description": "Alembic for migrations (NOT kei-migrate); legacy compat"}
]
},
{
"question": "Target scale?",
"header": "Scale",
"multiSelect": false,
"options": [
{"label": "Solo prototype", "description": "One dev, <1k rows, SQLite OK"},
{"label": "Team dev", "description": "Shared dev DB, staging, prod — standard"},
{"label": "Production multi-replica", "description": "Leader-election required for migrations; zero-downtime patterns mandatory"}
]
},
{
"question": "Design style?",
"header": "Style",
"multiSelect": false,
"options": [
{"label": "Schema-first (SQL → types)", "description": "Write DDL, generate types. Default with raw SQL / SQLx / Drizzle schema-first"},
{"label": "Code-first (types → SQL)", "description": "Define entities in code, generate DDL. Drizzle code-first / Prisma / SQLAlchemy"}
]
},
{
"question": "Migration control?",
"header": "MigCtl",
"multiSelect": false,
"options": [
{"label": "Manual (human runs kei-migrate up)", "description": "Safest; recommended for prod"},
{"label": "Auto-on-deploy", "description": "CI runs migrations; single-replica only — NO DOWNGRADE warning if multi-replica"},
{"label": "Hybrid (manual prod, auto dev)", "description": "Recommended default — dev velocity + prod safety"}
]
}
]
}
```
Store answers as `DB`, `ORM`, `SCALE`, `STYLE`, `MIGCTL`.
## Verify-criterion
- `INTAKE` non-empty.
- `DB`, `ORM`, `SCALE`, `STYLE`, `MIGCTL` each exactly one label.
- If `ORM ∈ {Prisma, SQLAlchemy}` → note in state: "Phase 4 will hand off
to that tool's native migration runner (Prisma migrate / Alembic); the
kei-migrate scaffold is skipped or wrapped." No downgrade — the skill
still emits a working plan.
- If `MIGCTL = Auto-on-deploy` AND `SCALE = Production multi-replica`
warn "race condition risk — every replica tries to apply" (see
`db-migration-hygiene.md`) and re-ask with the Hybrid option highlighted.

View file

@ -0,0 +1,82 @@
# Phase 2 — Entities + relations matrix
Collect the entity list (typed) once, then click the relations matrix. This
is the second (and last) phase that accepts typed input.
## 2a — Ask for entities (plain message, NOT AskUserQuestion)
> List the entities (tables) and for each a short comma-separated field
> list. One entity per line, format: `<Entity>: field1, field2, ...`.
> Example:
>
> ```
> User: email, name, created_at
> Organization: name, plan
> Membership: user_id, org_id, role
> ```
>
> 315 entities is typical. Keep it short — we'll refine fields in Phase 3.
Parse the reply into `ENTITIES = [{name, fields: [...]}, ...]`. Validate:
- Entity names are `PascalCase` (normalize if user types `user_profile`
`UserProfile`, record normalization in state).
- Each entity has ≥1 field.
- No duplicate entity names.
- If parse fails → re-ask once with a corrected example.
## 2b — Relations matrix click (AskUserQuestion, multi-select)
For each UNORDERED PAIR of entities `(A, B)`, ask one multi-select row.
Skip pairs the user hasn't mentioned any cross-reference for (heuristic:
if `A`'s fields include `b_id` or `B`'s fields include `a_id`, or the
user's intake paragraph mentions both).
Build ONE `AskUserQuestion` call with up to 5 questions. If the entity
count yields > 5 candidate pairs, batch into multiple calls (still counts
toward the ≥1 AskUserQuestion minimum).
Per-pair question template:
```json
{
"question": "Relation between <A> and <B>?",
"header": "<A><B>",
"multiSelect": false,
"options": [
{"label": "None", "description": "No direct FK; entities are independent"},
{"label": "One-to-one", "description": "A.b_id UNIQUE FK to B.id (or vice versa)"},
{"label": "One-to-many (A→B)", "description": "B.a_id FK to A.id; one A has many B"},
{"label": "One-to-many (B→A)", "description": "A.b_id FK to B.id; one B has many A"},
{"label": "Many-to-many", "description": "Requires a junction table; skill will auto-name it <A><B>"}
]
}
```
Store the result in `ENTITIES` as `.relations = [{from, to, kind}, ...]`.
## 2c — Auto-generate junction tables
For each pair marked `Many-to-many`, append a synthetic entity to
`ENTITIES`:
```
<A><B>:
<a>_id FK → <A>.id (ON DELETE CASCADE)
<b>_id FK → <B>.id (ON DELETE CASCADE)
PRIMARY KEY (<a>_id, <b>_id)
created_at TIMESTAMPTZ DEFAULT now()
```
Names must be deterministic (alphabetical order: `OrganizationUser`, not
`UserOrganization`, for pair `(User, Organization)`). Record the rule in
state so Phase 3 renders it consistently.
## Verify-criterion
- `ENTITIES` has ≥1 entry after parse.
- Every relation in `ENTITIES[*].relations` references two distinct
existing entities.
- Every `Many-to-many` has produced a junction entity.
- No entity is orphaned (zero relations AND not mentioned in INTAKE) —
warn the user with "Entity X has no relations; keep it?" (NO DOWNGRADE:
offer `keep / drop / add relation` as follow-up click).

View file

@ -0,0 +1,115 @@
# Phase 3 — Generate SQL DDL (tables, indexes, FKs, constraints)
Emit a full `db/schema.sql` file based on `DB`, `ORM`, `STYLE`, and
`ENTITIES`. Then ONE `AskUserQuestion` to review/revise.
## 3a — Pick primary-key strategy (inline, no AskUserQuestion — deterministic)
- Postgres 17: `id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY`
(SQL-standard, PG 10+; avoid legacy `SERIAL`). See `db-postgres.md`.
- SQLite: `id INTEGER PRIMARY KEY` (rowid alias; autoincrement discouraged
unless monotonic guarantee required). See `db-sqlite.md`.
- MySQL: `id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY`.
If the user's fields include `id` already, respect it; otherwise prepend
the surrogate PK. Record the choice in state for Phase 4 traceability.
## 3b — Per-entity DDL generation rules
For each entity in `ENTITIES`:
1. **CREATE TABLE**: `CREATE TABLE IF NOT EXISTS <snake_case_name>` (plural
unless user named it singular — default to pluralize: `User``users`,
`Organization``organizations`; junction `OrganizationUser`
`organization_users`).
2. **Columns**: infer SQL type from field name + heuristics:
- `*_id`, `id``BIGINT` (Postgres/MySQL) / `INTEGER` (SQLite).
- `*_at`, `created_at`, `updated_at``TIMESTAMPTZ` (PG) / `DATETIME`
(SQLite) / `TIMESTAMP` (MySQL), `DEFAULT now()` (PG) /
`DEFAULT CURRENT_TIMESTAMP` (others).
- `email`, `*_email``TEXT` / `VARCHAR(320)` (RFC 5321 limit) + `NOT
NULL` + `CHECK (email LIKE '%@%')` (cheap sanity, real validation is
app-side).
- `*_count`, `*_amount``INTEGER` or `NUMERIC(18,2)` (money);
default `0`.
- `is_*`, `has_*``BOOLEAN NOT NULL DEFAULT false`.
- Freeform → `TEXT NOT NULL` unless user said "optional".
3. **Timestamps default**: unless user opts out, add `created_at` +
`updated_at` to every non-junction entity.
4. **Soft-delete**: if `SCALE = Production multi-replica`, add
`deleted_at TIMESTAMPTZ NULL` + partial index (PG only) on
`WHERE deleted_at IS NULL`.
## 3c — Foreign keys
For each `relations` entry (kind ≠ `None`):
- `REFERENCES <parent_table>(id) ON DELETE <ACTION>`.
- Default `ON DELETE` action: junction → `CASCADE`; one-to-one → `CASCADE`
from owning side; one-to-many → `RESTRICT` (safer default; explicit
cascade is a product decision). See `db-migration-hygiene.md`.
- **Mandatory FK index**: every FK column gets `CREATE INDEX
IF NOT EXISTS idx_<table>_<col> ON <table>(<col>);`. Unindexed FKs =
join explosion (see `db-postgres.md` "Forbidden").
## 3d — Indexes + constraints
- Unique: `email`, `slug`, `username`, any field user marked `UNIQUE`
`UNIQUE` column constraint.
- Composite indexes: for junction tables, PK is composite; add a
reverse-order index if both directions of lookup are common.
- Check constraints: `CHECK (status IN (...))` for enum-ish text fields;
`CHECK (amount >= 0)` for non-negative numerics.
- Triggers: if `updated_at` present AND `DB = Postgres`, emit a
`CREATE OR REPLACE FUNCTION set_updated_at()` + `CREATE TRIGGER` per
table. SQLite uses `AFTER UPDATE` triggers; MySQL uses `ON UPDATE
CURRENT_TIMESTAMP` in the column definition.
## 3e — Emit the file
Write `db/schema.sql` with a top-level comment:
```sql
-- Generated by /schema-design — <YYYY-MM-DD>
-- DB: <DB> ORM: <ORM> Style: <STYLE>
-- Entities: <N> Relations: <M>
-- Edit freely; Phase 4 will package the first migration from this file.
```
Print the first ~40 lines of the file inline in chat (for review) plus a
`<path>:<total lines>` footer. Full contents live on disk.
## 3f — Review click (AskUserQuestion)
```json
{
"questions": [
{
"question": "Schema review — accept or revise?",
"header": "Review",
"multiSelect": false,
"options": [
{"label": "Accept — proceed to Phase 4", "description": "Schema is correct; package into a migration"},
{"label": "Revise entities (return to Phase 2)", "description": "Add/drop entities or fix a relation"},
{"label": "Revise types (edit db/schema.sql)", "description": "Skill will emit a 1-AskUserQuestion per-column-type fix loop"},
{"label": "Revise indexes (edit db/schema.sql)", "description": "Skill will emit a 1-AskUserQuestion per-index fix loop"}
]
}
]
}
```
If the user picks Revise, loop the relevant sub-step; then re-emit and
re-ask. Max 3 revise loops before the skill asks whether to accept-as-is
or escalate to code-implementer handoff (NO DOWNGRADE).
## Verify-criterion
- `db/schema.sql` exists on disk and is non-empty.
- Every FK has a companion index.
- Every non-junction entity has a PK + `created_at`.
- If `ORM = Drizzle` with `STYLE = code-first`: also emit
`db/schema.ts` alongside `db/schema.sql` (Drizzle introspection artefact;
see `db-drizzle.md`). For `ORM = SQLx`: record in state that Phase 4's
migration will also be picked up by `sqlx migrate run`.
- Accept click sets `DDL = "db/schema.sql"` and hands off to Phase 4.

View file

@ -0,0 +1,121 @@
# Phase 4 — Migration scaffold + first migration + kei-migrate wiring
Package `db/schema.sql` (from Phase 3) into a proper
timestamp-prefixed migration pair under `migrations/`, and emit the
`kei-migrate` invocation the user should run.
## 4a — Create `migrations/` directory (no AskUserQuestion)
If `migrations/` does not yet exist in the repo, create it. Emit one
`.keep` file or rely on the first migration to anchor it.
If `ORM = Prisma`: the directory is `prisma/migrations/` and the runner is
`prisma migrate dev` — skill notes this and skips kei-migrate wiring
(reference the handoff but DO NOT overwrite Prisma's own layout).
If `ORM = SQLAlchemy`: the directory is `alembic/versions/` and the runner
is `alembic upgrade head` — same rule, skip kei-migrate wiring.
For every other `ORM` value (none / Drizzle / SQLx): use `migrations/`
with kei-migrate.
## 4b — Generate timestamp + filename
- Timestamp format: `YYYYMMDDHHMMSS` (matches `kei-migrate create`'s
convention — see `_primitives/_rust/kei-migrate/src/cmd_create.rs`).
- Migration name: `init_schema`.
- Files:
- `migrations/<ts>_init_schema.sql` (up — full DDL from Phase 3)
- `migrations/<ts>_init_schema.down.sql` (down — `DROP TABLE` reverse order)
## 4c — Up migration content
Copy `db/schema.sql` contents into the up file verbatim, with a one-line
header:
```sql
-- kei-migrate: init_schema (generated <YYYY-MM-DD>)
-- See db/schema.sql for the schema SSoT.
```
**Do not split one migration per table** — the initial schema ships as ONE
migration by convention. Subsequent changes each get their own timestamp.
## 4d — Down migration content
Emit `DROP TABLE IF EXISTS <name> CASCADE;` for every entity, in REVERSE
dependency order (children before parents — junctions first, then leaf
entities, then referenced entities last).
If any table is flagged `-- IRREVERSIBLE` by the user (e.g. contains
critical data once populated), replace the `DROP TABLE` line with:
```sql
-- IRREVERSIBLE: this table holds production data; manual restore required.
-- Abort reverse migration.
SELECT RAISE(FAIL, 'irreversible: init_schema') ; -- or equivalent per DB
```
See `db-migration-hygiene.md` for the irreversible pattern.
## 4e — Wire kei-migrate (AskUserQuestion)
```json
{
"questions": [
{
"question": "Add kei-migrate to the project?",
"header": "Runner",
"multiSelect": false,
"options": [
{"label": "Add to Cargo workspace as path dep", "description": "Rust projects — edit root Cargo.toml members. Skill will NOT edit; emits the snippet for you to paste."},
{"label": "Install prebuilt binary (system-wide)", "description": "Any stack — `cargo install --path _primitives/_rust/kei-migrate` once; repo stays tool-agnostic"},
{"label": "Use existing runner (Prisma / Alembic / Drizzle-kit / goose / Atlas)", "description": "Skill skips kei-migrate; records the handoff in the report"},
{"label": "Decide later", "description": "Files land on disk; runner wiring deferred"}
]
}
]
}
```
Store the answer as `RUNNER`.
## 4f — Emit the next-step command (inline, no AskUserQuestion)
Print a fenced code block tailored to `DB` + `RUNNER`:
```bash
# Load DB URL from SSoT (RULE 0.8)
set -a && source secrets/db.env && set +a
# Preview pending migrations
kei-migrate --database-url "$DATABASE_URL" --dir migrations status
# Apply
kei-migrate --database-url "$DATABASE_URL" --dir migrations up
# Revert the latest (dev only!)
kei-migrate --database-url "$DATABASE_URL" --dir migrations down 1
```
Reminder (once): `secrets/db.env` must be `chmod 600` and listed in
`.gitignore` BEFORE the first write. Template entry:
```bash
# secrets/db.env — chmod 600 before first write
DATABASE_URL=
```
No values. RULE 0.8 secrets SSoT.
## Verify-criterion
- `migrations/<ts>_init_schema.sql` exists and equals `db/schema.sql` body
with the one-line header prepended.
- `migrations/<ts>_init_schema.down.sql` exists with DROP statements in
reverse dependency order.
- Filenames use the `kei-migrate create` timestamp convention.
- If `ORM ∈ {Prisma, SQLAlchemy}` — kei-migrate files are NOT created;
instead record a one-line handoff in state: "use `<native runner>`
schema.sql is the design SSoT, port it to the native format."
- Reminder about `secrets/db.env` emitted exactly once.

View file

@ -0,0 +1,92 @@
# Phase 5 — Optional seed data + test fixtures
Emit `db/seed.sql` with deterministic, safe-to-re-run seed rows, OR skip
this phase entirely. Single `AskUserQuestion` to decide.
## 5a — Seed decision (AskUserQuestion)
```json
{
"questions": [
{
"question": "Seed data for dev / tests?",
"header": "Seed",
"multiSelect": false,
"options": [
{"label": "Smoke fixture (1 row per entity)", "description": "Minimal — proves schema loads and FKs resolve. Recommended default."},
{"label": "Rich dev seed (520 rows per entity)", "description": "Realistic playground for local dev; deterministic IDs from a fixed seed"},
{"label": "Test fixtures only (for integration tests)", "description": "Small, labelled datasets keyed by test-case name"},
{"label": "Skip — no seed data", "description": "Schema-only delivery; tests will use mocks or runtime factories"}
]
}
]
}
```
Store as `SEED`.
## 5b — Generate `db/seed.sql` (inline, no AskUserQuestion)
Rules, regardless of choice (unless Skip):
- **Idempotent** — every `INSERT` uses `ON CONFLICT DO NOTHING` (PG/MySQL)
or `INSERT OR IGNORE` (SQLite). Re-running seed is safe.
- **Deterministic PKs** — use explicit IDs (`1`, `2`, ...) not relying on
sequences. For UUIDs, use fixed values from a documented seed (e.g.
`uuid_generate_v5(...)` or hard-coded dev-only UUIDs).
- **No secrets** — no real emails (use `user1@example.test`), no real
phone numbers, no real names. RULE 0.8 still applies: nothing in seed
should be or look like a production token.
- **Respect FK order** — insert parents before children, junctions last.
- **One file**`db/seed.sql`. Not split per entity (ordering matters).
Smoke-fixture shape:
```sql
-- Generated by /schema-design Phase 5 — <YYYY-MM-DD>
-- Smoke fixture: one row per entity, deterministic IDs.
INSERT INTO users (id, email, name) VALUES (1, 'user1@example.test', 'Seed User')
ON CONFLICT (id) DO NOTHING;
INSERT INTO organizations (id, name) VALUES (1, 'Seed Org')
ON CONFLICT (id) DO NOTHING;
INSERT INTO organization_users (user_id, organization_id, role)
VALUES (1, 1, 'owner') ON CONFLICT DO NOTHING;
```
For "Rich dev seed" — use counted loops in SQL (PG: `generate_series`;
SQLite: recursive CTE; MySQL: `WITH RECURSIVE`) to produce 520 rows per
non-junction entity, with FK references wrapping modulo the parent count.
For "Test fixtures" — group by test name via SQL comments:
```sql
-- fixture: test_user_signup
INSERT INTO users (id, email, name) VALUES (101, 'signup@example.test', 'Signup') ...
-- fixture: test_org_invite
INSERT INTO users (id, email, name) VALUES (201, 'invitee@example.test', 'Invitee') ...
```
## 5c — Test-First hook (inline)
If `SEED ≠ Skip`, emit a smoke-test snippet tailored to `DB`:
```bash
# Smoke-test: load schema + seed, assert row counts.
kei-migrate --database-url "$DATABASE_URL" --dir migrations up
psql "$DATABASE_URL" -f db/seed.sql # or: sqlite3 <file> < db/seed.sql
psql "$DATABASE_URL" -c "SELECT count(*) FROM users;"
```
Remind the user: this is a smoke test, not a Test-First contract. Real
integration tests live in the project's test suite — see
`_blocks/rule-test-first.md`.
## Verify-criterion
- If `SEED = Skip`: `db/seed.sql` is NOT created; state records "seed
skipped".
- Otherwise: `db/seed.sql` exists, is idempotent, respects FK order, and
uses `@example.test` (or equivalent RFC 2606 reserved) emails only.
- Smoke-test snippet printed inline in chat (once).
- Final report can now be emitted (see `SKILL.md` index).