Merge branch 'feat/v0.6-database' — 5 blocks + kei-migrate Rust + /schema-design
This commit is contained in:
commit
f205a12348
25 changed files with 3891 additions and 0 deletions
51
_blocks/db-drizzle.md
Normal file
51
_blocks/db-drizzle.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
# DB — Drizzle ORM (TypeScript) patterns
|
||||
|
||||
Use when the project is TypeScript/Next.js/Bun/Node and needs a type-safe SQL layer without Prisma's heavyweight engine process. Pairs with `stack-nextjs`. [E4 — expert assessment]
|
||||
|
||||
**Core versions:** `drizzle-orm` (latest on npm) + `drizzle-kit` (migrations CLI) as of 2026-04. Peer-deps: `pg` for Postgres, `better-sqlite3` / `@libsql/client` for SQLite, `mysql2` for MySQL. [UNVERIFIED: pin exact versions from npm before shipping]
|
||||
|
||||
**Schema-first, not code-first:**
|
||||
```ts
|
||||
// db/schema.ts
|
||||
import { pgTable, serial, text, timestamp, integer } from "drizzle-orm/pg-core";
|
||||
|
||||
export const users = pgTable("users", {
|
||||
id: serial("id").primaryKey(),
|
||||
email: text("email").notNull().unique(),
|
||||
createdAt: timestamp("created_at").defaultNow().notNull(),
|
||||
});
|
||||
|
||||
export const posts = pgTable("posts", {
|
||||
id: serial("id").primaryKey(),
|
||||
authorId: integer("author_id").references(() => users.id).notNull(),
|
||||
body: text("body").notNull(),
|
||||
});
|
||||
```
|
||||
`schema.ts` IS the source of truth. All types flow from it — `typeof users.$inferSelect` gives you the row type.
|
||||
|
||||
**Query with full inference:**
|
||||
```ts
|
||||
import { eq } from "drizzle-orm";
|
||||
const rows = await db.select().from(users).where(eq(users.id, 1));
|
||||
// rows: { id: number; email: string; createdAt: Date }[]
|
||||
```
|
||||
No codegen step, no separate `.prisma` file. Type errors surface in the IDE immediately.
|
||||
|
||||
**Migrations via drizzle-kit:**
|
||||
```bash
|
||||
drizzle-kit generate # diff schema.ts against prev snapshot → emit SQL in drizzle/
|
||||
drizzle-kit migrate # apply pending migrations
|
||||
drizzle-kit studio # local web UI to inspect data
|
||||
```
|
||||
Config in `drizzle.config.ts` — specify `dialect`, `schema`, `out`, `dbCredentials`.
|
||||
|
||||
**Connection / pool:**
|
||||
```ts
|
||||
import { drizzle } from "drizzle-orm/node-postgres";
|
||||
import { Pool } from "pg";
|
||||
const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 20 });
|
||||
export const db = drizzle(pool, { schema });
|
||||
```
|
||||
Serverless (Vercel / CF Workers): use `neon-serverless` or `@libsql/client` driver instead — the `pg` Pool doesn't survive cold-start boundaries.
|
||||
|
||||
**Forbidden:** template-string SQL with untrusted input (`sql\`SELECT * WHERE x = ${userInput}\`` — use `sql.placeholder` or the query builder); committing `drizzle/meta/_journal.json` conflicts (merge manually or regenerate); mixing drizzle-kit versions across dev machines.
|
||||
33
_blocks/db-migration-hygiene.md
Normal file
33
_blocks/db-migration-hygiene.md
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# DB — Migration hygiene (universal)
|
||||
|
||||
Applies to every migration tool — `kei-migrate`, Atlas, goose, sqlx-cli, drizzle-kit, Alembic, Prisma migrate, Ecto migrations. [E4 — expert assessment]
|
||||
|
||||
**Numbering:** timestamp prefix, not integer. `20260421_120000_add_users_email_index.sql` sorts correctly forever and doesn't collide on parallel branches. Integer sequences (`0001_`, `0002_`) collide on merge; reject them in review.
|
||||
|
||||
**Up + down pairs:** every migration has a reverse. If the reverse is destructive and unsafe (e.g. dropping a column with data), write a `-- IRREVERSIBLE` comment and stop the down-script there. NEVER auto-run destructive downs on prod without a human click.
|
||||
|
||||
**Idempotent where possible:**
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS users (...);
|
||||
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
|
||||
ALTER TABLE users ADD COLUMN IF NOT EXISTS bio TEXT; -- PG 9.6+, verify per-DB
|
||||
```
|
||||
Re-running a partially-applied migration should be safe. A migration that crashes mid-way and can't be re-run = 2AM incident waiting to happen.
|
||||
|
||||
**Zero-downtime pattern (add-then-drop):**
|
||||
1. Deploy migration that ADDS new column / table (old code still works).
|
||||
2. Deploy app code that writes BOTH old + new.
|
||||
3. Backfill old → new.
|
||||
4. Deploy app code that reads new, ignores old.
|
||||
5. Deploy migration that DROPS old column.
|
||||
|
||||
Never `DROP` + `ADD RENAME` in one migration on a live table. That's a table lock + app-downtime event.
|
||||
|
||||
**Backfill patterns:**
|
||||
- Small table (< 1M rows): `UPDATE ... SET new = f(old)` in a single migration.
|
||||
- Large table: background job with batched `UPDATE ... WHERE id BETWEEN ? AND ?` + `LIMIT`. Commit per batch. Monitor lag.
|
||||
- Very large (> 100M rows): use the DB's native tooling (PG `VACUUM FULL` not needed; `pg_repack` if column-add bloats). [UNVERIFIED: verify on current PG docs]
|
||||
|
||||
**Tracking table (`_kei_migrations` or equivalent):** stores (version, name, checksum, applied_at). Checksum prevents silent tampering with an already-applied file. If checksum mismatches on an applied migration → hard-fail, demand human intervention.
|
||||
|
||||
**Forbidden:** editing a migration file after it's been applied on any environment (checksum break); `DROP TABLE` without backup + 24h cooldown; mixing DDL + large DML in one transaction (long locks); running migrations automatically on app startup in multi-replica deploys without a leader-election guard (every replica tries to apply = race condition).
|
||||
28
_blocks/db-postgres.md
Normal file
28
_blocks/db-postgres.md
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# DB — PostgreSQL (current major — 17 as of 2026-04) patterns
|
||||
|
||||
Use when the project needs relational integrity, concurrent writes, or server-side indexing power that SQLite can't match. Default RDBMS for new multi-user services. [E4 — expert assessment]
|
||||
|
||||
**Version choice:** PostgreSQL 17 for new projects (current GA line, improved vacuum, JSON_TABLE, better parallel index builds). PostgreSQL 16 acceptable if hosting provider pins it. [UNVERIFIED: exact feature matrix — verify on postgresql.org/docs before committing to a minor-version-specific feature]
|
||||
|
||||
**Schema migrations:** every schema change ships as a numbered `.sql` file, never `ALTER TABLE` on prod. Use `kei-migrate` (this kit) or Atlas/goose/sqlx-cli — see `db-migration-hygiene.md`. One migration per logical change; no mega-migrations.
|
||||
|
||||
**Indexing:**
|
||||
- B-tree default for equality + range. `CREATE INDEX CONCURRENTLY` on prod to avoid table lock.
|
||||
- `GIN` for `jsonb` / array / full-text (`tsvector`).
|
||||
- `BRIN` only for massive append-only time-series (orders of magnitude smaller than B-tree).
|
||||
- Partial indexes (`WHERE active = true`) for sparse predicates.
|
||||
- **Verify with `EXPLAIN (ANALYZE, BUFFERS)`** before declaring an index necessary. No blind indexing.
|
||||
|
||||
**Connection pooling:** app-side connection pool is NOT enough at scale. Use:
|
||||
- **PgBouncer** (transaction mode) for most services — battle-tested, low overhead.
|
||||
- **Supavisor** if already on Supabase — serverless-friendly, wire-compatible. [E4]
|
||||
- Native server pooling (PG 17's improved but still not a substitute). [UNVERIFIED]
|
||||
|
||||
Sizing rule of thumb: `max_connections` on server × 1 pool layer. Don't stack pools (pool → PgBouncer → PG = deadlock risk).
|
||||
|
||||
**Backup:**
|
||||
- Logical: `pg_dump` nightly for schema + data portability.
|
||||
- Physical: `pg_basebackup` + WAL archiving (`archive_command`) for PITR.
|
||||
- Managed service (RDS / Supabase / Neon) — verify backup retention in their UI, don't assume.
|
||||
|
||||
**Forbidden:** `SELECT *` in hot paths (N+1 + column drift); unindexed FK columns (join explosion); `SERIAL` on new tables — prefer `GENERATED ALWAYS AS IDENTITY` (SQL standard, PG 10+); plaintext passwords in `pg_hba.conf`; committing `.env` with DB URL.
|
||||
34
_blocks/db-sqlite.md
Normal file
34
_blocks/db-sqlite.md
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
# DB — SQLite (prod-suitable) patterns
|
||||
|
||||
Use when the workload is read-heavy, single-writer-acceptable, or needs zero-ops embedded storage. SQLite is prod-suitable — Fly.io, Turso, Cloudflare D1, and countless CLI/mobile apps run it in production. [E4 — expert assessment]
|
||||
|
||||
**When NOT to use:** high-concurrency write workload (> ~1 writer/sec sustained), multi-region strong consistency, horizontal write scaling. Use Postgres instead.
|
||||
|
||||
**WAL mode is mandatory for prod:**
|
||||
```sql
|
||||
PRAGMA journal_mode = WAL; -- readers don't block writer, writer doesn't block readers
|
||||
PRAGMA synchronous = NORMAL; -- durable across app crash, NOT across power loss (use FULL if PSU-risk)
|
||||
PRAGMA busy_timeout = 5000; -- 5s wait for lock instead of instant SQLITE_BUSY
|
||||
PRAGMA foreign_keys = ON; -- default OFF in SQLite (!), always enable
|
||||
PRAGMA temp_store = MEMORY;
|
||||
```
|
||||
Apply these on every connection open — they are per-connection, not per-database (except `journal_mode` which persists).
|
||||
|
||||
**Distributed patterns:**
|
||||
- **Turso** (libSQL fork): edge-replicated read replicas with HTTP/WebSocket wire protocol. Primary single-writer, replicas read-only. [E4]
|
||||
- **LiteFS** (Fly.io): file-system replication, leader-election via Consul. Primary+replicas. [E4]
|
||||
- **Cloudflare D1**: managed SQLite on edge with their own replication. [UNVERIFIED: current throughput limits]
|
||||
- **Litestream**: continuous replication to S3/R2 for backup + PITR; single node, not HA.
|
||||
|
||||
**Full-text search (FTS5):**
|
||||
```sql
|
||||
CREATE VIRTUAL TABLE docs_fts USING fts5(title, body, content=docs, content_rowid=id);
|
||||
CREATE TRIGGER docs_ai AFTER INSERT ON docs BEGIN
|
||||
INSERT INTO docs_fts(rowid, title, body) VALUES (new.id, new.title, new.body);
|
||||
END;
|
||||
```
|
||||
FTS5 outperforms bolt-on `LIKE '%x%'` by 100×+ on large text corpora. Native, no extension install.
|
||||
|
||||
**Backup:** `sqlite3 db '.backup /path/backup.db'` while app runs (safe with WAL). Or Litestream for continuous.
|
||||
|
||||
**Forbidden:** multiple writer processes without a coordination layer; opening the same DB over NFS (lock semantics broken); `DELETE FROM bigtable` without `VACUUM` after (doesn't shrink file); committing the `.db` / `.db-wal` / `.db-shm` files to git.
|
||||
39
_blocks/db-sqlx.md
Normal file
39
_blocks/db-sqlx.md
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
# DB — SQLx (Rust) patterns
|
||||
|
||||
Use when the project is Rust and needs a SQL-first (not ORM) query layer with compile-time checking. Pairs with `stack-rust-axum`, `stack-rust-cli`. [E4 — expert assessment]
|
||||
|
||||
**Core versions:** `sqlx = "0.8"` (current as of 2026-04) with features `runtime-tokio`, `tls-rustls`, and one of `postgres` / `sqlite` / `mysql`. Never mix `runtime-async-std` and `runtime-tokio` — they clash at link time. [UNVERIFIED: verify latest on crates.io before pinning]
|
||||
|
||||
**Compile-time checked queries:**
|
||||
```rust
|
||||
let row = sqlx::query!("SELECT id, name FROM users WHERE id = $1", user_id)
|
||||
.fetch_one(&pool).await?;
|
||||
```
|
||||
Requires either:
|
||||
- `DATABASE_URL` env set during `cargo build` (live DB) — convenient in dev, brittle in CI.
|
||||
- **Offline mode** (recommended for CI): `cargo sqlx prepare` commits `.sqlx/query-*.json` to the repo, then CI builds with `SQLX_OFFLINE=true` and no DB access.
|
||||
|
||||
**Connection pool:**
|
||||
```rust
|
||||
let pool = sqlx::postgres::PgPoolOptions::new()
|
||||
.max_connections(20) // tune to server max_connections / replica count
|
||||
.acquire_timeout(Duration::from_secs(3))
|
||||
.connect(&database_url).await?;
|
||||
```
|
||||
Single `PgPool` per process, `Arc`-cloned into handlers. Don't open per-request.
|
||||
|
||||
**Migrations:**
|
||||
```rust
|
||||
sqlx::migrate!("./migrations").run(&pool).await?;
|
||||
```
|
||||
Built-in runner reads `YYYYMMDDHHMMSS_<name>.sql` files. For richer UX (up/down, status, create scaffolding) use the `kei-migrate` primitive in this kit.
|
||||
|
||||
**Transactions:**
|
||||
```rust
|
||||
let mut tx = pool.begin().await?;
|
||||
sqlx::query!("...").execute(&mut *tx).await?;
|
||||
sqlx::query!("...").execute(&mut *tx).await?;
|
||||
tx.commit().await?; // explicit; Drop = rollback
|
||||
```
|
||||
|
||||
**Forbidden:** `sqlx::query` (non-macro) with untrusted input without `bind()` — that's string concat, i.e. SQL injection; `.unwrap()` on DB calls in prod paths; enabling both `runtime-tokio` and `runtime-async-std`; committing a live `DATABASE_URL` to `.env.example`.
|
||||
1
_primitives/_rust/kei-migrate/.gitignore
vendored
Normal file
1
_primitives/_rust/kei-migrate/.gitignore
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
/target
|
||||
2344
_primitives/_rust/kei-migrate/Cargo.lock
generated
Normal file
2344
_primitives/_rust/kei-migrate/Cargo.lock
generated
Normal file
File diff suppressed because it is too large
Load diff
28
_primitives/_rust/kei-migrate/Cargo.toml
Normal file
28
_primitives/_rust/kei-migrate/Cargo.toml
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
[package]
|
||||
name = "kei-migrate"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
description = "Universal SQL migration runner — Postgres/SQLite/MySQL autodetect from DATABASE_URL"
|
||||
license = "MIT"
|
||||
|
||||
[[bin]]
|
||||
name = "kei-migrate"
|
||||
path = "src/main.rs"
|
||||
|
||||
[dependencies]
|
||||
anyhow = "1"
|
||||
chrono = { version = "0.4", default-features = false, features = ["clock"] }
|
||||
clap = { version = "4", features = ["derive", "env"] }
|
||||
sha2 = "0.10"
|
||||
sqlx = { version = "0.8", default-features = false, features = [
|
||||
"runtime-tokio",
|
||||
"tls-rustls",
|
||||
"any",
|
||||
"postgres",
|
||||
"sqlite",
|
||||
"mysql",
|
||||
] }
|
||||
tokio = { version = "1", features = ["rt-multi-thread", "macros", "fs"] }
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
44
_primitives/_rust/kei-migrate/src/cli.rs
Normal file
44
_primitives/_rust/kei-migrate/src/cli.rs
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
//! CLI surface — clap argument parsing for `kei-migrate`.
|
||||
|
||||
use clap::{Parser, Subcommand};
|
||||
|
||||
#[derive(Parser, Debug)]
|
||||
#[command(
|
||||
name = "kei-migrate",
|
||||
about = "Universal SQL migration runner (Postgres / SQLite / MySQL)",
|
||||
version
|
||||
)]
|
||||
pub struct Cli {
|
||||
/// Database URL. Overrides $DATABASE_URL.
|
||||
/// Formats:
|
||||
/// postgres://user:pass@host:port/db
|
||||
/// sqlite:///absolute/path.db or sqlite::memory:
|
||||
/// mysql://user:pass@host:port/db
|
||||
#[arg(long, env = "DATABASE_URL")]
|
||||
pub database_url: String,
|
||||
|
||||
/// Migrations directory (default: ./migrations)
|
||||
#[arg(long, default_value = "migrations")]
|
||||
pub dir: String,
|
||||
|
||||
#[command(subcommand)]
|
||||
pub command: Command,
|
||||
}
|
||||
|
||||
#[derive(Subcommand, Debug)]
|
||||
pub enum Command {
|
||||
/// Apply all pending migrations.
|
||||
Up,
|
||||
/// Revert the last N migrations (requires <ts>_<name>.down.sql).
|
||||
Down {
|
||||
#[arg(default_value_t = 1)]
|
||||
n: u32,
|
||||
},
|
||||
/// List applied vs pending migrations.
|
||||
Status,
|
||||
/// Create a new timestamped migration scaffold: <ts>_<name>.sql (+ .down.sql).
|
||||
Create {
|
||||
/// Short migration name, e.g. "add_users_email_index".
|
||||
name: String,
|
||||
},
|
||||
}
|
||||
44
_primitives/_rust/kei-migrate/src/cmd_create.rs
Normal file
44
_primitives/_rust/kei-migrate/src/cmd_create.rs
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
//! `kei-migrate create <name>` — scaffold a new timestamped migration pair.
|
||||
|
||||
use anyhow::{bail, Context, Result};
|
||||
use chrono::Utc;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
const UP_TEMPLATE: &str = "-- up migration\n-- Write forward-direction SQL below.\n\n";
|
||||
const DOWN_TEMPLATE: &str =
|
||||
"-- down migration\n-- Write reverse SQL below, or add `-- IRREVERSIBLE` to block reversion.\n\n";
|
||||
|
||||
/// Create `<dir>/<utc-timestamp>_<sanitized-name>.sql` + `.down.sql`. Returns paths written.
|
||||
pub fn run(dir: &Path, name: &str) -> Result<(PathBuf, PathBuf)> {
|
||||
validate_name(name)?;
|
||||
fs::create_dir_all(dir).with_context(|| format!("mkdir -p {}", dir.display()))?;
|
||||
let ts = Utc::now().format("%Y%m%d%H%M%S").to_string();
|
||||
let sanitized = sanitize(name);
|
||||
let up = dir.join(format!("{}_{}.sql", ts, sanitized));
|
||||
let down = dir.join(format!("{}_{}.down.sql", ts, sanitized));
|
||||
if up.exists() || down.exists() {
|
||||
bail!("collision: {} or {} already exists", up.display(), down.display());
|
||||
}
|
||||
fs::write(&up, UP_TEMPLATE)?;
|
||||
fs::write(&down, DOWN_TEMPLATE)?;
|
||||
println!("[create] {}", up.display());
|
||||
println!("[create] {}", down.display());
|
||||
Ok((up, down))
|
||||
}
|
||||
|
||||
fn validate_name(name: &str) -> Result<()> {
|
||||
if name.is_empty() {
|
||||
bail!("migration name must not be empty");
|
||||
}
|
||||
if name.len() > 80 {
|
||||
bail!("migration name too long ({} chars, max 80)", name.len());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn sanitize(name: &str) -> String {
|
||||
name.chars()
|
||||
.map(|c| if c.is_ascii_alphanumeric() { c.to_ascii_lowercase() } else { '_' })
|
||||
.collect()
|
||||
}
|
||||
58
_primitives/_rust/kei-migrate/src/cmd_down.rs
Normal file
58
_primitives/_rust/kei-migrate/src/cmd_down.rs
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
//! `kei-migrate down [n]` — revert the last N applied migrations.
|
||||
//!
|
||||
//! Requires a sibling `<version>_<name>.down.sql` for each target. Missing
|
||||
//! down-file = hard error — we don't guess reversals.
|
||||
|
||||
use crate::discover::Migration;
|
||||
use crate::tracker;
|
||||
use anyhow::{bail, Context, Result};
|
||||
use sqlx::AnyPool;
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Revert the last `n` applied migrations in reverse order.
|
||||
pub async fn run(pool: &AnyPool, migrations: &[Migration], n: u32) -> Result<u32> {
|
||||
let mut applied: Vec<i64> = tracker::applied_versions(pool).await?;
|
||||
applied.sort_unstable();
|
||||
applied.reverse(); // newest first
|
||||
let by_version: HashMap<i64, &Migration> =
|
||||
migrations.iter().map(|m| (m.version, m)).collect();
|
||||
let mut reverted = 0u32;
|
||||
for v in applied.into_iter().take(n as usize) {
|
||||
let m = by_version.get(&v).with_context(|| {
|
||||
format!("applied version {} has no matching file on disk", v)
|
||||
})?;
|
||||
revert_one(pool, m).await?;
|
||||
reverted += 1;
|
||||
println!("[down] {} {} — reverted", m.version, m.name);
|
||||
}
|
||||
Ok(reverted)
|
||||
}
|
||||
|
||||
async fn revert_one(pool: &AnyPool, m: &Migration) -> Result<()> {
|
||||
let down_path = m.down_path.as_ref().ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"no down-sql for migration {} ({}) — create {}_{}.down.sql",
|
||||
m.version,
|
||||
m.name,
|
||||
m.version,
|
||||
m.name
|
||||
)
|
||||
})?;
|
||||
let sql = std::fs::read_to_string(down_path)
|
||||
.with_context(|| format!("read {}", down_path.display()))?;
|
||||
if sql.contains("-- IRREVERSIBLE") {
|
||||
bail!(
|
||||
"migration {} ({}) is marked IRREVERSIBLE — refusing to run down-sql",
|
||||
m.version,
|
||||
m.name
|
||||
);
|
||||
}
|
||||
let mut tx = pool.begin().await?;
|
||||
sqlx::raw_sql(&sql)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.with_context(|| format!("revert migration {} ({})", m.version, m.name))?;
|
||||
tx.commit().await?;
|
||||
tracker::record_down(pool, m.version).await?;
|
||||
Ok(())
|
||||
}
|
||||
29
_primitives/_rust/kei-migrate/src/cmd_status.rs
Normal file
29
_primitives/_rust/kei-migrate/src/cmd_status.rs
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
//! `kei-migrate status` — list applied + pending migrations.
|
||||
|
||||
use crate::discover::Migration;
|
||||
use crate::tracker;
|
||||
use anyhow::Result;
|
||||
use sqlx::AnyPool;
|
||||
use std::collections::HashSet;
|
||||
|
||||
/// Print a human-readable table. Returns (applied_count, pending_count).
|
||||
pub async fn run(pool: &AnyPool, migrations: &[Migration]) -> Result<(u32, u32)> {
|
||||
let applied: HashSet<i64> = tracker::applied_versions(pool).await?.into_iter().collect();
|
||||
let mut a = 0u32;
|
||||
let mut p = 0u32;
|
||||
println!("{:>14} {:<8} name", "version", "status");
|
||||
println!("{:>14} {:<8} ----", "-------", "------");
|
||||
for m in migrations {
|
||||
let status = if applied.contains(&m.version) {
|
||||
a += 1;
|
||||
"APPLIED"
|
||||
} else {
|
||||
p += 1;
|
||||
"PENDING"
|
||||
};
|
||||
println!("{:>14} {:<8} {}", m.version, status, m.name);
|
||||
}
|
||||
println!();
|
||||
println!("{} applied, {} pending", a, p);
|
||||
Ok((a, p))
|
||||
}
|
||||
42
_primitives/_rust/kei-migrate/src/cmd_up.rs
Normal file
42
_primitives/_rust/kei-migrate/src/cmd_up.rs
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
//! `kei-migrate up` — apply all pending migrations in version-ASC order.
|
||||
|
||||
use crate::discover::Migration;
|
||||
use crate::tracker;
|
||||
use anyhow::{Context, Result};
|
||||
use chrono::Utc;
|
||||
use sqlx::AnyPool;
|
||||
use std::collections::HashSet;
|
||||
|
||||
/// Apply every migration whose version is not in the applied set.
|
||||
/// Each migration runs in its own transaction; failure aborts and leaves
|
||||
/// prior applied migrations committed.
|
||||
pub async fn run(pool: &AnyPool, migrations: &[Migration]) -> Result<u32> {
|
||||
let applied: HashSet<i64> = tracker::applied_versions(pool).await?.into_iter().collect();
|
||||
let on_disk: Vec<(i64, &str, &str)> = migrations
|
||||
.iter()
|
||||
.map(|m| (m.version, m.name.as_str(), m.checksum.as_str()))
|
||||
.collect();
|
||||
tracker::verify_checksums(pool, on_disk).await?;
|
||||
let mut count = 0u32;
|
||||
for m in migrations {
|
||||
if applied.contains(&m.version) {
|
||||
continue;
|
||||
}
|
||||
apply_one(pool, m).await?;
|
||||
count += 1;
|
||||
println!("[up] {} {} — applied", m.version, m.name);
|
||||
}
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
async fn apply_one(pool: &AnyPool, m: &Migration) -> Result<()> {
|
||||
let mut tx = pool.begin().await?;
|
||||
sqlx::raw_sql(&m.up_sql)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.with_context(|| format!("apply migration {} ({})", m.version, m.name))?;
|
||||
tx.commit().await?;
|
||||
let now = Utc::now().to_rfc3339();
|
||||
tracker::record_up(pool, m.version, &m.name, &m.checksum, &now).await?;
|
||||
Ok(())
|
||||
}
|
||||
68
_primitives/_rust/kei-migrate/src/db.rs
Normal file
68
_primitives/_rust/kei-migrate/src/db.rs
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
//! Database backend detection + pool construction.
|
||||
//!
|
||||
//! Uses `sqlx::Any` so one binary covers Postgres / SQLite / MySQL.
|
||||
//! Detection is purely on URL scheme — no live probe needed.
|
||||
|
||||
use anyhow::{bail, Result};
|
||||
use sqlx::any::{install_default_drivers, AnyPoolOptions};
|
||||
use sqlx::AnyPool;
|
||||
|
||||
/// Backend inferred from the URL scheme. Determines dialect quirks.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Backend {
|
||||
Postgres,
|
||||
Sqlite,
|
||||
Mysql,
|
||||
}
|
||||
|
||||
impl Backend {
|
||||
/// Backend-specific CREATE TABLE for `_kei_migrations`.
|
||||
pub fn create_tracker_sql(self) -> &'static str {
|
||||
match self {
|
||||
Backend::Postgres | Backend::Mysql => {
|
||||
"CREATE TABLE IF NOT EXISTS _kei_migrations (
|
||||
version BIGINT PRIMARY KEY,
|
||||
name VARCHAR(255) NOT NULL,
|
||||
checksum CHAR(64) NOT NULL,
|
||||
applied_at VARCHAR(32) NOT NULL
|
||||
)"
|
||||
}
|
||||
Backend::Sqlite => {
|
||||
"CREATE TABLE IF NOT EXISTS _kei_migrations (
|
||||
version INTEGER PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
checksum TEXT NOT NULL,
|
||||
applied_at TEXT NOT NULL
|
||||
)"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a database URL into a [`Backend`]. Never touches the network.
|
||||
pub fn detect_backend(url: &str) -> Result<Backend> {
|
||||
let lower = url.to_ascii_lowercase();
|
||||
if lower.starts_with("postgres://") || lower.starts_with("postgresql://") {
|
||||
Ok(Backend::Postgres)
|
||||
} else if lower.starts_with("sqlite:") {
|
||||
Ok(Backend::Sqlite)
|
||||
} else if lower.starts_with("mysql://") || lower.starts_with("mariadb://") {
|
||||
Ok(Backend::Mysql)
|
||||
} else {
|
||||
bail!(
|
||||
"unsupported or unrecognised DATABASE_URL scheme: {}. \
|
||||
Expected postgres://, sqlite:, or mysql://",
|
||||
url
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a sqlx `AnyPool` for the given URL (max 4 conns — migration runner is not a server).
|
||||
pub async fn connect(url: &str) -> Result<AnyPool> {
|
||||
install_default_drivers();
|
||||
let pool = AnyPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.connect(url)
|
||||
.await?;
|
||||
Ok(pool)
|
||||
}
|
||||
87
_primitives/_rust/kei-migrate/src/discover.rs
Normal file
87
_primitives/_rust/kei-migrate/src/discover.rs
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
//! Filesystem migration discovery.
|
||||
//!
|
||||
//! Convention: `migrations/<version>_<name>.sql` (up) and optional
|
||||
//! `migrations/<version>_<name>.down.sql` (down). Version is a monotonic
|
||||
//! integer, typically a UTC timestamp like `20260421120000`.
|
||||
|
||||
use anyhow::{bail, Context, Result};
|
||||
use sha2::{Digest, Sha256};
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// One discovered migration (up-side). `down_path` is `Some` iff the sibling file exists.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Migration {
|
||||
pub version: i64,
|
||||
pub name: String,
|
||||
pub up_path: PathBuf,
|
||||
pub down_path: Option<PathBuf>,
|
||||
pub up_sql: String,
|
||||
pub checksum: String,
|
||||
}
|
||||
|
||||
/// Read every `<version>_<name>.sql` file (ignoring `.down.sql`), sort by version ASC.
|
||||
pub fn scan(dir: &Path) -> Result<Vec<Migration>> {
|
||||
if !dir.exists() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
for entry in fs::read_dir(dir).with_context(|| format!("read_dir {}", dir.display()))? {
|
||||
let entry = entry?;
|
||||
let path = entry.path();
|
||||
if path.extension().and_then(|s| s.to_str()) != Some("sql") {
|
||||
continue;
|
||||
}
|
||||
let fname = path.file_name().and_then(|s| s.to_str()).unwrap_or("");
|
||||
if fname.ends_with(".down.sql") {
|
||||
continue;
|
||||
}
|
||||
let m = parse_migration(&path)?;
|
||||
out.push(m);
|
||||
}
|
||||
out.sort_by_key(|m| m.version);
|
||||
check_unique(&out)?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn parse_migration(path: &Path) -> Result<Migration> {
|
||||
let stem = path
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.context("non-utf8 filename")?;
|
||||
let (ver_str, name) = stem
|
||||
.split_once('_')
|
||||
.with_context(|| format!("filename not <version>_<name>.sql: {}", stem))?;
|
||||
let version: i64 = ver_str
|
||||
.parse()
|
||||
.with_context(|| format!("version must be integer, got {}", ver_str))?;
|
||||
let up_sql = fs::read_to_string(path)
|
||||
.with_context(|| format!("read {}", path.display()))?;
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(up_sql.as_bytes());
|
||||
let checksum = format!("{:x}", hasher.finalize());
|
||||
let down_path = path.with_file_name(format!("{}_{}.down.sql", version, name));
|
||||
let down = if down_path.exists() { Some(down_path) } else { None };
|
||||
Ok(Migration {
|
||||
version,
|
||||
name: name.to_string(),
|
||||
up_path: path.to_path_buf(),
|
||||
down_path: down,
|
||||
up_sql,
|
||||
checksum,
|
||||
})
|
||||
}
|
||||
|
||||
fn check_unique(migs: &[Migration]) -> Result<()> {
|
||||
for w in migs.windows(2) {
|
||||
if w[0].version == w[1].version {
|
||||
bail!(
|
||||
"duplicate migration version {} ({} and {})",
|
||||
w[0].version,
|
||||
w[0].up_path.display(),
|
||||
w[1].up_path.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
54
_primitives/_rust/kei-migrate/src/lib.rs
Normal file
54
_primitives/_rust/kei-migrate/src/lib.rs
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
//! kei-migrate — universal SQL migration runner.
|
||||
//!
|
||||
//! Single binary, three backends (Postgres / SQLite / MySQL) autodetected
|
||||
//! from `DATABASE_URL`. Sequential `.sql` files in `migrations/`, tracked in
|
||||
//! `_kei_migrations` with SHA-256 checksums.
|
||||
//!
|
||||
//! Library surface exists so integration tests can drive the primitive
|
||||
//! without `process::Command` gymnastics.
|
||||
|
||||
pub mod cli;
|
||||
pub mod cmd_create;
|
||||
pub mod cmd_down;
|
||||
pub mod cmd_status;
|
||||
pub mod cmd_up;
|
||||
pub mod db;
|
||||
pub mod discover;
|
||||
pub mod tracker;
|
||||
|
||||
use anyhow::Result;
|
||||
use std::path::Path;
|
||||
|
||||
/// End-to-end `up` entry: connect, ensure tracker, scan dir, apply pending.
|
||||
/// Returns number of migrations applied.
|
||||
pub async fn do_up(database_url: &str, dir: &Path) -> Result<u32> {
|
||||
let backend = db::detect_backend(database_url)?;
|
||||
let pool = db::connect(database_url).await?;
|
||||
tracker::ensure_table(&pool, backend).await?;
|
||||
let migs = discover::scan(dir)?;
|
||||
let n = cmd_up::run(&pool, &migs).await?;
|
||||
pool.close().await;
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// End-to-end `down` entry: revert last N applied.
|
||||
pub async fn do_down(database_url: &str, dir: &Path, n: u32) -> Result<u32> {
|
||||
let backend = db::detect_backend(database_url)?;
|
||||
let pool = db::connect(database_url).await?;
|
||||
tracker::ensure_table(&pool, backend).await?;
|
||||
let migs = discover::scan(dir)?;
|
||||
let reverted = cmd_down::run(&pool, &migs, n).await?;
|
||||
pool.close().await;
|
||||
Ok(reverted)
|
||||
}
|
||||
|
||||
/// End-to-end `status` entry: returns (applied, pending) counts.
|
||||
pub async fn do_status(database_url: &str, dir: &Path) -> Result<(u32, u32)> {
|
||||
let backend = db::detect_backend(database_url)?;
|
||||
let pool = db::connect(database_url).await?;
|
||||
tracker::ensure_table(&pool, backend).await?;
|
||||
let migs = discover::scan(dir)?;
|
||||
let r = cmd_status::run(&pool, &migs).await?;
|
||||
pool.close().await;
|
||||
Ok(r)
|
||||
}
|
||||
30
_primitives/_rust/kei-migrate/src/main.rs
Normal file
30
_primitives/_rust/kei-migrate/src/main.rs
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
//! kei-migrate CLI entry. Dispatches to the library surface in `lib.rs`.
|
||||
|
||||
use anyhow::Result;
|
||||
use clap::Parser;
|
||||
use kei_migrate::cli::{Cli, Command};
|
||||
use kei_migrate::{cmd_create, do_down, do_status, do_up};
|
||||
use std::path::Path;
|
||||
|
||||
#[tokio::main(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn main() -> Result<()> {
|
||||
let cli = Cli::parse();
|
||||
let dir = Path::new(&cli.dir);
|
||||
match cli.command {
|
||||
Command::Up => {
|
||||
let n = do_up(&cli.database_url, dir).await?;
|
||||
println!("up: {} migration(s) applied", n);
|
||||
}
|
||||
Command::Down { n } => {
|
||||
let r = do_down(&cli.database_url, dir, n).await?;
|
||||
println!("down: {} migration(s) reverted", r);
|
||||
}
|
||||
Command::Status => {
|
||||
do_status(&cli.database_url, dir).await?;
|
||||
}
|
||||
Command::Create { name } => {
|
||||
cmd_create::run(dir, &name)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
86
_primitives/_rust/kei-migrate/src/tracker.rs
Normal file
86
_primitives/_rust/kei-migrate/src/tracker.rs
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
//! `_kei_migrations` tracking table operations.
|
||||
//!
|
||||
//! Row shape: (version, name, checksum, applied_at). Checksum guards against
|
||||
//! silent edits of an applied file — mismatch = hard fail, requires human ack.
|
||||
|
||||
use crate::db::Backend;
|
||||
use anyhow::{bail, Result};
|
||||
use sqlx::{AnyPool, Row};
|
||||
|
||||
/// Create tracker table if missing. Idempotent.
|
||||
pub async fn ensure_table(pool: &AnyPool, backend: Backend) -> Result<()> {
|
||||
sqlx::query(backend.create_tracker_sql()).execute(pool).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Versions of all applied migrations, ASC.
|
||||
pub async fn applied_versions(pool: &AnyPool) -> Result<Vec<i64>> {
|
||||
let rows = sqlx::query("SELECT version FROM _kei_migrations ORDER BY version ASC")
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
let mut out = Vec::with_capacity(rows.len());
|
||||
for r in rows {
|
||||
out.push(r.try_get::<i64, _>(0)?);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Checksum of a specific applied version, or `None` if not applied.
|
||||
pub async fn applied_checksum(pool: &AnyPool, version: i64) -> Result<Option<String>> {
|
||||
let row = sqlx::query("SELECT checksum FROM _kei_migrations WHERE version = $1")
|
||||
.bind(version)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
Ok(row.map(|r| r.try_get::<String, _>(0)).transpose()?)
|
||||
}
|
||||
|
||||
/// Insert a tracker row after a successful up-migration.
|
||||
pub async fn record_up(
|
||||
pool: &AnyPool,
|
||||
version: i64,
|
||||
name: &str,
|
||||
checksum: &str,
|
||||
applied_at: &str,
|
||||
) -> Result<()> {
|
||||
sqlx::query(
|
||||
"INSERT INTO _kei_migrations (version, name, checksum, applied_at) VALUES ($1, $2, $3, $4)",
|
||||
)
|
||||
.bind(version)
|
||||
.bind(name)
|
||||
.bind(checksum)
|
||||
.bind(applied_at)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Delete a tracker row after a successful down-migration.
|
||||
pub async fn record_down(pool: &AnyPool, version: i64) -> Result<()> {
|
||||
sqlx::query("DELETE FROM _kei_migrations WHERE version = $1")
|
||||
.bind(version)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Abort if any applied migration's recorded checksum doesn't match the on-disk file.
|
||||
pub async fn verify_checksums<'a, I>(pool: &AnyPool, on_disk: I) -> Result<()>
|
||||
where
|
||||
I: IntoIterator<Item = (i64, &'a str, &'a str)>, // (version, name, checksum)
|
||||
{
|
||||
for (version, name, disk_sum) in on_disk {
|
||||
if let Some(db_sum) = applied_checksum(pool, version).await? {
|
||||
if db_sum != disk_sum {
|
||||
bail!(
|
||||
"checksum drift on applied migration {} ({}): db={}, disk={}. \
|
||||
Refusing to proceed — someone edited an already-applied file.",
|
||||
version,
|
||||
name,
|
||||
db_sum,
|
||||
disk_sum
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
174
_primitives/_rust/kei-migrate/tests/integration.rs
Normal file
174
_primitives/_rust/kei-migrate/tests/integration.rs
Normal file
|
|
@ -0,0 +1,174 @@
|
|||
//! Integration tests for kei-migrate against a SQLite file (safe, no deps).
|
||||
//!
|
||||
//! SQLite is chosen as the test backend because it has no server dependency
|
||||
//! and the sqlx-Any path through it exercises the same code path as Postgres
|
||||
//! / MySQL for everything except dialect-specific DDL (which we abstract in
|
||||
//! `db::Backend::create_tracker_sql`).
|
||||
|
||||
use kei_migrate::{cmd_create, db, discover, do_down, do_status, do_up, tracker};
|
||||
use std::fs;
|
||||
use std::path::PathBuf;
|
||||
use tempfile::TempDir;
|
||||
|
||||
struct Env {
|
||||
_tmp: TempDir,
|
||||
url: String,
|
||||
dir: PathBuf,
|
||||
}
|
||||
|
||||
fn setup() -> Env {
|
||||
let tmp = TempDir::new().unwrap();
|
||||
let db_path = tmp.path().join("test.db");
|
||||
let url = format!("sqlite://{}?mode=rwc", db_path.display());
|
||||
let dir = tmp.path().join("migrations");
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
Env { _tmp: tmp, url, dir }
|
||||
}
|
||||
|
||||
fn write_migration(dir: &std::path::Path, version: i64, name: &str, up: &str, down: Option<&str>) {
|
||||
fs::write(dir.join(format!("{}_{}.sql", version, name)), up).unwrap();
|
||||
if let Some(d) = down {
|
||||
fs::write(dir.join(format!("{}_{}.down.sql", version, name)), d).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_backend_from_url_scheme() {
|
||||
assert_eq!(
|
||||
db::detect_backend("postgres://u:p@h/d").unwrap(),
|
||||
db::Backend::Postgres
|
||||
);
|
||||
assert_eq!(
|
||||
db::detect_backend("sqlite:///tmp/x.db").unwrap(),
|
||||
db::Backend::Sqlite
|
||||
);
|
||||
assert_eq!(
|
||||
db::detect_backend("mysql://u:p@h/d").unwrap(),
|
||||
db::Backend::Mysql
|
||||
);
|
||||
assert!(db::detect_backend("mongodb://h").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scan_empty_dir_is_empty() {
|
||||
let env = setup();
|
||||
let migs = discover::scan(&env.dir).unwrap();
|
||||
assert!(migs.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scan_sorts_by_version_and_skips_down_files() {
|
||||
let env = setup();
|
||||
write_migration(&env.dir, 2, "second", "SELECT 1;", Some("SELECT 2;"));
|
||||
write_migration(&env.dir, 1, "first", "SELECT 3;", None);
|
||||
let migs = discover::scan(&env.dir).unwrap();
|
||||
assert_eq!(migs.len(), 2);
|
||||
assert_eq!(migs[0].version, 1);
|
||||
assert_eq!(migs[1].version, 2);
|
||||
assert!(migs[0].down_path.is_none());
|
||||
assert!(migs[1].down_path.is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scan_rejects_duplicate_versions() {
|
||||
let env = setup();
|
||||
write_migration(&env.dir, 1, "a", "SELECT 1;", None);
|
||||
// same version, different name
|
||||
fs::write(env.dir.join("1_b.sql"), "SELECT 2;").unwrap();
|
||||
let err = discover::scan(&env.dir).unwrap_err();
|
||||
assert!(err.to_string().contains("duplicate migration version"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn up_applies_pending_and_tracks_them() {
|
||||
let env = setup();
|
||||
write_migration(
|
||||
&env.dir,
|
||||
1,
|
||||
"create_t",
|
||||
"CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT);",
|
||||
Some("DROP TABLE t;"),
|
||||
);
|
||||
write_migration(
|
||||
&env.dir,
|
||||
2,
|
||||
"insert_row",
|
||||
"INSERT INTO t (id, v) VALUES (1, 'a');",
|
||||
Some("DELETE FROM t WHERE id = 1;"),
|
||||
);
|
||||
let n = do_up(&env.url, &env.dir).await.unwrap();
|
||||
assert_eq!(n, 2);
|
||||
// re-running is a no-op
|
||||
let n2 = do_up(&env.url, &env.dir).await.unwrap();
|
||||
assert_eq!(n2, 0);
|
||||
// status: 2 applied, 0 pending
|
||||
let (a, p) = do_status(&env.url, &env.dir).await.unwrap();
|
||||
assert_eq!(a, 2);
|
||||
assert_eq!(p, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn down_reverts_last_n() {
|
||||
let env = setup();
|
||||
write_migration(
|
||||
&env.dir,
|
||||
1,
|
||||
"create_t",
|
||||
"CREATE TABLE t (id INTEGER PRIMARY KEY);",
|
||||
Some("DROP TABLE t;"),
|
||||
);
|
||||
write_migration(
|
||||
&env.dir,
|
||||
2,
|
||||
"add_col",
|
||||
"ALTER TABLE t ADD COLUMN v TEXT;",
|
||||
Some("-- down unsupported on sqlite but we just drop the table for this test\nDROP TABLE t; CREATE TABLE t (id INTEGER PRIMARY KEY);"),
|
||||
);
|
||||
do_up(&env.url, &env.dir).await.unwrap();
|
||||
let reverted = do_down(&env.url, &env.dir, 1).await.unwrap();
|
||||
assert_eq!(reverted, 1);
|
||||
let (a, p) = do_status(&env.url, &env.dir).await.unwrap();
|
||||
assert_eq!(a, 1);
|
||||
assert_eq!(p, 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn down_refuses_irreversible_marker() {
|
||||
let env = setup();
|
||||
write_migration(
|
||||
&env.dir,
|
||||
1,
|
||||
"make_t",
|
||||
"CREATE TABLE t (id INTEGER PRIMARY KEY);",
|
||||
Some("-- IRREVERSIBLE\n-- don't auto-revert"),
|
||||
);
|
||||
do_up(&env.url, &env.dir).await.unwrap();
|
||||
let err = do_down(&env.url, &env.dir, 1).await.unwrap_err();
|
||||
assert!(err.to_string().contains("IRREVERSIBLE"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn up_detects_checksum_drift() {
|
||||
let env = setup();
|
||||
let up_path = env.dir.join("1_seed.sql");
|
||||
fs::write(&up_path, "CREATE TABLE t (id INTEGER PRIMARY KEY);").unwrap();
|
||||
do_up(&env.url, &env.dir).await.unwrap();
|
||||
// someone edits the already-applied file
|
||||
fs::write(&up_path, "CREATE TABLE t (id INTEGER PRIMARY KEY, extra TEXT);").unwrap();
|
||||
let err = do_up(&env.url, &env.dir).await.unwrap_err();
|
||||
assert!(err.to_string().contains("checksum drift"));
|
||||
// tracker module is exercised end-to-end here
|
||||
let _ = tracker::applied_versions; // keep tracker import used
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn create_scaffolds_up_and_down_files() {
|
||||
let env = setup();
|
||||
let (up, down) = cmd_create::run(&env.dir, "add_users_index").unwrap();
|
||||
assert!(up.exists());
|
||||
assert!(down.exists());
|
||||
let up_txt = fs::read_to_string(&up).unwrap();
|
||||
let down_txt = fs::read_to_string(&down).unwrap();
|
||||
assert!(up_txt.contains("up migration"));
|
||||
assert!(down_txt.contains("down migration"));
|
||||
}
|
||||
115
skills/schema-design/SKILL.md
Normal file
115
skills/schema-design/SKILL.md
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
---
|
||||
name: schema-design
|
||||
description: Hub-and-spoke pipeline that converts "I need a database for app X" into a designed relational schema, a generated first migration, and optional seed/fixture data — via pure-click decisions across five phases. Emits SQL DDL, a kei-migrate-shaped migrations directory, and a library/ORM pick; never writes production secrets.
|
||||
argument-hint: <one-line app description, e.g. "multi-tenant B2B SaaS, 6-8 entities, Postgres + Drizzle">
|
||||
---
|
||||
|
||||
# Schema-Design — Relational Schema & Migration Pipeline (index)
|
||||
|
||||
You are converting "I need a database for app X" into a concrete, reviewable
|
||||
design: chosen DB + ORM, entity list + relations, SQL DDL with indexes and
|
||||
FKs, a scaffolded migrations directory with the first migration, and (if
|
||||
asked) seed data for tests and dev. Every decision is a click; the only
|
||||
typed inputs are the one-line app description in Phase 1 and the entity
|
||||
list in Phase 2.
|
||||
|
||||
This skill does NOT run migrations or touch production. It produces files
|
||||
under `db/schema.sql`, `migrations/<ts>_init.sql` (+ `.down.sql`), and
|
||||
optionally `db/seed.sql`. Applying them is a separate command (`kei-migrate
|
||||
up`), owned by the project's code-implementer.
|
||||
|
||||
The skill reads the five database blocks heavily — every phase references
|
||||
at least one of them:
|
||||
|
||||
- `_blocks/db-postgres.md` — PG 17 patterns, indexing, pooling.
|
||||
- `_blocks/db-sqlite.md` — single-node / edge pragmas.
|
||||
- `_blocks/db-sqlx.md` — Rust query + migration flow.
|
||||
- `_blocks/db-drizzle.md` — TS schema-first ORM.
|
||||
- `_blocks/db-migration-hygiene.md` — universal up/down + checksum rules.
|
||||
|
||||
Primitive used for scaffolding: `_primitives/_rust/kei-migrate` (universal
|
||||
Postgres / SQLite / MySQL migration runner — create + up + down + status).
|
||||
|
||||
---
|
||||
|
||||
## Pipeline overview (5 phases, ≥5 AskUserQuestion calls)
|
||||
|
||||
| Phase | File | Purpose | AskUserQuestion |
|
||||
|---|---|---|---|
|
||||
| 1 | [phase-1-intake.md](phase-1-intake.md) | DB, ORM, scale, style, migration control | 5× (batched) |
|
||||
| 2 | [phase-2-entities.md](phase-2-entities.md) | Entity list + relations matrix | 1× |
|
||||
| 3 | [phase-3-schema.md](phase-3-schema.md) | Generate DDL + indexes + FKs + constraints; review/revise | 1× |
|
||||
| 4 | [phase-4-migrations.md](phase-4-migrations.md) | Scaffold `migrations/` + first migration + kei-migrate wiring | 1× |
|
||||
| 5 | [phase-5-seed.md](phase-5-seed.md) | Optional seed + test fixtures | 1× |
|
||||
|
||||
Minimum AskUserQuestion count across a full session: **9** (5 in Phase 1 +
|
||||
1 each in Phases 2–5). Exceeds the ≥5 hub-and-spoke contract.
|
||||
|
||||
---
|
||||
|
||||
## Variables the pipeline produces
|
||||
|
||||
| Name | Set in | Meaning |
|
||||
|---|---|---|
|
||||
| `INTAKE` | Phase 1 | one-paragraph app description (verbatim) |
|
||||
| `DB` | Phase 1 | Postgres / SQLite / MySQL |
|
||||
| `ORM` | Phase 1 | none (raw SQL) / Drizzle / SQLx / Prisma / SQLAlchemy |
|
||||
| `SCALE` | Phase 1 | solo-prototype / team-dev / production-multi-replica |
|
||||
| `STYLE` | Phase 1 | schema-first (SQL → types) / code-first (types → SQL) |
|
||||
| `MIGCTL` | Phase 1 | manual / auto-on-deploy / hybrid (manual prod, auto dev) |
|
||||
| `ENTITIES` | Phase 2 | list of entities + fields + relations matrix |
|
||||
| `DDL` | Phase 3 | generated SQL (tables, indexes, FKs, constraints) |
|
||||
| `MIGDIR` | Phase 4 | path of migrations dir + first migration filenames |
|
||||
| `SEED` | Phase 5 | seed-data plan (or "skipped") |
|
||||
|
||||
---
|
||||
|
||||
## Final report (emit after Phase 5)
|
||||
|
||||
```
|
||||
=== SCHEMA-DESIGN REPORT ===
|
||||
App: <first 80 chars of INTAKE>...
|
||||
DB / ORM: <DB> + <ORM> (style: <STYLE>)
|
||||
Scale: <SCALE> migration control: <MIGCTL>
|
||||
Entities: <N> tables, <M> relations
|
||||
Schema: db/schema.sql (<LOC> lines, <I> indexes, <F> FKs, <C> constraints)
|
||||
Migrations: migrations/<ts>_init.sql (+ .down.sql) runner: kei-migrate
|
||||
Seed: <SEED summary or "skipped">
|
||||
Libraries: <ORM pick + driver crate/package, one line>
|
||||
Next: run `kei-migrate up` against dev DB, then hand off to code-implementer
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rules (apply throughout)
|
||||
|
||||
- **Pure-click contract.** Only the Phase 1 intake paragraph and the Phase 2
|
||||
entity list are typed. Every other decision is an `AskUserQuestion` call.
|
||||
- **RULE 0.8 Secrets SSoT.** The skill never emits a live `DATABASE_URL`
|
||||
value, never writes to `secrets/*.env`, never hard-codes credentials in
|
||||
DDL or seed. It emits ENV-VAR NAMES only; storage path is
|
||||
`<repo>/secrets/db.env` per `domain-has-secrets.md`.
|
||||
- **NO DOWNGRADE.** If the chosen combination is unsafe (e.g.
|
||||
`auto-on-deploy` + multi-replica without leader-election) the skill
|
||||
returns 2–3 constructive alternatives, never "not supported".
|
||||
- **Migration hygiene enforced.** Every migration emitted is
|
||||
timestamp-prefixed, has a `.down.sql` counterpart, and uses
|
||||
`IF NOT EXISTS` / `IF EXISTS` where safe. See `db-migration-hygiene.md`.
|
||||
- **Test-First.** If Phase 5 is selected, seed includes at minimum a
|
||||
smoke-test fixture (one row per entity) to verify schema loads.
|
||||
- **Surgical scope.** Reads the five db-* blocks; writes only to
|
||||
`db/schema.sql`, `migrations/<ts>_init.{sql,down.sql}`, and optionally
|
||||
`db/seed.sql`. Never touches application code.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- `_blocks/db-postgres.md`, `_blocks/db-sqlite.md`, `_blocks/db-sqlx.md`,
|
||||
`_blocks/db-drizzle.md`, `_blocks/db-migration-hygiene.md`.
|
||||
- `_primitives/_rust/kei-migrate` — universal migration runner
|
||||
(autodetects Postgres / SQLite / MySQL from `DATABASE_URL`).
|
||||
- `_blocks/domain-has-secrets.md` — DB URL storage convention.
|
||||
- `_blocks/rule-pre-dev-gate.md` — check existing schema before inventing.
|
||||
- Evidence grade [E4] — pipeline mirrors standard relational-modelling
|
||||
practice (Codd 1NF-3NF, surrogate keys, FK-first indexing).
|
||||
92
skills/schema-design/phase-1-intake.md
Normal file
92
skills/schema-design/phase-1-intake.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Phase 1 — Intake (DB, ORM, scale, style, migration control)
|
||||
|
||||
One free-text paragraph, then ONE batched `AskUserQuestion` call with all
|
||||
five click decisions. This is the only phase that accepts typed input.
|
||||
|
||||
## 1a — Ask for the app description
|
||||
|
||||
Emit a regular message (NOT AskUserQuestion):
|
||||
|
||||
> Describe the app in one paragraph: what is it, how many entities (rough
|
||||
> count), any constraint I should know (existing DB, regulated data,
|
||||
> multi-tenant, edge / serverless, expected row counts). Reply in one
|
||||
> message.
|
||||
|
||||
Store the reply verbatim as `INTAKE`.
|
||||
|
||||
## 1b — Batched click (AskUserQuestion, 5 questions in ONE call)
|
||||
|
||||
The UI cap per `AskUserQuestion` call is 4–5 questions; emit all five at
|
||||
once for a smooth click-through.
|
||||
|
||||
```json
|
||||
{
|
||||
"questions": [
|
||||
{
|
||||
"question": "Which database engine?",
|
||||
"header": "DB",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "PostgreSQL 17", "description": "Default for multi-user / relational integrity. See _blocks/db-postgres.md"},
|
||||
{"label": "SQLite", "description": "Single-node, edge-friendly, ~100k users ceiling. See _blocks/db-sqlite.md"},
|
||||
{"label": "MySQL / MariaDB", "description": "Existing stack compatibility; kei-migrate supports it"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"question": "ORM / query layer?",
|
||||
"header": "ORM",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "None (raw SQL)", "description": "Hand-written queries; max control, no magic"},
|
||||
{"label": "Drizzle (TS)", "description": "Schema-first or code-first; see _blocks/db-drizzle.md"},
|
||||
{"label": "SQLx (Rust)", "description": "Compile-time checked queries; see _blocks/db-sqlx.md"},
|
||||
{"label": "Prisma (TS)", "description": "Code-first; own migration engine (NOT kei-migrate)"},
|
||||
{"label": "SQLAlchemy (Py)", "description": "Alembic for migrations (NOT kei-migrate); legacy compat"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"question": "Target scale?",
|
||||
"header": "Scale",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "Solo prototype", "description": "One dev, <1k rows, SQLite OK"},
|
||||
{"label": "Team dev", "description": "Shared dev DB, staging, prod — standard"},
|
||||
{"label": "Production multi-replica", "description": "Leader-election required for migrations; zero-downtime patterns mandatory"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"question": "Design style?",
|
||||
"header": "Style",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "Schema-first (SQL → types)", "description": "Write DDL, generate types. Default with raw SQL / SQLx / Drizzle schema-first"},
|
||||
{"label": "Code-first (types → SQL)", "description": "Define entities in code, generate DDL. Drizzle code-first / Prisma / SQLAlchemy"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"question": "Migration control?",
|
||||
"header": "MigCtl",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "Manual (human runs kei-migrate up)", "description": "Safest; recommended for prod"},
|
||||
{"label": "Auto-on-deploy", "description": "CI runs migrations; single-replica only — NO DOWNGRADE warning if multi-replica"},
|
||||
{"label": "Hybrid (manual prod, auto dev)", "description": "Recommended default — dev velocity + prod safety"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Store answers as `DB`, `ORM`, `SCALE`, `STYLE`, `MIGCTL`.
|
||||
|
||||
## Verify-criterion
|
||||
|
||||
- `INTAKE` non-empty.
|
||||
- `DB`, `ORM`, `SCALE`, `STYLE`, `MIGCTL` each exactly one label.
|
||||
- If `ORM ∈ {Prisma, SQLAlchemy}` → note in state: "Phase 4 will hand off
|
||||
to that tool's native migration runner (Prisma migrate / Alembic); the
|
||||
kei-migrate scaffold is skipped or wrapped." No downgrade — the skill
|
||||
still emits a working plan.
|
||||
- If `MIGCTL = Auto-on-deploy` AND `SCALE = Production multi-replica` →
|
||||
warn "race condition risk — every replica tries to apply" (see
|
||||
`db-migration-hygiene.md`) and re-ask with the Hybrid option highlighted.
|
||||
82
skills/schema-design/phase-2-entities.md
Normal file
82
skills/schema-design/phase-2-entities.md
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
# Phase 2 — Entities + relations matrix
|
||||
|
||||
Collect the entity list (typed) once, then click the relations matrix. This
|
||||
is the second (and last) phase that accepts typed input.
|
||||
|
||||
## 2a — Ask for entities (plain message, NOT AskUserQuestion)
|
||||
|
||||
> List the entities (tables) and for each a short comma-separated field
|
||||
> list. One entity per line, format: `<Entity>: field1, field2, ...`.
|
||||
> Example:
|
||||
>
|
||||
> ```
|
||||
> User: email, name, created_at
|
||||
> Organization: name, plan
|
||||
> Membership: user_id, org_id, role
|
||||
> ```
|
||||
>
|
||||
> 3–15 entities is typical. Keep it short — we'll refine fields in Phase 3.
|
||||
|
||||
Parse the reply into `ENTITIES = [{name, fields: [...]}, ...]`. Validate:
|
||||
- Entity names are `PascalCase` (normalize if user types `user_profile` →
|
||||
`UserProfile`, record normalization in state).
|
||||
- Each entity has ≥1 field.
|
||||
- No duplicate entity names.
|
||||
- If parse fails → re-ask once with a corrected example.
|
||||
|
||||
## 2b — Relations matrix click (AskUserQuestion, multi-select)
|
||||
|
||||
For each UNORDERED PAIR of entities `(A, B)`, ask one multi-select row.
|
||||
Skip pairs the user hasn't mentioned any cross-reference for (heuristic:
|
||||
if `A`'s fields include `b_id` or `B`'s fields include `a_id`, or the
|
||||
user's intake paragraph mentions both).
|
||||
|
||||
Build ONE `AskUserQuestion` call with up to 5 questions. If the entity
|
||||
count yields > 5 candidate pairs, batch into multiple calls (still counts
|
||||
toward the ≥1 AskUserQuestion minimum).
|
||||
|
||||
Per-pair question template:
|
||||
|
||||
```json
|
||||
{
|
||||
"question": "Relation between <A> and <B>?",
|
||||
"header": "<A>↔<B>",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "None", "description": "No direct FK; entities are independent"},
|
||||
{"label": "One-to-one", "description": "A.b_id UNIQUE FK to B.id (or vice versa)"},
|
||||
{"label": "One-to-many (A→B)", "description": "B.a_id FK to A.id; one A has many B"},
|
||||
{"label": "One-to-many (B→A)", "description": "A.b_id FK to B.id; one B has many A"},
|
||||
{"label": "Many-to-many", "description": "Requires a junction table; skill will auto-name it <A><B>"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Store the result in `ENTITIES` as `.relations = [{from, to, kind}, ...]`.
|
||||
|
||||
## 2c — Auto-generate junction tables
|
||||
|
||||
For each pair marked `Many-to-many`, append a synthetic entity to
|
||||
`ENTITIES`:
|
||||
|
||||
```
|
||||
<A><B>:
|
||||
<a>_id FK → <A>.id (ON DELETE CASCADE)
|
||||
<b>_id FK → <B>.id (ON DELETE CASCADE)
|
||||
PRIMARY KEY (<a>_id, <b>_id)
|
||||
created_at TIMESTAMPTZ DEFAULT now()
|
||||
```
|
||||
|
||||
Names must be deterministic (alphabetical order: `OrganizationUser`, not
|
||||
`UserOrganization`, for pair `(User, Organization)`). Record the rule in
|
||||
state so Phase 3 renders it consistently.
|
||||
|
||||
## Verify-criterion
|
||||
|
||||
- `ENTITIES` has ≥1 entry after parse.
|
||||
- Every relation in `ENTITIES[*].relations` references two distinct
|
||||
existing entities.
|
||||
- Every `Many-to-many` has produced a junction entity.
|
||||
- No entity is orphaned (zero relations AND not mentioned in INTAKE) —
|
||||
warn the user with "Entity X has no relations; keep it?" (NO DOWNGRADE:
|
||||
offer `keep / drop / add relation` as follow-up click).
|
||||
115
skills/schema-design/phase-3-schema.md
Normal file
115
skills/schema-design/phase-3-schema.md
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
# Phase 3 — Generate SQL DDL (tables, indexes, FKs, constraints)
|
||||
|
||||
Emit a full `db/schema.sql` file based on `DB`, `ORM`, `STYLE`, and
|
||||
`ENTITIES`. Then ONE `AskUserQuestion` to review/revise.
|
||||
|
||||
## 3a — Pick primary-key strategy (inline, no AskUserQuestion — deterministic)
|
||||
|
||||
- Postgres 17: `id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY`
|
||||
(SQL-standard, PG 10+; avoid legacy `SERIAL`). See `db-postgres.md`.
|
||||
- SQLite: `id INTEGER PRIMARY KEY` (rowid alias; autoincrement discouraged
|
||||
unless monotonic guarantee required). See `db-sqlite.md`.
|
||||
- MySQL: `id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY`.
|
||||
|
||||
If the user's fields include `id` already, respect it; otherwise prepend
|
||||
the surrogate PK. Record the choice in state for Phase 4 traceability.
|
||||
|
||||
## 3b — Per-entity DDL generation rules
|
||||
|
||||
For each entity in `ENTITIES`:
|
||||
|
||||
1. **CREATE TABLE**: `CREATE TABLE IF NOT EXISTS <snake_case_name>` (plural
|
||||
unless user named it singular — default to pluralize: `User` → `users`,
|
||||
`Organization` → `organizations`; junction `OrganizationUser` →
|
||||
`organization_users`).
|
||||
2. **Columns**: infer SQL type from field name + heuristics:
|
||||
- `*_id`, `id` → `BIGINT` (Postgres/MySQL) / `INTEGER` (SQLite).
|
||||
- `*_at`, `created_at`, `updated_at` → `TIMESTAMPTZ` (PG) / `DATETIME`
|
||||
(SQLite) / `TIMESTAMP` (MySQL), `DEFAULT now()` (PG) /
|
||||
`DEFAULT CURRENT_TIMESTAMP` (others).
|
||||
- `email`, `*_email` → `TEXT` / `VARCHAR(320)` (RFC 5321 limit) + `NOT
|
||||
NULL` + `CHECK (email LIKE '%@%')` (cheap sanity, real validation is
|
||||
app-side).
|
||||
- `*_count`, `*_amount` → `INTEGER` or `NUMERIC(18,2)` (money);
|
||||
default `0`.
|
||||
- `is_*`, `has_*` → `BOOLEAN NOT NULL DEFAULT false`.
|
||||
- Freeform → `TEXT NOT NULL` unless user said "optional".
|
||||
3. **Timestamps default**: unless user opts out, add `created_at` +
|
||||
`updated_at` to every non-junction entity.
|
||||
4. **Soft-delete**: if `SCALE = Production multi-replica`, add
|
||||
`deleted_at TIMESTAMPTZ NULL` + partial index (PG only) on
|
||||
`WHERE deleted_at IS NULL`.
|
||||
|
||||
## 3c — Foreign keys
|
||||
|
||||
For each `relations` entry (kind ≠ `None`):
|
||||
|
||||
- `REFERENCES <parent_table>(id) ON DELETE <ACTION>`.
|
||||
- Default `ON DELETE` action: junction → `CASCADE`; one-to-one → `CASCADE`
|
||||
from owning side; one-to-many → `RESTRICT` (safer default; explicit
|
||||
cascade is a product decision). See `db-migration-hygiene.md`.
|
||||
- **Mandatory FK index**: every FK column gets `CREATE INDEX
|
||||
IF NOT EXISTS idx_<table>_<col> ON <table>(<col>);`. Unindexed FKs =
|
||||
join explosion (see `db-postgres.md` "Forbidden").
|
||||
|
||||
## 3d — Indexes + constraints
|
||||
|
||||
- Unique: `email`, `slug`, `username`, any field user marked `UNIQUE` →
|
||||
`UNIQUE` column constraint.
|
||||
- Composite indexes: for junction tables, PK is composite; add a
|
||||
reverse-order index if both directions of lookup are common.
|
||||
- Check constraints: `CHECK (status IN (...))` for enum-ish text fields;
|
||||
`CHECK (amount >= 0)` for non-negative numerics.
|
||||
- Triggers: if `updated_at` present AND `DB = Postgres`, emit a
|
||||
`CREATE OR REPLACE FUNCTION set_updated_at()` + `CREATE TRIGGER` per
|
||||
table. SQLite uses `AFTER UPDATE` triggers; MySQL uses `ON UPDATE
|
||||
CURRENT_TIMESTAMP` in the column definition.
|
||||
|
||||
## 3e — Emit the file
|
||||
|
||||
Write `db/schema.sql` with a top-level comment:
|
||||
|
||||
```sql
|
||||
-- Generated by /schema-design — <YYYY-MM-DD>
|
||||
-- DB: <DB> ORM: <ORM> Style: <STYLE>
|
||||
-- Entities: <N> Relations: <M>
|
||||
-- Edit freely; Phase 4 will package the first migration from this file.
|
||||
```
|
||||
|
||||
Print the first ~40 lines of the file inline in chat (for review) plus a
|
||||
`<path>:<total lines>` footer. Full contents live on disk.
|
||||
|
||||
## 3f — Review click (AskUserQuestion)
|
||||
|
||||
```json
|
||||
{
|
||||
"questions": [
|
||||
{
|
||||
"question": "Schema review — accept or revise?",
|
||||
"header": "Review",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "Accept — proceed to Phase 4", "description": "Schema is correct; package into a migration"},
|
||||
{"label": "Revise entities (return to Phase 2)", "description": "Add/drop entities or fix a relation"},
|
||||
{"label": "Revise types (edit db/schema.sql)", "description": "Skill will emit a 1-AskUserQuestion per-column-type fix loop"},
|
||||
{"label": "Revise indexes (edit db/schema.sql)", "description": "Skill will emit a 1-AskUserQuestion per-index fix loop"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If the user picks Revise, loop the relevant sub-step; then re-emit and
|
||||
re-ask. Max 3 revise loops before the skill asks whether to accept-as-is
|
||||
or escalate to code-implementer handoff (NO DOWNGRADE).
|
||||
|
||||
## Verify-criterion
|
||||
|
||||
- `db/schema.sql` exists on disk and is non-empty.
|
||||
- Every FK has a companion index.
|
||||
- Every non-junction entity has a PK + `created_at`.
|
||||
- If `ORM = Drizzle` with `STYLE = code-first`: also emit
|
||||
`db/schema.ts` alongside `db/schema.sql` (Drizzle introspection artefact;
|
||||
see `db-drizzle.md`). For `ORM = SQLx`: record in state that Phase 4's
|
||||
migration will also be picked up by `sqlx migrate run`.
|
||||
- Accept click sets `DDL = "db/schema.sql"` and hands off to Phase 4.
|
||||
121
skills/schema-design/phase-4-migrations.md
Normal file
121
skills/schema-design/phase-4-migrations.md
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
# Phase 4 — Migration scaffold + first migration + kei-migrate wiring
|
||||
|
||||
Package `db/schema.sql` (from Phase 3) into a proper
|
||||
timestamp-prefixed migration pair under `migrations/`, and emit the
|
||||
`kei-migrate` invocation the user should run.
|
||||
|
||||
## 4a — Create `migrations/` directory (no AskUserQuestion)
|
||||
|
||||
If `migrations/` does not yet exist in the repo, create it. Emit one
|
||||
`.keep` file or rely on the first migration to anchor it.
|
||||
|
||||
If `ORM = Prisma`: the directory is `prisma/migrations/` and the runner is
|
||||
`prisma migrate dev` — skill notes this and skips kei-migrate wiring
|
||||
(reference the handoff but DO NOT overwrite Prisma's own layout).
|
||||
|
||||
If `ORM = SQLAlchemy`: the directory is `alembic/versions/` and the runner
|
||||
is `alembic upgrade head` — same rule, skip kei-migrate wiring.
|
||||
|
||||
For every other `ORM` value (none / Drizzle / SQLx): use `migrations/`
|
||||
with kei-migrate.
|
||||
|
||||
## 4b — Generate timestamp + filename
|
||||
|
||||
- Timestamp format: `YYYYMMDDHHMMSS` (matches `kei-migrate create`'s
|
||||
convention — see `_primitives/_rust/kei-migrate/src/cmd_create.rs`).
|
||||
- Migration name: `init_schema`.
|
||||
- Files:
|
||||
- `migrations/<ts>_init_schema.sql` (up — full DDL from Phase 3)
|
||||
- `migrations/<ts>_init_schema.down.sql` (down — `DROP TABLE` reverse order)
|
||||
|
||||
## 4c — Up migration content
|
||||
|
||||
Copy `db/schema.sql` contents into the up file verbatim, with a one-line
|
||||
header:
|
||||
|
||||
```sql
|
||||
-- kei-migrate: init_schema (generated <YYYY-MM-DD>)
|
||||
-- See db/schema.sql for the schema SSoT.
|
||||
```
|
||||
|
||||
**Do not split one migration per table** — the initial schema ships as ONE
|
||||
migration by convention. Subsequent changes each get their own timestamp.
|
||||
|
||||
## 4d — Down migration content
|
||||
|
||||
Emit `DROP TABLE IF EXISTS <name> CASCADE;` for every entity, in REVERSE
|
||||
dependency order (children before parents — junctions first, then leaf
|
||||
entities, then referenced entities last).
|
||||
|
||||
If any table is flagged `-- IRREVERSIBLE` by the user (e.g. contains
|
||||
critical data once populated), replace the `DROP TABLE` line with:
|
||||
|
||||
```sql
|
||||
-- IRREVERSIBLE: this table holds production data; manual restore required.
|
||||
-- Abort reverse migration.
|
||||
SELECT RAISE(FAIL, 'irreversible: init_schema') ; -- or equivalent per DB
|
||||
```
|
||||
|
||||
See `db-migration-hygiene.md` for the irreversible pattern.
|
||||
|
||||
## 4e — Wire kei-migrate (AskUserQuestion)
|
||||
|
||||
```json
|
||||
{
|
||||
"questions": [
|
||||
{
|
||||
"question": "Add kei-migrate to the project?",
|
||||
"header": "Runner",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "Add to Cargo workspace as path dep", "description": "Rust projects — edit root Cargo.toml members. Skill will NOT edit; emits the snippet for you to paste."},
|
||||
{"label": "Install prebuilt binary (system-wide)", "description": "Any stack — `cargo install --path _primitives/_rust/kei-migrate` once; repo stays tool-agnostic"},
|
||||
{"label": "Use existing runner (Prisma / Alembic / Drizzle-kit / goose / Atlas)", "description": "Skill skips kei-migrate; records the handoff in the report"},
|
||||
{"label": "Decide later", "description": "Files land on disk; runner wiring deferred"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Store the answer as `RUNNER`.
|
||||
|
||||
## 4f — Emit the next-step command (inline, no AskUserQuestion)
|
||||
|
||||
Print a fenced code block tailored to `DB` + `RUNNER`:
|
||||
|
||||
```bash
|
||||
# Load DB URL from SSoT (RULE 0.8)
|
||||
set -a && source secrets/db.env && set +a
|
||||
|
||||
# Preview pending migrations
|
||||
kei-migrate --database-url "$DATABASE_URL" --dir migrations status
|
||||
|
||||
# Apply
|
||||
kei-migrate --database-url "$DATABASE_URL" --dir migrations up
|
||||
|
||||
# Revert the latest (dev only!)
|
||||
kei-migrate --database-url "$DATABASE_URL" --dir migrations down 1
|
||||
```
|
||||
|
||||
Reminder (once): `secrets/db.env` must be `chmod 600` and listed in
|
||||
`.gitignore` BEFORE the first write. Template entry:
|
||||
|
||||
```bash
|
||||
# secrets/db.env — chmod 600 before first write
|
||||
DATABASE_URL=
|
||||
```
|
||||
|
||||
No values. RULE 0.8 secrets SSoT.
|
||||
|
||||
## Verify-criterion
|
||||
|
||||
- `migrations/<ts>_init_schema.sql` exists and equals `db/schema.sql` body
|
||||
with the one-line header prepended.
|
||||
- `migrations/<ts>_init_schema.down.sql` exists with DROP statements in
|
||||
reverse dependency order.
|
||||
- Filenames use the `kei-migrate create` timestamp convention.
|
||||
- If `ORM ∈ {Prisma, SQLAlchemy}` — kei-migrate files are NOT created;
|
||||
instead record a one-line handoff in state: "use `<native runner>` —
|
||||
schema.sql is the design SSoT, port it to the native format."
|
||||
- Reminder about `secrets/db.env` emitted exactly once.
|
||||
92
skills/schema-design/phase-5-seed.md
Normal file
92
skills/schema-design/phase-5-seed.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Phase 5 — Optional seed data + test fixtures
|
||||
|
||||
Emit `db/seed.sql` with deterministic, safe-to-re-run seed rows, OR skip
|
||||
this phase entirely. Single `AskUserQuestion` to decide.
|
||||
|
||||
## 5a — Seed decision (AskUserQuestion)
|
||||
|
||||
```json
|
||||
{
|
||||
"questions": [
|
||||
{
|
||||
"question": "Seed data for dev / tests?",
|
||||
"header": "Seed",
|
||||
"multiSelect": false,
|
||||
"options": [
|
||||
{"label": "Smoke fixture (1 row per entity)", "description": "Minimal — proves schema loads and FKs resolve. Recommended default."},
|
||||
{"label": "Rich dev seed (5–20 rows per entity)", "description": "Realistic playground for local dev; deterministic IDs from a fixed seed"},
|
||||
{"label": "Test fixtures only (for integration tests)", "description": "Small, labelled datasets keyed by test-case name"},
|
||||
{"label": "Skip — no seed data", "description": "Schema-only delivery; tests will use mocks or runtime factories"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Store as `SEED`.
|
||||
|
||||
## 5b — Generate `db/seed.sql` (inline, no AskUserQuestion)
|
||||
|
||||
Rules, regardless of choice (unless Skip):
|
||||
|
||||
- **Idempotent** — every `INSERT` uses `ON CONFLICT DO NOTHING` (PG/MySQL)
|
||||
or `INSERT OR IGNORE` (SQLite). Re-running seed is safe.
|
||||
- **Deterministic PKs** — use explicit IDs (`1`, `2`, ...) not relying on
|
||||
sequences. For UUIDs, use fixed values from a documented seed (e.g.
|
||||
`uuid_generate_v5(...)` or hard-coded dev-only UUIDs).
|
||||
- **No secrets** — no real emails (use `user1@example.test`), no real
|
||||
phone numbers, no real names. RULE 0.8 still applies: nothing in seed
|
||||
should be or look like a production token.
|
||||
- **Respect FK order** — insert parents before children, junctions last.
|
||||
- **One file** — `db/seed.sql`. Not split per entity (ordering matters).
|
||||
|
||||
Smoke-fixture shape:
|
||||
|
||||
```sql
|
||||
-- Generated by /schema-design Phase 5 — <YYYY-MM-DD>
|
||||
-- Smoke fixture: one row per entity, deterministic IDs.
|
||||
|
||||
INSERT INTO users (id, email, name) VALUES (1, 'user1@example.test', 'Seed User')
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO organizations (id, name) VALUES (1, 'Seed Org')
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO organization_users (user_id, organization_id, role)
|
||||
VALUES (1, 1, 'owner') ON CONFLICT DO NOTHING;
|
||||
```
|
||||
|
||||
For "Rich dev seed" — use counted loops in SQL (PG: `generate_series`;
|
||||
SQLite: recursive CTE; MySQL: `WITH RECURSIVE`) to produce 5–20 rows per
|
||||
non-junction entity, with FK references wrapping modulo the parent count.
|
||||
|
||||
For "Test fixtures" — group by test name via SQL comments:
|
||||
|
||||
```sql
|
||||
-- fixture: test_user_signup
|
||||
INSERT INTO users (id, email, name) VALUES (101, 'signup@example.test', 'Signup') ...
|
||||
-- fixture: test_org_invite
|
||||
INSERT INTO users (id, email, name) VALUES (201, 'invitee@example.test', 'Invitee') ...
|
||||
```
|
||||
|
||||
## 5c — Test-First hook (inline)
|
||||
|
||||
If `SEED ≠ Skip`, emit a smoke-test snippet tailored to `DB`:
|
||||
|
||||
```bash
|
||||
# Smoke-test: load schema + seed, assert row counts.
|
||||
kei-migrate --database-url "$DATABASE_URL" --dir migrations up
|
||||
psql "$DATABASE_URL" -f db/seed.sql # or: sqlite3 <file> < db/seed.sql
|
||||
psql "$DATABASE_URL" -c "SELECT count(*) FROM users;"
|
||||
```
|
||||
|
||||
Remind the user: this is a smoke test, not a Test-First contract. Real
|
||||
integration tests live in the project's test suite — see
|
||||
`_blocks/rule-test-first.md`.
|
||||
|
||||
## Verify-criterion
|
||||
|
||||
- If `SEED = Skip`: `db/seed.sql` is NOT created; state records "seed
|
||||
skipped".
|
||||
- Otherwise: `db/seed.sql` exists, is idempotent, respects FK order, and
|
||||
uses `@example.test` (or equivalent RFC 2606 reserved) emails only.
|
||||
- Smoke-test snippet printed inline in chat (once).
|
||||
- Final report can now be emitted (see `SKILL.md` index).
|
||||
Loading…
Reference in a new issue