KeiSeiKit-1.0/_blocks/domain-ml-training.md

DOMAIN — ML Training

Math-First block (rule-math-first.md) MUST be included alongside this one.

Pre-Experiment Check — blocking checklist (answer all before launch — each GPU run costs real money):

1. TOKENIZATION — BPE / character / byte / morphological? Different tokenizations produce different units and are NOT directly comparable.
2. ARCHITECTURE — exact class / file / commit. No ambiguity.
3. INIT / MATRICES — random / structured / pretrained? Note initialization distribution and rank if relevant.
4. TRAINING DIRECTION — forward / reverse / mixed? State it; some models are only tested one way.
5. METRIC — what EXACT metric and on what EXACT data split. State units (PPL on which tokenizer, accuracy on which set).
6. RESEARCH QUESTION — "This run tests hypothesis: ___". Cannot formulate → DO NOT LAUNCH.
7. PRIOR RESULTS — check your memory/{project}.md + any wrong-paths*.md notes. Don't repeat failed configs.
8. KNOWN BUGS — list the known-broken configurations for the current architecture. Don't re-hit them.

Results logging — IMMEDIATELY after every run (success / timeout / failed / NaN): Record in memory/{project}.md BEFORE analysis. Mandatory fields: Model name, Architecture, Dimensions, Key config, Params EXACT (never "~7M"), Data + count, Steps/Epochs, Batch/Seq, Seed, Metric, Best, Time, Hardware, Status, Cost actual, Notes.

Multi-seed rigor (for any claim going into DECISIONS.md, a paper, or a public result):

• Minimum ≥ 5 seeds (3 for smoke tests). Default [42, 137, 256, ...].
• Report cross-validation mean ± std, NOT single-fold cherry-pick. Single-fold cherry-picking can inflate published numbers by double-digit percentage points.
• Cache ablation table (full / zero / random / shuffled) on zero-model AND one-trained-model.

Baseline-first discipline: before running ANY exploration-heavy training (hill-climb, ES, PPO, RL) on a task, SEARCH for an existing published baseline (env source tree, paper README, leaderboards). If one exists — run it locally, extract trajectories, distill your model via supervised loss, THEN fine-tune. Pure exploration from scratch when a baseline exists is wasted compute.

Forbidden: launching without the checklist; "~N M" params; analyzing before logging; single-seed claims for anything public; class weighting when val matches train prior; cosine LR on < 50 epochs; tuning before ablating what's unnecessary.