The 125M base model, fine-tuned to answer a question from a passage you provide. Paste a legal or financial excerpt, ask about it, and it answers grounded in that text (or says it doesn't have enough info).
This is the base SLM-125M after supervised fine-tuning on grounded (RAFT-style) question-answer data: 16,284 pairs generated from our own cleaned case-law / SEC corpus (14,787 used for training after balancing), where every answer is supported by a provided passage. Training saw ~6.6M tokens, with loss computed on the ~490K answer tokens only, over 4 epochs on 4×H100 for $0.42 (plus a few dollars of Gemini teacher generation).
Why grounded? At 125M a model can't reliably memorize facts (~2 bits/param), so the honest, reliable design is to make it a fluent reader of text you give it. Answers are grounded and fluent but, at this size, can be imprecise on exact figures or occasionally over-cautious, this is a teaching demonstration of the fine-tuning method, not a production legal tool. Fine-tuned from the 10-epoch base (val perplexity 8.36). First call may take ~15–30s while the model wakes.
| Stage | Cost | Share |
|---|---|---|
| Data pipelinecorpus clean + dedup + 16K tokenizer + tokenize (CPU) | $2.15 | 1.8% |
| Pretraining10 epochs on 8x H100, 21.9B tokens seen, incl. smoke runs | $118.64 | 97.2% |
| Fine-tuning (grounded SFT)instruction set build + full fine-tune on 4x H100 | $1.27 | 1.0% |
| Total to assemble | $122.06 | 100% |
Pretraining, with the data pipeline that fed it, is 99.0% of the bill. Everything after it (fine-tuning) came to $1.27. Teaching a model to behave is cheap; teaching it to know is not.
Figures are actual invoiced Modal usage (GPU + CPU + memory) for this model’s lineage, not estimates. Shared inputs (the QA and preference datasets) are charged at this model’s share, and their generation also used ~$1–2 of external LLM API calls, included above. Serving is billed separately and scales to zero.