İvme-Conversate — Tiny Language Models, Text Continuation
Sub-25M-parameter decoder-only base models · not instruction-tuned · these models continue text, they do not chat or answer questions.
Model
Try a prompt
16 512
0.1 2
0 200
1 2
Run the same prompt and settings through both models at once to see the difference training data made, plus the benchmark deltas below.
Try a prompt
16 512
0.1 2
0 200
1 2
Benchmark improvement, v1 → v2
WikiText-2 byte perplexity ↓ | 61.40 | 75.09 | +13.69 (+22%) |