125M Transformer Autocompletes Piano at 108 Notes/Sec On-Device
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- RollTab is a 125M-parameter transformer that autocompletes piano performances at ~108 notes/sec on an iPhone 15, built across 14 experiments over nearly a year and offered free for iPhone/iPad with a MIDI keyboard.
- The author's winning MIDI representation packs each note into a single token whose pitch, delta-onset, duration, and velocity are summed embeddings, letting the transformer advance music by one complete note per pass instead of four autoregressive steps.
- The training set comprised a few hundred thousand MIDI files (~300M note events); scaling to roughly 5x that volume made models worse, with deduplication, density filtering, and piano-focused selection producing bigger gains than raw size.
- Scheduled sampling rose from 0% to 50% during training — raising validation loss while improving actual continuations — and was paired with cross-entropy across five output heads (type, pitch, delta, duration, velocity) plus DPO post-training.
- Sustain pedal handling was baked into note duration during preprocessing, so notes released while the pedal is held extend to pedal-up time, leaving the model to predict only pitch, onset, duration, and velocity.
- Evaluation relied on Gemini 3.5 Flash doing pairwise comparisons rather than absolute scoring, with each comparison mirrored to reduce position bias; automatic metrics (pitch entropy, n-gram repeats, density) caught failures but couldn't pick the best model.
- The medium ~64M model nearly matched the large 125M, and the author is currently compressing it further to cut iOS footprint and latency.
Why it matters: This is a working case study showing how careful tokenization, aggressive data cleaning, and targeted post-training can let a 125M-parameter model deliver real-time musical continuation on a phone — and the author explicitly found that scaling the dataset 5x hurt performance, a finding that reinforces the data-quality-over-quantity thesis now playing out across model development.
Ask SkimNews
