The practice room — predicting the next note.
The model only ever guesses the next note — so why does that add up to real musicianship?
The answer: to guess the next note well, you have to know a surprising amount. Finish an ascending C-major scale and you need the key. Resolve a tense dominant chord and you need harmony. Bend into the right blue note and you need the style. Leave a phrase hanging and you need a feel for tension. Each blank quietly demands real musical knowledge.
A player in the practice room — shut away where musicians drill alone for hours — doesn't memorise every tune; by working through thousands of them they slowly soak up the patterns of the music, until the right note becomes instinct. A model learns the same way. During a phase called training, it reads an enormous amount of music and text and does nothing but try to predict the next token, nudging itself a little whenever it's wrong.
Do that across billions of phrases and memorising isn't an option — there's far too much. What remains is to absorb the regularities underneath: scales, harmony, rhythm, the shape of a line, how a style behaves. That hard-won sense of how music fits together is where the apparent musicianship comes from. And it's learned once, up front: by the time the model plays for you, its training is finished and those patterns are fixed.
Watch a few reps — each one is a "guess the next note". Hear it land, and see what the model must already know to play it.
One note, learned from listening.
The model only ever predicts the next token. Nobody wrote these notes in by hand — it picked them up from music, the way a player learns the language by playing it.
model.predict_next("C D E F G A B") # → "C" (one token) # then it adds that note and calls predict_next again — note after note. # never hard-coded — learned from billions of next-note guesses.
Predicting well is learning how the music works.
Five reps, five different kinds of knowing — the key, then harmony, then style, then tension, then the shape of a phrase. None of it was programmed in. The only way to get good at "guess the next note", across everything people have played, is to take on how scales, harmony and feel actually behave.
That's where the musicianship comes from. Like a player in the practice room, the model never memorised every tune; by drilling enough of them it learned the patterns underneath — and those patterns are what make each small bet land somewhere musical.