The Improviser · Lesson 1 — The Next Note

The next note — how a model decides.

An AI picks its next word the way a musician takes a solo: choose the next note, then the next, then the next.

Set the feel and let the model play the next note — watch which notes it weighed, and hear what happens when many notes fit at once.

Watch someone improvise and it can look like pure invention. It isn't — but it isn't planned out in advance either. They build the line one note at a time, listening to what they've played so far — the key, the groove, where the phrase is heading — and each note narrows what should come next.

Some moments almost decide themselves: a phrase falling home to the tonic, the note the whole key leans on. Others stay wide open, where a dozen notes would all work. The player feels the difference and commits — firmly when one note clearly fits, more lightly when several do.

A large language model (or LLM) improvises the same way, only in words. It takes the text so far, weighs every possible next word against everything it absorbed in training, and plays one. Then again, and again — one piece at a time, which is why a chatbot types itself out in front of you instead of appearing all at once.

Pick a feel. The model weighs the notes that could come next, then plays one.

the line so far

Pick a feel to play the next note ↓

You hand over the notes. The model plays the next one.

You don't write the melody — you pass the notes so far and the feel, and ask. The model weighs it against every phrase it ever heard, and hands back a note and how sure it is.

# Hand over the line so far — the model predicts the next note.
step = model.predict_next(
    played = ["C", "D", "E", "G"],
    key    = "C major",
    feel   = "resolve",
)

# step.note -> "C"   ·   step.confidence -> 0.74

Many options, one note.

Same four notes, three different feels. No single rule picked the next note — the model weighed every candidate against the key, the line and the style. When one note clearly led, it played with confidence. When many fit at once, it barely committed.

An LLM works the same way. It doesn't recall one right answer — it holds a set of odds over what could come next and plays the likeliest: confident when one option towers, hedged when they tie.

Quick check · your ear

The model puts most of the probability on one clear next note. That means it is…

Next: the odds →