How Does ChatGPT Work? The Journey of Your Message

You’ve surely lived this scene before: you ask ChatGPT a question, and the answer writes itself, word after word, as if someone were typing at full speed on the other side of the screen. One day, I wondered what was actually happening during those few seconds. The answer is simpler than anything I had imagined — a machine betting on the next word, dozens of times per second — and far more dizzying too. This article opens a series I’ve called “The Dive”: eight levels to descend, article after article, all the way down to the mathematical heart of AI. Here, we stay at the surface: no equations, I promise. Just the complete journey of your message. Mask, snorkel: down we go!

The essentials

• ChatGPT doesn’t look your answer up in a database: it calculates it, word after word.
• Every word on your screen is the result of a probability calculation over roughly 200,000 possible tokens (the dictionary of OpenAI’s recent models).
• Large language models (AI programs like the one behind ChatGPT) hold hundreds of billions of parameters — the little adjustable numbers where everything the model has learned is stored — and were trained on trillions of words — over 15 trillion tokens for Meta’s Llama 3, for instance.
• The text scrolling across your screen is not a visual effect: it’s the calculation itself, displayed live.
• This article is Level 1 of “The Dive” — the complete map of the eight levels is at the end of the article.

Where does your message go when you press Enter?

Your message doesn’t stay in your phone or your computer: it travels, via your browser or the app, to a data center — a building packed with servers fitted with graphics processing units (GPUs). That’s where, and only where, ChatGPT “exists” and computes its answer.

A Microsoft server room — aisles like these are where ChatGPT computes its answers. Photo: Microsoft.

This is the first reflex to correct: the intelligence is not in your device. Your phone plays the same role as when it displays a website: it sends a request to a remote server and displays whatever comes back. All the magic happens on the other side, in air-conditioned halls where tens of thousands of machines compute day and night.

Let’s run with the diving metaphor — it’s going to stay with us for eight articles: your screen is the surface of the water. Everything that matters happens below. And the first thing you meet on the way down is those famous GPUs.

How can a machine read your sentence?

It doesn’t read it the way you do: your text is first split into tokens, fragments of words (“incredible” might become “incred” + “ible”), and each token is then replaced with numbers. From that point on, the machine never handles letters again — only numbers.

A computer can do nothing but calculate. For a text to get in, it must therefore be turned into something calculable. Splitting into tokens is the first step of that transformation. The “model” is the name of the AI program itself — the one you’re talking to through ChatGPT. A recent model carries a fixed dictionary of about 200,000 text fragments, each filed under a number, and your message gets rewritten with those fragments. To the machine, “hello” is not a word: it’s something like entry no. 21,437 of that immense dictionary. You can actually watch your own text get split on OpenAI’s official tool — try it with one of your own sentences.

Hold on to this idea, because it explains a whole crowd of ChatGPT’s strange behaviors: the AI never sees your letters, nor even your words. That’s why, for instance, counting the letters in a word can make it break a sweat: it doesn’t see s-t-r-a-w-b-e-r-r-y, it sees one or two tokens — opaque blocks. (Yes, that’s where the famous “how many r’s in strawberry” failures come from.) We’ll devote all of Level 3 to this splitting, and Level 4 to the question that follows from it: how can plain numbers carry meaning?

What does ChatGPT actually do to build its answer?

One single thing, repeated in a loop: predicting the next word. For the first word, the model has nothing but your message: it calculates the probability of every possible token, picks one, writes it. Then it takes the whole thing — your message plus what it has just written — and starts over. The entire answer is born from this loop, one fragment at a time.

This is the heart of everything, so let’s take the time to look at it properly:

Yes, you read that right: there is no draft, no hidden plan, no answer written backstage and then revealed with a typewriter effect. When the words appear one by one on your screen, you are not watching an animation: you are watching the calculation happen, in real time. Every displayed word has just been picked, that very instant, from among tens of thousands of candidates.

The first time I understood this, I replayed all my conversations with ChatGPT in a different light. The machine doesn’t “know” where it’s going: it moves forward word by word, and every chosen word becomes a constraint on all the following ones. A tightrope walker inventing the rope as they walk on it.

Why does a simple “word predictor” seem so intelligent?

Because predicting well requires having absorbed a lot. To guess the next word of a medical diagnosis, you need to “know” medicine; to complete a line of reasoning, you need to follow the reasoning. By learning to predict across trillions of sentences, the model was forced to internalize the structure of the world those sentences describe.

This is THE question everyone asks at this point “Predicting the next word”, said like that, sounds like the suggestions on your phone’s keyboard — and frankly, those don’t shine. So, what gives?

The game is always the same — guess what comes next — but the knowledge it demands has no ceiling. That’s the trap the model fell into during its training, and it’s a wonderfully fertile trap.

Imagine a friend who spent years training at a single game: finishing other people’s sentences. Not just yours: those of a doctor in the middle of a consultation, of a lawyer drafting her closing arguments, of a developer writing code, of a poet. To correctly finish a doctor’s sentence, our friend is forced to learn some medicine. To finish a calculation, he’s forced to learn how to calculate. After trillions of sentences, saying that he “merely completes text” remains technically true — but it no longer describes, at all, the breadth of what he had to learn to pull it off.

And let’s be honest all the way: does that deserve the word “understanding”? Researchers themselves are still debating it, and I’ll be careful not to settle a debate that divides the best of them. What we can describe precisely is the mechanism — and that’s already a lot. Note, in passing, a point that never fails to surprise: that knowledge isn’t stored anywhere in readable form. There is no “Wikipedia” file inside the model, no index cards, no database: everything is diluted across hundreds of billions of parameters, those little numbers adjusted during training. We’ll go look at that up close at Levels 5 and 7.

Why does ChatGPT never give the same answer twice?

Because at every word, the model doesn’t produce a single answer but a list of candidates with their probabilities — and it draws lots among the best ones. Ask the same question twice: the first draws differ, and every chosen word alters all the calculations that follow.

You’ve surely noticed it: the same question, asked twice, gives two different answers. That’s neither a bug nor some great mystery — it’s a direct consequence of what we’ve just seen. Look at what actually comes out of the “big calculation”:

On “the capital of France is…”, the model is almost sure of itself: “Paris” crushes everything. But on an open-ended sentence, the probabilities spread out — and rather than always taking the top candidate (which would produce stiff, repetitive text), the model draws lots among the most probable ones, like rolling a subtly loaded die. Two conversations, two draws, two answers.

And there’s a snowball effect: since every chosen word feeds into the calculation of all the following ones, a single different draw at the start can send the whole answer down another path. Go back to the diagram above: if the draw lands on “sleeping”, the most probable continuation heads toward “in until noon, guilt-free”; if it lands on “traveling”, off it goes toward “a few days in Italy”. Two words drawn by lot, two sentences that will never cross paths again. This randomness is actually adjustable — developers know it as “temperature” — and that’s exactly what we’ll dissect at Level 2.

What changed for me the day the black box opened

These days I spend my working life with these models — I build my projects with them, my tools too, and this very site benefits from them widely. Yet for a long time, I talked to them like a search engine: keywords, a dry request, and fingers crossed.

Understanding the mechanics in this article changed everything in my practice. Since the model does nothing but continue a text in the most probable way, my message is not a command: it’s the beginning of a document that the machine is going to extend. And suddenly, everything clicks:

• Giving context works, because a document that opens with rich context has far more precise continuations than a dry question.
• Giving examples works, because the model extends the pattern: show it two rewrites in the style you want, and the third will follow the same mold.
• Giving a role works (“you are a demanding proofreader…”), because it shifts the probabilities toward the texts that kind of expert would have written.

A very simple example, lived while writing for this site: “fix this text” kept sending me schoolteacher corrections that flattened my conversational tone. The day I wrote “you are a proofreader for a blog with a warm, conversational tone; fix only spelling and grammar, without rephrasing”, the result changed completely. Same model, same text — but the beginning of the document was no longer the same, so neither were the probable continuations.

And in the other direction, I learned caution: an answer phrased with perfect confidence is still a sequence of probable words. On a precise fact, a date, a reference, the confidence of the tone guarantees strictly nothing — we’ll come back to this in depth at Level 8.

What we imagine, and what actually happens

What we imagine	What actually happens
ChatGPT looks the answer up in a database	Nothing is stored as-is: the answer is calculated, word after word
It writes its answer, then displays it nicely	There is no draft: each word is picked live, before your eyes
It reads English the way we do	It sees neither letters nor words: only tokens converted into numbers
It learns from our conversation	The model is frozen after training; your chat doesn’t modify it
It’s permanently connected to the internet	Its knowledge dates from its training; web search is a separate tool

What does the rest of the dive look like?

“The Dive” series has eight levels, from the most accessible to the deepest: probabilities, tokens, embeddings, artificial neurons, the attention mechanism, training, then the ascent through data and hallucinations. Each article assumes only the previous ones, and the difficulty climbs one notch at a time.

My goal with this series is simple, and I’m handing it to you as a contract: by Level 4, you’ll have an accurate mental model of what an AI is — far more accurate than most of the articles out there on the subject. By Level 8, you’ll be able to understand what “Attention Is All You Need” is about — the 2017 research paper that launched this whole revolution. And at every level, diagrams instead of jargon.

Ready to descend to Level 2?

Let’s recap the surface: your message leaves for a data center, becomes numbers, and a loop picks every word of the answer by calculating probabilities. That’s all — and it’s dizzying.

At Level 2, we open up precisely the most intriguing part of this mechanism: those famous probabilities. Why draw lots instead of taking the best word? What exactly is this “temperature”? And why does a model tuned too “wise” become boring, while a model tuned too “hot” goes off the rails? The article arrives next week.

Questions about this first level? A notion you’d like to see covered later in the series? That’s what the comments are for, in a spirit of openness and kindness — your questions will directly feed the next articles.

– blaminhor