Artificial Neurons: An AI's Big Calculation Explained

Since Level 1, I’ve been drawing the heart of an AI as a blue box labeled “the big calculation”. Today, we open it. Welcome to Level 5, the engine room of “The Dive”: the one where you finally meet the artificial neuron, its weights, its layers — and the series’ first real equation. I can see you going pale: I promise, it fits on one line, it contains nothing but multiplications and additions, and you’re even going to compute it in your head. The basic building block of an AI is simple; what’s dizzying is the number. Come on, let’s push the door open.

The essentials

• An artificial neuron does nothing but multiply, add, filter — the equation fits on one line.
• The weights (how much each connection matters) are the famous parameters: GPT-3 has 175 billion across 96 layers, Meta’s Llama 3.1 405B 405 billion across 126 layers.
• All of the model’s knowledge lives in its weights — there is nothing else inside: no rules, no stored texts.
• A layer = a huge table of weights (a matrix); the GPUs’ job is to multiply those tables at full speed.
• The biological inspiration is real but distant: an artificial neuron is a mathematical object, not a copy of the brain.

What does an artificial neuron actually look like?

Like a weighing of arguments. The neuron receives numbers, multiplies each one by a weight that reflects its importance — positive or negative —, adds it all up, adds its bias, and outputs a single number. Here is the promised equation:

output = (x₁ × w₁) + (x₂ × w₂) + (x₃ × w₃) + b

Let’s give our neuron a job: deciding whether it’s a good time to garden. It looks at three things, each scored from 0 (not at all) to 1 (fully):

• x₁, x₂, x₃, the inputs — what the neuron receives to evaluate. Here, three weather readings: sunshine (x₁ = 0.9 on a bright day), rain (x₂ = 0.1, a few drops), the warmth of the air (x₃ = 0.8). These numbers don’t come from nowhere: in a real AI, they’re the coordinates of the Level 4 embedding — the numbers that represent your word.
• w₁, w₂, w₃, the weights — how much importance, learned during training, the neuron gives to each input. Sunshine strongly encourages (w₁ = +0.7); rain is a deal-breaker, so its weight is negative (w₂ = −0.8); warmth helps a little (w₃ = +0.4). Each input has its own: x₁ is multiplied by w₁, x₂ by w₂, x₃ by w₃.
• b, the bias — an offset added right at the end, the neuron’s starting mood. Cautious here (b = −0.3): under neutral conditions, it leans toward “no”.
• output — the single number that results, then passed on to the next neurons.

In other words: multiply each input by its weight, add the three together, add the bias. All that’s left is to run the calculation.

Take the time to redo the diagram’s calculation — that’s all the machinery of a neuron; there is nothing else to understand. One last touch remains: before passing it on, the output gets filtered (the “activation function” — the most common one merely replaces negative numbers with zero). This filter looks decorative; it is in fact indispensable, and I come back to it in the FAQ. What to remember: a neuron’s knowledge is not in its calculation, which never changes — it is in its weights. Our gardener neuron “knows” that rain ruins gardening only because w₂ is −0.8.

Why do we talk about “layers”?

Because neurons work in stacked teams: a layer is thousands of neurons reading the same inputs in parallel, each with its own weights; their outputs become the inputs of the next layer. Information thus crosses dozens of floors of calculation — hence “deep” learning. Rather than describe it, I’ll have you calculate it: we keep our gardening day — sun 0.9, rain 0.1, warmth 0.8 — and add two floors to it.

Layer 1 — three jobs. First thing to do: demote our neuron. On its own, it decided whether to garden — it was the whole model. Slipped into a first layer, it no longer decides anything: it becomes neuron A, one opinion among others, and its question shrinks along with its role — no longer “should we garden?” but “is it pleasant outside?”. Its weights, though, haven’t moved an inch, and neither has its result. Same weighing; what changed is the question asked around it. Next to it, two colleagues read exactly the same three readings — sun 0.9, rain 0.1, warmth 0.8 — each with its own job, and therefore its own weights.

• A, “is it pleasant outside?” — the weights and bias of the previous diagram, unchanged (+0.7, −0.8, +0.4, bias −0.3): (0.9 × 0.7) + (0.1 × −0.8) + (0.8 × 0.4) − 0.3 = 0.63 − 0.08 + 0.32 − 0.3 = 0.57
• B, “is the soil workable?” — sun dries it out (−0.2), recent rain softens it (+0.9), warmth matters little (+0.1), bias +0.1: (0.9 × −0.2) + (0.1 × 0.9) + (0.8 × 0.1) + 0.1 = −0.18 + 0.09 + 0.08 + 0.1 = 0.09
• C, “are the plants suffering from drought?” — sun makes it worse (+0.6), rain relieves it (−0.7), mild air makes it slightly worse (+0.3), bias −0.1: (0.9 × 0.6) + (0.1 × −0.7) + (0.8 × 0.3) − 0.1 = 0.54 − 0.07 + 0.24 − 0.1 = 0.61

Three readings of the same three numbers: A is clearly in favor, B tempers it (the ground is hard), C raises a flag (the plants are thirsty). Look at the rain: a negative weight in A, positive in B, negative in C — same input, three jobs, three signs. That is what makes a layer a team rather than a redundancy.

Layer 2 — two decisions. One floor up, two neurons no longer see the weather at all: their inputs are the three opinions, 0.57, 0.09 and 0.61 (all positive — the activation filter mentioned earlier had nothing to touch up, and that will hold on every floor of this example). And they no longer answer the same question: the layer has split the problem into two decisions.

• D, “a good window for planting?” — it listens to A (+0.8) and above all B (+0.9), and distrusts C (−0.6), bias −0.1: (0.57 × 0.8) + (0.09 × 0.9) + (0.61 × −0.6) − 0.1 = 0.456 + 0.081 − 0.366 − 0.1 = 0.071, call it 0.07
• E, “should we water?” — A barely interests it (+0.1), B holds it back (−0.3), C settles it (+0.9), bias 0: (0.57 × 0.1) + (0.09 × −0.3) + (0.61 × 0.9) = 0.057 − 0.027 + 0.549 = 0.579, so 0.58

Two opposite verdicts, drawn from the same three opinions: D says no, E says yes.

Layer 3 — the decision. A last neuron, F, reads only those two verdicts, 0.07 and 0.58.

• F, “what do we do in the garden today?” — it weighs D (+0.9) and E (+0.6), bias −0.1: (0.07 × 0.9) + (0.58 × 0.6) − 0.1 = 0.063 + 0.348 − 0.1 = 0.311, so 0.31

The diagram below lays out all six calculations, one pill per neuron.

Now reread the badges on the diagram, left to right. Alone, neuron A said 0.57: go out, the weather is fine. Three floors later, the network says 0.31 — a lukewarm “yes” — but above all it says what: planting gets only 0.07, watering 0.58. We go out, yes: to water, not to plant. Nobody wrote that conclusion anywhere; none of the six neurons did anything but multiply and add, none of them “knows” what a garden is. Each floor works on the conclusions of the floor before it: it’s the assembly that produces nuance, not the neuron. Notice the shape of the building, too: three inputs, three neurons, then two, then one — the layers narrow toward the decision. Go from three neurons per layer to tens of thousands, and from three floors to 96: that’s GPT-3.

In a language model, what enters the first layer are the embeddings from Level 4. Then, floor after floor, the representations get refined — the early layers tend to capture local regularities of the language, the deep layers more abstract things. I owe you an honesty warning here: this “floors of understanding” description is a comfortable simplification, and the reality observed by interpretability researchers is more entangled. What is certain is the mechanism: weighings, whose results are weighed, whose results are weighed — 96 times in a row for GPT-3.

Where are the famous “parameters” everyone talks about?

You have already met them: the parameters are the weights (and the biases). Our toy neuron had 4; GPT-3 has 175 billion, Meta’s biggest Llama 405 billion. So when we say a model “has 405 billion parameters”, we’re counting its adjustable numbers — the total volume of its settings.

And here comes the junction with everything we’ve seen, the moment the dive closes its technical loop. The weights of a layer arrange themselves naturally into a huge table of numbers — in mathematics, a matrix. Passing your embeddings through a layer means multiplying those tables together. Now, remember Level 1: GPUs, those graphics cards invented for video games, excel precisely at multiplying enormous tables of numbers. That’s why the world’s AI runs on gamer hardware: generating one word means marching matrices through 96 floors of multiplications.

Model (year)	Parameters	Layers
GPT-2 (2019)	1.5 billion	48
GPT-3 (2020)	175 billion	96
Llama 3.1, the largest (2024)	405 billion	126
Small “local” models (Ollama…)	1 to 20 billion	16 to 40

How does this calculation produce meaning?

By deforming the embedding space, layer after layer. Each floor moves the word-points around; over the floors, the representation of your sentence migrates toward a position from which the final operation can directly read the probabilities of the next word — our bag of tokens from Level 2.

It’s the image that ties Levels 4 and 5 together. Take the map of meaning again: your sentence enters as a constellation of points. The first layer moves them a little, the second a little more, and so on — the big calculation is a choreography: 96 successive displacements in the space of meaning. Type in “The garden strawberry is”: over the layers, the representation drifts toward the region where “red”, “sweet” and “ripe” are the natural outputs, and away from the one where “traffic jam” lives.

One confession to finish: why does this learned choreography work so well? The question isn’t fully resolved — network interpretability is an active field of research, not a closed chapter. We know how to build these machines, train them, measure them; we only partially know how to tell what each layer is doing. I’d rather say it plainly: at the bottom of the sea, there is still dark water.

The day I heard the billions of multiplications

Want to make all this physical? Run a small model at home. With a free tool like Ollama, an open source model of a few billion parameters installs in ten minutes on a recent computer.

What struck me most the first time wasn’t the quality of the answers: it was the fan. You ask a question, and the machine starts blowing — every generated word is billions of multiplications heating up your processor, the big calculation made audible. You also get to feel the difference in size: a small 3-billion-parameter model answers fast and sometimes foolishly; an 8-billion is already sharper but demands its memory — count roughly 1 GB per billion parameters in compressed form. After that, GPT-3’s “175 billion” stops being a marketing number: you know what your machine endures for a tiny fraction of it.

Where does the dive stand?

Ready to descend to Level 6?

Let’s sum up the engine room: neurons that weigh, stacked into layers that deform the space of meaning, and billions of weights where all the knowledge resides. Multiply, add, filter — 96 floors in a row.

But one part is missing, and not just any part. The network I’ve just described treats each position in the sentence a little too much in isolation. Yet in “the mouse the cat was chasing was gray”, which one is gray? To find out, the word “gray” has to go look at “mouse”, peering right over the shoulder of “cat”. This gaze the words cast at one another is attention — the mechanism that gave its name to the paper “Attention Is All You Need” and its architecture to all of modern AI. Level 6, next week: we finally dissect the centerpiece — including the softmax promised back at Level 2.

New to the series? Everything starts at Level 1, the journey of your message. And the comments are waiting for you, in a spirit of openness and kindness.

– blaminhor