Attention Is All You Need — longer guide

This is the engine room on the lunch-box map. The map’s Quick easy read is the gist. This page is the next layer: enough detail that a busy engineer can get the Transformer without grinding the PDF.

What problem it solves

Old models read words in a single-file line. This one lets every word look at every other word at the same time.

Recurrent nets (and the “one step, then the next” habit) were hard to train on long sentences and wasted GPU parallelism. Vaswani et al. (2017) showed a stack of self-attention plus small feed-forward layers could beat those pipelines on translation. The same stack later became the default for chat, code, and pictures.

Before	After this paper
Hidden state walks left → right	Every token attends to every token in one shot
Hard to parallelise across the sentence	GPUs like the big matrix multiplies
Separate tricks for “who is who”	Attention is the alignment

How the idea works

Think of each word as a person in a meeting who can look around.

Turn words into numbers. Each token becomes a vector (a short list of numbers).
Ask, look, answer. For each token, the model builds three views of that vector:
- Query — “what am I looking for?”
- Key — “what do I advertise?”
- Value — “what do I actually pass along if you pick me?”
Score the looks. Query of bank is compared to the keys of river, money, the, … Softmax turns those scores into “how much to listen.”
Mix the values. The token’s new meaning is a weighted mix of the values it listened to.
Many heads. Several of these looks run in parallel (grammar vs topic vs punctuation). Then a small feed-forward net cleans the mix.
Stack. Repeat. Early layers catch local glue. Later layers catch “this whole sentence is about a loan.”

Put all of that into one line and you get scaled dot-product attention:

\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left(\frac{QK^{T}}{\sqrt{d_k}}\right)V

Why divide by √d_k? A key vector has d_k dimensions. The raw dot product QKᵀ grows as that number grows. If you leave the scores raw, softmax gets too sharp — one token takes almost all the weight, the rest get ~0. Dividing by √d_k keeps the scores in a friendlier range, so attention can still listen to more than one neighbour.

flowchart LR
  T[tokens] --> QKV[query / key / value]
  QKV --> A[self-attention]
  A --> FF[feed-forward]
  FF --> O[richer tokens]

Every word can look at every other word

The original paper is an encoder–decoder for translation: the encoder reads the source sentence both ways; the decoder writes the target one token at a time, attending to what it has already written and to the encoder. Later chat models often keep only a decoder (next-token prediction). Same attention idea.

Two other pieces you will meet in every later paper:

Position. Attention itself does not know order. The 2017 paper adds a seat-number signal (sinusoidal position). Later open models often use RoPE instead.
No recurrence as the main trick. Residual connections and layer-norm keep the deep stack trainable.

Why it mattered / what it unlocked

Parallel training. You can process a whole sentence (or a whole batch) without waiting for token 17 before token 18.
One engine, many jobs. BERT reads both ways. GPT-style stacks write. ViT treats image patches as tokens. Same lunch box.
The “if you only open one paper” paper. Most of this tray is “Transformer + a new habit” (position, retrieval, adapters, experts).
The cost you now feel. Attention is O(n²) in sequence length. That is why later work cares about context windows, RoPE, and sparse tricks.

The paper’s own test was machine translation on WMT 2014 English–German and English–French news, not a chatbot. The chatbot era borrowed the engine.

What to remember

Self-attention = every token asks “who matters for me right now?” and mixes those answers.
Multi-head = several of those questions at once.
Divide the scores by √d_k so softmax does not collapse onto one token.
The stack + residuals is the Transformer; attention is the new building block, not a side gadget.
Position must be added somehow. The 2017 recipe is not the only recipe.
If a later paper feels mysterious, ask: “what did they change around this engine?”

When to read the real paper

Open the PDF when you want:

encoder–decoder attention vs self-attention
the WMT setup, BLEU numbers, and the ablation table (how many heads, how deep)
the original sinusoidal position formula

Skip the grind if you only needed “why ChatGPT-shaped models exist.” You already have that.

What problem it solves

How the idea works

Why it mattered / what it unlocked

What to remember

When to read the real paper

Links