Tale & Theory

Books / Learning comic

Attention Is All You Need - The Transformer Comic

Theme: Computing & AI

Across twelve chapters, one English–French translation problem becomes a complete explanation of the original 2017 Transformer. Fictional conversations among the paper's authors lead from token IDs and embeddings through queries, keys, values, scaled dot-product attention, multiple heads, and position encodings. Worked arithmetic makes attention concrete before the series assembles encoder and decoder stacks, causal masking, cross-attention, vocabulary prediction, and sequential generation. Training chapters introduce teacher forcing, loss, label smoothing, gradients, Adam, and dropout. Historical experiment results are kept separate from invented examples, with careful distinctions between BLEU, accuracy, development scores, and test scores. The final walkthrough and misconception challenges connect the mechanisms while emphasizing what the model's architecture, attention weights, and illustrative translation cannot prove.

Start reading →View in the interactive library
Attention Is All You Need - The Transformer Comic cover

About this book

This twelve-chapter comic develops the original encoder–decoder Transformer through one recurring task: translating “I love music” into “J’aime la musique.” The paper's eight authors appear as fictional teaching voices, with explicit reminders that their roles do not reconstruct historical conversations or allocate credit for individual ideas. The opening failure of word-by-word substitution motivates learned representations and sentence context. Arbitrary vocabulary IDs select embedding vectors; they are not numerical measures of meaning. A recurrent-state comparison explains the dependency chain that self-attention replaces for source processing. The attention mechanism is then opened carefully. Separate learned projections create queries for receiving positions, keys for comparison, and values carrying information. Scaled dot products pass through softmax, producing normalized weights that mix whole value vectors. A complete numerical example follows scores [1, 0, 1] through scaling, exponentiation, normalization, and a two-coordinate output. Every attention distribution belongs to a particular query, while generation depends on earlier chosen output tokens even without recurrent layers. Readers distinguish source-position weights from output coordinates, including why a zero score still contributes. Multiple heads supply parallel learned views whose outputs are concatenated and projected, without assigning guaranteed grammatical jobs to individual heads. Position information resolves the next missing ingredient. Equal-width positional vectors are added to scaled embeddings, and sine-and-cosine patterns are compared with learned position embeddings without claiming unlimited-length generalization. The encoder combines multi-head self-attention, residual routes, post-addition layer normalization, and positionwise feed-forward networks. Six separately learned base-model layers preserve one 512-coordinate row per English position, with feed-forward interiors of width 2048. This contextual output is a source representation, not a translation or a prediction of its length. The decoder starts with a beginning marker and predicts one French token at a time. Shifted reference inputs and causal masks prevent future-token leakage during parallel training; padding masks serve a separate purpose. Cross-attention supplies decoder queries with keys and values projected from the fixed encoder output. After six decoder layers, vocabulary projection and another softmax produce token probabilities. An invented greedy path reaches the end marker, while the paper's beam-search evaluation is identified separately. Teacher forcing, negative log-probability loss, label smoothing, backpropagation, Adam's warmup schedule, and dropout explain how adjustable parameters acquire useful behavior. The experimental chapter preserves context around every historical number. It distinguishes base from big configurations, their training requirements, German and French benchmarks, development and test evaluations, checkpoint averaging, and separately trained ensembles. BLEU is neither percentage accuracy nor proof of understanding. Full attention's quadratic pair growth qualifies its advantages in parallelism and short information paths. An end-to-end reconstruction then connects embeddings, the encoder, reusable source memory, the growing French prefix, token selection, and detokenization. The final challenges separate parameters from activations, attention weights from vocabulary probabilities, and autoregressive generation from recurrent layers. New tiny examples show why weights alone do not determine an attention output or explain an entire prediction. Throughout, reader checks and explicit limits encourage a checkable account of what enters each calculation, what emerges, which numbers are learned, and which numbers exist only to teach.

Published chapters

  1. 01 - From words to vectors
  2. 02 - Choosing what to mix
  3. 03 - Calculate one mixture
  4. 04 - Keep more than one view
  5. 05 - Give the sentence its order
  6. 06 - Build the English reader
  7. 07 - Start writing French
  8. 08 - Consult the English
  9. 09 - Teach the translator
  10. 10 - Put the translator to the test
  11. 11 - One translation, end to end
  12. 12 - Make the explanation yours