Interactive edition
Natural Language Processing with Transformers
An interactive study companion to the book by Lewis Tunstall, Leandro von Werra, and Thomas Wolf (O'Reilly, Revised Edition, 2022). Self-attention, question answering, and the frontiers of transformer research — rebuilt as labeled diagrams, step-through derivations, live attention widgets, and quizzes. The same ideas, made visual.
- ⚡ Live attention labs
- 🧮 Step-through derivations
- 📊 Interactive plots & heatmaps
- 🖼️ Every figure redrawn as SVG
- ✅ Quizzes with explanations
Read a chapter straight through, or jump to any section below. Use ← / → to move between sections; the sidebar tracks what you've read.
Part I — Transformer Internals
How a transformer actually works under the hood — building self-attention from scratch, stacking it into an encoder and decoder, and touring the zoo of BERT/GPT/T5-style models that grew from the original 2017 architecture.
Part II — Building Applications
Building a real review-based question-answering system end to end — span-based reading comprehension, retriever–reader pipelines with Haystack, and how to measure and improve every stage.
Part III — Frontiers
Where transformers are headed — scaling laws and their limits, making self-attention efficient, and moving beyond text into vision, tables, speech, and multimodal models.