§6.8–6.9Using the Spam Classifier & Chapter Summary

Part III pp. 107–108 · ~4 min read

The classifier is trained — 95.67% accurate on unseen messages. The final step is the payoff: putting it to work on brand-new text. (A scope note: the source notes cover §6.8–6.9 as headings only, so this page walks the inference path using the machinery the chapter already established — nothing new is introduced.)

6.8 Classifying a new message

Every piece of the inference path was built earlier in the chapter; using the model is just running them in order:

"You are a winner you have been specially selected to receive $1000…" BPE tokenize → truncate → pad with 50256 to max_length model.eval() · torch.no_grad() frozen tower + fine-tuned cap logits = model(ids)[:, -1, :] two numbers: [not-spam score, spam score] argmax → 1 "spam" 🚫 same softmax→argmax mechanics as generation (ch4) and evaluation (§6.6) — the model just "generates" one of two possible answers
1

A new, unlabeled message arrives

No label, no loader — just raw text the model has never seen. (A textbook lottery scam, as it happens.)

1 / 5

Classifying a new SMS end to end — every stage reuses a component established in §6.3–§6.6.

6.9 Summary — what this chapter established

  • Fine-tuning comes in two flavors: classification (this chapter — a specialist over fixed classes) and instruction (next chapter — a flexible instruction-follower).
  • Supervised classification data pairs each padded token sequence with one class label; <|endoftext|> (50256) moonlights as the pad token.
  • The recipe: load pretrained weights → freeze everything → replace the output head with Linear(768 → num_classes) → unfreeze the last block + final norm → fine-tune with the ch5 loop (examples counted, accuracy reported).
  • The last token is the classification read-out because the causal mask makes it the only position informed by the whole sequence.
  • Cross-entropy on the last-token logits trains the model; accuracy — not differentiable — is what’s reported: 97.21% train / 97.32% validation / 95.67% test, in ~5 laptop-minutes.
  • Inference needs only: matching preprocessing, eval() + no_grad(), and an argmax.

One path down, one to go

The spam classifier proves the fine-tuning bargain: a few thousand labeled examples and minutes of compute turned the general-purpose foundation into a 95%+ specialist. But it can only say “spam” or “not spam”. Making the same foundation follow arbitrary natural-language instructions — summarize this, translate that, answer questions — is chapter 7’s job.

📝 Check yourself: the classifier in production

0 / 4

  1. 1.A brand-new SMS arrives. Which preprocessing must match the TRAINING setup exactly before the model sees it?

  2. 2.At inference time, which parts of ch5/ch6's training apparatus are still needed?

  3. 3.The classifier answers "spam"/"not spam" — nothing else. What would it take to make it classify, say, URGENT vs NORMAL vs LOW-priority messages?

  4. 4.Looking back at the whole chapter: which single fact made the last-token read-out valid in the first place?