Recommender Systems · Knowledge Representation & Reasoning

§12.5.1 Recommender systems

A huge slice of commercial machine learning is recommendation — online advertising and item recommendation. Both predict a user–item association: usually framed as supervised learning (predict a click, a rating, a purchase), either a regression or a probabilistic classification.

Early systems had only user and item IDs, so the only way to generalize was by similarity between preference patterns — collaborative filtering . The workhorse is a bilinear prediction from learned user and item embeddings:

Bilinear rating prediction (eq 12.20)

\hat R_{u,i} = b_u + c_i + \sum_j A_{u,j}\, B_{j,i}

$A_{u,\cdot}$	the user embedding — a learned vector of user u's tastes	user vector
$B_{\cdot,i}$	the item embedding — a learned vector of item i's attributes	item vector
$\sum_j A_{u,j} B_{j,i}$	their DOT PRODUCT — high when the user's tastes align with the item's attributes	affinity
$b_u,\ c_i$	per-user and per-item biases — how generous the user is, how popular the item is	biases

Drag the user embedding and watch the predicted ratings follow the geometry — items the user’s vector points toward score highest:

Fig (interactive) — bilinear collaborative filtering. Items sit at fixed points in a 2-D taste space; the predicted rating is a user bias + item bias + the dot product of the user and item embeddings. Similar users → similar predictions.

A₁ 0.80A₂ 0.40bias b_u 0.00

Die Hard

4.5★

Mad Max

4.1★

Notebook

3.3★

Titanic

3.7★

Superbad

2.7★

Bridesmaids

2.0★

Predicted rating R̂ = b_u + c_i + A·B — the dot product of the user embedding with each item embedding. This user's vector points toward Die Hard (action), so that item scores highest. Two users with similar embeddings get similar predictions — that's collaborative filtering. A brand-new user has no embedding yet: the cold-start problem.

One way to obtain these embeddings is a singular value decomposition of the ratings matrix (the SVD and bilinear prediction were components of the Netflix-prize winners) — but it’s better to minimize squared error on observed ratings by gradient descent, so missing entries cost nothing. A basic limitation: a brand-new user or item has no history, so its similarity to others is unknown — the cold-start problem . The fix is content-based recommenders that map rich user/item features to an embedding (e.g. a CNN embedding a song’s audio, so it’s recommendable before anyone rates it).

§12.5.1.1 Exploration vs exploitation

Recommendation isn’t quite ordinary supervised learning. When you use the system to collect data, you only see how users respond to the items you recommended — not the others. This makes it a contextual bandit : given a context (the user), take one action (recommend an item), observe a single reward.

Exploitation vs exploration

Exploitation takes the action the current policy thinks is best (a known good reward). Exploration tries other actions to gather data — you might do better, or worse, but you learn either way. Supervised learning has no such tradeoff: the correct label is always given. Without exploration, a recommender can keep picking a wrong item forever, because the right one started with low probability and never gets tried. How much to explore depends on factors like the time horizon (short horizon → exploit; long horizon → explore early).

§12.5.2 Knowledge representation, reasoning, question answering

Embeddings capture semantic knowledge about words; a research frontier is embeddings for relations between entities. A relation is a syntactically simple sentence — a triplet of a subject, a verb, and an object:

Relations and attributes (eqs 12.21–12.23)

(\text{subject},\ \text{verb},\ \text{object}) = (\text{entity}_i,\ \text{relation}_j,\ \text{entity}_k), \qquad (\text{entity}_i,\ \text{attribute}_j)

$(\text{entity}_i, \text{relation}_j, \text{entity}_k)$	a relation triplet, e.g. (dog, is_a_type_of, mammal)	fact
$(\text{entity}_i, \text{attribute}_j)$	an attribute — a relation taking one argument, e.g. (dog, has_fur)	property

A knowledge base (Freebase, WordNet, …) stores many such triplets. A model can learn embeddings for entities and relations by treating each triplet as a training example — often entities as vectors and relations as operator matrices acting on them:

A knowledge graph. Solid edges are known relations; link prediction proposes a missing arc (dashed) — a new fact inferred from old ones.

A practical use is link prediction — predicting missing arcs in the knowledge graph (most knowledge bases are hand-built and incomplete). It’s evaluated by ranking held-out true facts against corrupted ones (e.g. precision-at-10%). Combined with reasoning and language understanding, this points toward question answering — which today relies on explicit memory (§10.12: memory networks, with GRU readers) to hold and retrieve facts.

End of Part II

This closes Part II — the most successful modern deep-learning methods, all built on using the gradient of a cost function to fit a model that approximates a desired function. With enough data, this is extraordinarily powerful. Part III steps into research territory: methods for less data, or a wider variety of tasks, where the challenges are harder and less settled.

Check yourself — recommender systems and knowledge representation

1.How is the recommendation problem usually framed as machine learning?

2.What is collaborative filtering with bilinear prediction? (See the RecommenderLab.)

3.What is the cold-start problem, and how is it addressed?

4.Why does recommendation involve exploration vs exploitation, unlike ordinary supervised learning?

5.How does the book propose representing knowledge for reasoning?

6.What is link prediction, and how does it connect to question answering?

6 questions