§9.5b–9.6Convolution Variants II — Locally Connected, Tiled, Transpose · Structured Outputs

§9.5b Sharing less — locally connected and tiled

Sometimes we want convolution’s connectivity but not its sharing. A locally connected layer locally connected layer / unshared convolution Same sparse connectivity as convolution but EVERY connection has its own weight (a 6-D tensor, eq 9.9) — no parameter sharing. Use when a feature belongs to a specific location (e.g. a mouth in the lower half of a face). defined in ch. 9 — open in glossary (unshared convolution) keeps each output tied to a small local patch, but gives every connection its own weight:

Between the two extremes sits tiled convolution tiled convolution A compromise (Gregor & LeCun): cycle through t different kernels across space (eq 9.10). t=1 is ordinary convolution; t = output width is a locally connected layer; memory grows only ×t. defined in ch. 9 — open in glossary : cycle through $t$ kernels across space, so neighbors differ (like locally connected) but memory grows only $\times t$. All three share the same connectivity — only the weight-sharing pattern differs. Switch modes:

Two related tricks: restricted channel connectivity (each output channel sees only a subset of input channels — fig 9.15, fewer parameters without fewer units), and choosing how to share the biases (one per output channel for convolution; one per unit for locally connected; per-location if the input is fixed-size, since zero-padded border units get less input).

The three operations that train a CNN

Convolution is linear, so it is matrix multiplication by a sparse matrix. To train a conv net you need that matrix’s transpose too — the transpose convolution transpose convolution Multiplication by the transpose of the convolution matrix — the operation to back-propagate error to the inputs (and to reconstruct visibles in autoencoders/RBMs). One of the three ops (conv, kernel-gradient g, input-gradient h) needed to train a CNN. defined in ch. 9 — open in glossary . Altogether, three operations suffice for any depth:

§9.6 Structured outputs

A conv net need not output a single label — it can emit a whole tensor. For pixel labeling, $S_{i,j,k}$ is the probability that pixel $(j,k)$ belongs to class $i$, letting the model draw a segmentation mask structured output A CNN emitting a high-dimensional tensor (e.g. Sᵢⱼₖ = probability pixel (j,k) is class i) rather than a single label — enabling pixel-wise labeling / segmentation masks. defined in ch. 9 — open in glossary that follows an object’s outline. One wrinkle: pooling with large stride shrinks the output plane, so to keep it near input size you avoid pooling, emit a lower-resolution label grid, or pool with unit stride.

A powerful strategy is to make an initial label guess and then refine it using interactions between neighboring pixels — applying the same convolutions at each refinement step (sharing weights across steps) makes this a recurrent convolutional network recurrent convolutional network Iteratively refining a structured output (e.g. pixel labels) by re-applying the SAME convolutions with weights shared across steps (fig 9.17) — a recurrent network (ch10). defined in ch. 9 — open in glossary (fig 9.17):

Once each pixel is labeled, graphical models (or a CNN trained to approximate their objective) clean up the result into coherent regions, assuming contiguous pixels tend to share a label.

Next: data types, efficient convolution algorithms, and random/unsupervised features — data types, efficiency & random features.