5.4Heterogeneous Systems

Book pp. 99–101 · ~3 min read

  • region coherence
  • TLP-aware LLC management

The book closes where it opened — the CPU and GPU together — but now they share silicon, and sharing means contention. All three papers here circle one truth: a GPU hides latency and cache misses with thread-level parallelism, so when it shares memory resources with a latency-sensitive CPU, its sheer traffic volume can steamroll its partner unless policies are made GPU-aware.

Concurrency management

Kayiran et al. throttle GPU TLP based on congestion in the shared memory controllers and interconnect of an integrated system (their tiled model: two GPU cores per CPU core, justified by an SM being ~half a modern out-of-order CPU’s area). Two policies:

CPU/GPU TLP throttling (Kayiran et al.)cycle 0 / 3

Step 1: The problem: a shared memory system

On an integrated CPU+GPU, the GPU's flood of memory traffic congests the memory controllers and interconnect that the CPU also depends on — dragging down concurrent CPU applications. The fix is to throttle GPU thread-level parallelism (TLP) based on observed congestion.

CPU app  ─┐
          ├─→ shared MC + network ─→ DRAM
GPU cores ─┘   (GPU traffic drowns the CPU)

One tick = one rebalancing interval of Kayiran et al.'s CPU/GPU concurrency management (a tiled system, 2 GPU cores per CPU core, sharing memory controllers and network).

The keystone observation — throttling GPU TLP can help or hurt the GPU, but never hurts the CPU — is what makes the CPU-centric policy safe; the balanced policy’s phase 2 adds a user-tunable k so you can decide how much GPU throughput to trade for CPU relief.

Coherence and cache sharing

  • Heterogeneous system coherenceregion coherenceCoarse-grained CPU/GPU coherence: acquire permissions per region, route most traffic over an incoherent direct bus, spare the directory.Glossary → (Power et al.): the GPU’s memory traffic overwhelms a conventional per-cache-block coherence directory. Coarse-grained region coherence acquires permissions for a whole region at once; afterward, most accesses skip the directory entirely and travel over a cheap incoherent direct-access bus instead of the low-bandwidth coherent interconnect.
  • TLP-aware LLC managementTLP-aware LLC managementSample GPU cores (bypass vs MRU-insert) to detect cache sensitivity; correct utility partitioning for the GPU's 5–10× access rate.Glossary → (Lee & Kim): a shared last-level cache between CPU and GPU cores needs two corrections. First, detect GPU cache sensitivity by sampling — some GPU cores bypass the LLC, some insert at MRU; compare their performance to set the policy, since many GPU apps are cache-insensitive (TLP hides the misses). Second, fix utility-based partitioning for the fact that GPU cores generate 5–10× more LLC traffic — naive access-frequency-favoring management would hand the GPU most of the cache and starve the CPU, so the CPU’s way-allocation is boosted beyond plain utility partitioning when the GPU is cache-sensitive.
Concurrency managementMemory controllers + networkThrottle GPU TLP on congestion; balanced phase 2 protects GPU (dial k)
Region coherenceCoherence directory bandwidthPer-region permissions + incoherent direct bus
TLP-aware LLCShared last-level cacheSample GPU cache sensitivity; access-ratio-corrected partitioning
Heterogeneous CPU/GPU resource sharing. Click a row for the mechanism.

That closes the book’s tour — from why GPUs exist through the SIMT core and memory system to the research frontier. The landing page ties the whole learning path together.

Check yourself

  1. 1. In Kayiran et al.'s shared CPU/GPU system, what did limiting GPU thread-level parallelism (TLP) do?

  2. 2. Predict the policy (use the walkthrough): the balanced scheme has entered phase 2 and further throttling would raise GPU stalls at the target level well above the current level (difference > k). What does it do?

  3. 3. Why is directory bandwidth the bottleneck Power et al.'s heterogeneous system coherence attacks?

  4. 4. Lee & Kim found that managing a shared last-level cache between CPU and GPU cores needs which two corrections?

  5. 5. A recurring theme unifies §5.4's three papers. What is it?

0 / 5 correct · 5 unanswered