This chapter used consistency-directed consistency-directed coherence Coherence in which writes propagate asynchronously but the order writes become visible must obey the consistency model; the newer GPU-era category. defined in Chapter 2 — open in glossary protocols to keep GPUs coherent — but the idea of targeting the coherence protocol at the consistency model, rather than at the SWMR invariant, has deep CPU roots. Most of the GPU protocols of §10.1 are adaptations of classic or multicore-era CPU proposals:
| CPU-side proposal | Key idea | GPU-side descendant |
|---|---|---|
| Lazy caching — Afek, Brown & Merritt, 1993 | directly enforces SC without satisfying SWMR — the earliest consistency-directed design | conceptual ancestor of the whole family |
| Dynamic self-invalidation — Lebeck & Wood, 1995 | first self-invalidation self-invalidation The reading core invalidates blocks in its own cache to make other threads' stores visible, replacing writer-initiated invalidation — the enabling idea of GPU coherence. defined in Chapter 10 — open in glossary protocol; targets SC and weaker models directly | the self-invalidation core of §10.1’s proposals |
| Lazy release consistency — Kontothanassis, Scott & Bianchini, 1995 | defer release work until the acquire, on hardware-coherent CPUs | LRCC for GPUs — Alsop, Orr, Beckmann & Wood, 2016 (§10.1.4) |
| Library coherence — Lis, Shim, Cho & Devadas, 2011 | timestamp/lease-based coherence for multicores | Temporal coherence — Singh, Shriraman, Fung, O’Connor & Aamodt, 2013 (§10.1.3) |
| DeNovo — Choi et al., 2011 | target coherence at DRF models ⇒ simpler, scalable protocols; ownership for stores | GPU ownership protocols — Sinclair, Alsop, Komuravelli & Adve, 2015/2017 (RCC-O’s GetO); showed non-scoped ≈ scoped performance |
| VIPS — Ros & Kaxiras, 2012 | directory-less, directly enforces RC; TLBs track private/read-only data | — |
| TSO-CC — Elver & Nagarajan, 2014 | consistency-directed protocol targeting TSO | — |
| Tardis — Yu & Devadas, 2015 | timestamp “time-traveling” protocol targeting SC | Relativistic cache coherence — Ren & Lis, 2017: efficient SC on GPUs, without globally synchronized clocks |
| DRF — Adve & Hill (see §5.4) | SC for data-race-free programs | HRF — Hower et al., 2014: DRF extended with scopes for heterogeneous systems (§10.2.1) |
Two threads run through the table. Reading down the left column: the consistency-directed idea kept resurfacing on CPUs — for SC (lazy caching, DSI, Tardis), for TSO (TSO-CC), for RC/DRF (LRC, DeNovo, VIPS) — every time someone noticed that the model being enforced was weaker than what SWMR provides. Reading across: the GPU era did not invent its coherence; it ported it, adding the one ingredient CPUs never needed — scopes.
Check yourself: the consistency-directed lineage
1.What do the classic works — Afek et al.'s lazy caching and Lebeck & Wood's dynamic self-invalidation — have in common with this chapter's GPU protocols?
2.Match the GPU protocol to its CPU ancestor: temporal coherence (§10.1.3) was adapted from…
3.What did the DeNovo-for-GPUs line of work (Sinclair et al.) demonstrate about scopes?
4.Beyond caches, what accelerator memory structure remains an open integration challenge for global shared memory?
4 questions
Chapter 10 references (31)
- The AMBA CHI Specification. developer.arm.com/architectures/system-architectures/amba/amba-5
- The CCIX Consortium. ccixconsortium.com
- The GenZ Consortium. genzconsortium.org
- The OpenCAPI Consortium. opencapi.org
- T. M. Aamodt, W. W. L. Fung, and T. G. Rogers. General-Purpose Graphics Processor Architectures. Synthesis Lectures on Computer Architecture, Morgan & Claypool, 2018.
- Y. Afek, G. M. Brown, and M. Merritt. Lazy caching. ACM TOPLAS, 15(1):182–205, 1993.
- N. Agarwal, D. W. Nellans, E. Ebrahimi, T. F. Wenisch, J. Danskin, and S. W. Keckler. Selective GPU caches to eliminate CPU-GPU HW cache coherence. In HPCA, 2016.
- J. Alglave, M. Batty, A. F. Donaldson, G. Gopalakrishnan, J. Ketema, D. Poetzl, T. Sorensen, and J. Wickerson. GPU concurrency: Weak behaviours and programming assumptions. In ASPLOS, pp. 577–591, 2015.
- J. Alsop, M. S. Orr, B. M. Beckmann, and D. A. Wood. Lazy release consistency for GPUs. In MICRO, 2016.
- J. Alsop, M. D. Sinclair, and S. V. Adve. Spandex: A flexible interface for efficient heterogeneous coherence. In ISCA, 2018.
- L. Alvarez, L. Vilanova, M. Moretó, M. Casas, M. González, X. Martorell, N. Navarro, E. Ayguadé, and M. Valero. Coherence protocol for transparent management of scratchpad memories in shared memory manycore architectures. In ISCA, pp. 720–732, 2015.
- J. F. Cantin, J. E. Smith, M. H. Lipasti, A. Moshovos, and B. Falsafi. Coarse-grain coherence tracking: RegionScout and region coherence arrays. IEEE Micro, 26(1):70–79, 2006.
- B. Choi, R. Komuravelli, H. Sung, R. Smolinski, N. Honarmand, S. V. Adve, V. S. Adve, N. P. Carter, and C. Chou. DeNovo: Rethinking the memory hierarchy for disciplined parallelism. In PACT, pp. 155–166, 2011.
- M. Elver and V. Nagarajan. TSO-CC: Consistency directed cache coherence for TSO. In HPCA, pp. 165–176, 2014.
- D. R. Hower, B. A. Hechtman, B. M. Beckmann, B. R. Gaster, M. D. Hill, S. K. Reinhardt, and D. A. Wood. Heterogeneous-race-free memory models. In ASPLOS, pp. 427–440, 2014.
- R. Komuravelli, M. D. Sinclair, J. Alsop, M. Huzaifa, M. Kotsifakou, P. Srivastava, S. V. Adve, and V. S. Adve. Stash: Have your scratchpad and cache it too. In ISCA, pp. 707–719, 2015.
- L. I. Kontothanassis, M. L. Scott, and R. Bianchini. Lazy release consistency for hardware-coherent multiprocessors. In Supercomputing, 1995.
- A. R. Lebeck and D. A. Wood. Dynamic self-invalidation: Reducing coherence overhead in shared-memory multiprocessors. In ISCA, pp. 48–59, 1995.
- M. Lis, K. S. Shim, M. H. Cho, and S. Devadas. Memory coherence in the age of multicores. In ICCD, 2011.
- D. Lustig, S. Sahasrabuddhe, and O. Giroux. A formal analysis of the NVIDIA PTX memory consistency model. In ASPLOS, 2019.
- D. Lustig, C. Trippel, M. Pellauer, and M. Martonosi. ArMOR: Defending against memory consistency model mismatches in heterogeneous architectures. In ISCA, pp. 388–400, 2015.
- L. E. Olson, M. D. Hill, and D. A. Wood. Crossing guard: Mediating host-accelerator coherence interactions. In ASPLOS, pp. 163–176, 2017.
- M. S. Orr, S. Che, A. Yilmazer, B. M. Beckmann, M. D. Hill, and D. A. Wood. Synchronization using remote-scope promotion. In ASPLOS, pp. 73–86, 2015.
- J. Power, A. Basu, J. Gu, S. Puthoor, B. M. Beckmann, M. D. Hill, S. K. Reinhardt, and D. A. Wood. Heterogeneous system coherence for integrated CPU-GPU systems. In MICRO-46, pp. 457–467, 2013.
- X. Ren and M. Lis. Efficient sequential consistency in GPUs via relativistic cache coherence. In HPCA, pp. 625–636, 2017.
- A. Ros and S. Kaxiras. Complexity-effective multicore coherence. In PACT, pp. 241–252, 2012.
- Y. S. Shao, S. L. Xi, V. Srinivasan, G. Wei, and D. M. Brooks. Co-designing accelerators and SOC interfaces using gem5-aladdin. In MICRO, 2016.
- D. E. Shasha and M. Snir. Efficient and correct execution of parallel programs that share memory. ACM TOPLAS, 10(2):282–312, 1988.
- M. D. Sinclair, J. Alsop, and S. V. Adve. Chasing away RAts: Semantics and evaluation for relaxed atomics on heterogeneous systems. In ISCA, pp. 161–174, 2017.
- I. Singh, A. Shriraman, W. W. L. Fung, M. O’Connor, and T. M. Aamodt. Cache coherence for GPU architectures. In HPCA, pp. 578–590, 2013.
- X. Yu and S. Devadas. Tardis: Time traveling coherence algorithm for distributed shared memory. In PACT, pp. 227–240, 2015.