Glossary

Every term defined across the book, grouped by the chapter that introduces it. Inline dotted terms throughout the site link here.

109 of 109 terms

Chapter 1 · Introduction

bandwidth filter: The role of per-core L1 caches: reducing traffic sent to lower levels of the memory system.
computation accelerator: Specialized hardware that trades generality for energy efficiency (up to ~500× vs. general-purpose cores).
Dennard Scaling: Classical transistor-scaling rules whose ~2005 breakdown ended easy clock-frequency gains, pushing architecture toward specialization.
device memory: The GPU's DRAM (throughput-optimized, GDDR) in a discrete-GPU system.
discrete GPU: GPU on an add-in card with its own DRAM, connected to the CPU by a bus (e.g. PCIe).
driver: CPU-side software that conveys the kernel, thread count and data locations to the GPU and signals it to start.
dynamic parallelism: The ability to launch new GPU threads from code already running on the GPU.
integrated GPU: GPU sharing one die and one DRAM space with the CPU (e.g. AMD APUs, mobile SoCs).
kernel: The code a GPU-computing application launches to run across many GPU threads.
latency hiding: Using many ready threads so a core keeps issuing while other threads wait on memory.
memory partition: A slice of last-level cache paired with a memory channel, connected to cores via the on-chip interconnect.
performance valley: Guz et al. model region where threads overflow the cache but are too few to hide memory latency (between the MC and MT regions).
scratchpad memory: Fast per-core, software-managed memory through which threads on a core communicate.
SIMT: Single-instruction, multiple-thread: the execution model GPU cores use to run thousands of scalar threads on SIMD hardware.
SIMT core: This book's name for a GPU core; NVIDIA "streaming multiprocessor (SM)", AMD "compute unit (CU)".
system memory: The CPU's DRAM (latency-optimized, DDR) in a discrete-GPU system.
Tensor Core: Volta-generation unit specialized for machine-learning matrix operations.
Turing Complete: Able to run any computation given enough time and memory; what separates GPUs from fixed-function accelerators.
unified memory: Software/hardware support (NVIDIA Pascal+) that automatically migrates data between CPU and GPU memory via virtual memory.
vertex/pixel shaders: The first programmable GPU stages (GeForce 3, 2001), the seed of GPGPU.

Chapter 2 · Programming Model

barrier: Hardware-supported instruction letting all threads of a CTA synchronize cheaply.
constant memory: Banked read-only memory (SASS c[bank][offset]) holding e.g. kernel parameters; readable by non-load/store instructions.
control instructions: Kepler+ scheduling words (1 per 3 instructions on Maxwell/Pascal) carrying stall counts, yield hints and dependency barriers — replacing hardware scoreboard checks.
CUDA Dynamic Parallelism (CDP): Kepler-era feature letting kernels launch kernels, targeting load imbalance in irregular applications.
exec mask: GCN special register predicating which vector lanes execute; the mechanism behind if/else on SIMT hardware.
global data store (GDS): AMD GCN's GPU-wide scratchpad shared by all compute units.
global memory: The address space accessible to all threads; slower and more energy-costly than shared memory; the only way threads in different CTAs communicate.
grid: The full set of thread blocks launched by one kernel call.
HSAIL: AMD's virtual ISA in the Heterogeneous System Architecture stack.
kernel configuration syntax: CUDA's <<<blocks, threads>>> launch notation setting grid/CTA dimensions.
local data store (LDS): AMD GCN's per-CU scratchpad memory, also used to pass parameters between graphics shaders.
operand reuse cache: Maxwell+ structure implied by .reuse flags: lets an operand be read multiple times per register-file access, saving energy.
predication: Executing an instruction conditionally under a predicate register (@%p1 in PTX, @P0 in SASS) instead of branching.
PTX: NVIDIA's documented virtual RISC-like ISA (limitless virtual registers); the compiler target that gets lowered to hardware code.
S_WAITCNT: GCN instruction waiting until outstanding-operation counters (vector memory, LDS/GDS, export) drop below a threshold; software-managed dependency resolution.
SASS: NVIDIA's hardware ISA ("Streaming ASSembler"); only partially documented, free to change every generation.
SAXPY: Single-precision a·x + y (BLAS); the book's running example kernel.
scalar vs. vector instructions: GCN split: v_ ops compute per-thread values on 4 vector units; s_ ops compute one value shared by the wavefront on a scalar unit.
shared memory: NVIDIA's name for the per-SM scratchpad (16–64 KB), declared with __shared__; a software-controlled cache.
thread block / CTA: Cooperative thread array: a group of warps that shares a core's scratchpad and can barrier-synchronize; the unit assigned to a SIMT core.
warp: NVIDIA's group of 32 scalar threads executed in lockstep on SIMD hardware; the book's generic unit of SIMT execution.
wavefront: AMD's equivalent of a warp: 64 threads executed in lockstep.

Chapter 3 · The SIMT Core

active mask: Bit per thread of a warp saying who executes the current instruction (e.g. 1110 = threads 1–3 on, thread 4 masked).
affine variable: Value that is base + stride·threadID; storable as a (base, stride) pair.
branch divergence: Threads of one warp taking different targets at a data-dependent branch, forcing serialized execution of each path.
convergence barrier: Volta-style stackless divergence handling: barrier participation mask + per-thread state; ADD arms, WAIT blocks, scheduler interleaves warp splits freely.
dynamic warp formation (DWF): Compacting same-PC threads from different warps into new warps at divergence; sensitive to scheduling, breaks implicit warp sync.
Independent Thread Scheduling: NVIDIA's (Volta) name for stackless divergence management that lets diverged paths interleave and locks make progress.
instruction buffer (I-buffer): Per-warp staging storage after fetch/decode enabling a second scheduling loop and I-cache miss hiding.
instruction replay: Re-executing an instruction from the I-buffer on structural hazard instead of stalling the pipeline.
likely convergence points: Extra stack fields (LPC/LPos) capturing probable early reconvergence (e.g. before loop breaks) ahead of the IPDOM.
MSHR: Miss-status holding register: tracks an outstanding cache miss so further misses to the same line merge instead of replaying to memory.
multi-path execution: Letting multiple warp-splits of one warp interleave (DWS, dual-path, MPM) to recover TLP during divergence.
operand collector: Collector units buffering each instruction's source operands so an arbiter can overlap reads across banked register files; the third scheduling loop.
reconvergence point / immediate post-dominator: Earliest program point where diverged threads are compile-time guaranteed to rejoin; where stack entries pop.
register bank conflict: Two needed operands living in the same single-ported bank, serializing their reads.
register file cache (RFC): Small cache over the main register file exploiting that ~70% of values are read once; paired with liveness bits to skip dead writebacks.
register file virtualization: Renaming physical registers on demand (+ final-read metadata) to halve RF size or double threads without loss.
scalarization: Executing uniform/affine work once on a scalar path (or one gated lane) instead of per-lane.
scoreboard (GPU): Coon et al. design: 3–4 entries/warp of pending destination registers; instructions carry a dependency bit-vector cleared on writeback, issue when all-zero.
SIMD efficiency: Fraction of SIMD lanes doing useful work; divergence's main casualty.
SIMT deadlock: Stack-induced deadlock: a lock-winner waits at the reconvergence point while spinning losers hold the TOS, so the lock is never released.
SIMT stack: Per-warp stack of (reconvergence PC, next PC, active mask) entries that serializes divergent paths and reconverges them; TOS decides what issues next.
swizzled register layout: Offsetting each warp's registers across banks (w1:r0→bank1) so equal-progress warps hit different banks.
thread block compaction (TBC): Block-wide SIMT stack: compaction only at divergence/reconvergence, warps of other blocks keep the core busy.
thread frontiers: Per-thread PCs + topologically sorted code; always run the lowest-PC threads so laggards catch up — no stack needed.
two-level warp scheduler: Small active warp pool issues; inactive warps swap in on long-latency ops — shrinks RFC/FRF working set.
uniform variable: Same value for every thread in a kernel; storable in one scalar register.
warp compaction: Research family merging threads scattered across diverged static warps into denser dynamic warps (DWF, TBC, LWM, CAPRI…).
warp split: Subset of a warp's threads sharing a PC after divergence; the schedulable unit in stackless/multi-path designs.

Chapter 4 · Memory System

cache bypassing: Sending selected accesses around a cache (on associativity stalls, low-reuse PCs, non-all-hit warps…) to cut contention and queueing.
L2 slice: Half of a partition's L2: own tags + data, in-order, sectored; write-miss sectors fully covered by coalesced stores skip the DRAM read.
local memory: Per-thread private memory space, used mostly for register spills and stack data; cached write-back in L1.
MCM-GPU: Multi-chip-module GPU: several GPU modules on one package with local caching of remote data, locality-aware CTA scheduling, first-touch pages — near-monolithic performance past reticle limits.
memory access scheduler: The partition's "frame buffer" logic reordering DRAM reads/writes (read request sorter + store tables) to hit open rows.
memory coalescing: Merging a warp's per-thread global accesses that fall in one cache block into a single memory request; uncoalesced access = multiple requests.
MRPB: Memory request prioritization buffer: warp-ID-signature FIFOs reordering requests just before the L1 to restore locality after scheduling.
pending request table (PRT): The GPU L1's MSHR-like structure: merges misses to the same block and remembers which deferred access to replay when the fill returns.
protection distance: Bypass technique giving each inserted line N protected accesses before it may be evicted; inserts finding no unprotected line bypass instead.
ROP unit: Raster operations unit in each partition: alpha blending, surface compression, and the GPU's atomic/reduction operations (with a small cache to pipeline same-address atomics).
row buffer: The DRAM bank's staging row: precharge + activate move a page into it; hits in the open row are fast, switches cost dead cycles.
row-buffer locality: Consecutive requests touching the same DRAM row of the same bank; created by one SM's access stream, destroyed by interleaving many SMs.
sector: 32-byte quarter of a 128-byte cache line, matching the GDDR5 access atom; Maxwell/Pascal L1s and L2 slices are sectored.
shared-memory bank conflict: Two threads targeting different locations in the same shared-memory bank in one cycle; the arbiter splits the access and replays the conflicting part.
texture cache: Throughput-first L1: the tag array runs a memory-round-trip AHEAD of the data array, a fragment FIFO delays hits, a reorder buffer keeps fills in order — hits and misses see similar latency.
texture mapping / texel: Applying an image (texture) to a 3D surface; a texel is one texture sample — neighboring pixels read neighboring texels, giving caches their locality.
virtually-tagged L1: GPU L1s are virtually indexed AND tagged (unlike CPU VIPT): no per-cycle translation needed, acceptable because all warps share one address space.

Chapter 5 · Crosscutting Research

cache access re-execution (CAR): Side buffer for stalled memory instructions so hits-under-miss can proceed past a backed-up load/store unit.
CCWS: Cache-conscious wavefront scheduling: throttle actively-scheduled warps when victim-tag hits show intra-warp locality being lost to cache thrashing.
Chimera: GPU preemption picking per-CTA among context save, drain-to-finish, or idempotent restart to hit a context-switch latency target.
CTA throttling: Limiting resident threadblocks per core (watching idle vs memory-delay cycles) to cut memory-system contention from over-subscription.
DAWS: Divergence-aware warp scheduling: predict each warp's LOOP cache footprint (shrinking as threads exit) and throttle preemptively.
DTBL: Dynamic thread block launch: aggregate device-launched blocks onto running kernels of the same code, fixing CDP's tiny-kernel overheads.
Equalizer: Runtime that tunes CTA count, core frequency and memory frequency from four warp-state counters, in energy-saving or performance mode.
fetch group: Subset of warps a two-level scheduler rotates through, letting groups reach long-latency ops at different times to preserve locality.
hardware worklist: Per-lane on-chip queues with greedy/needy rebalancing (and inter-core sorting) for data-driven irregular parallelism.
idempotent restart: Dropping a CTA without saving state because re-executing it from the start is provably safe (relaxed idempotence).
intra-warp locality: A warp reloading data it itself brought in; the dominant locality in cache-sensitive GPU kernels (vs inter-warp sharing).
Kilo TM: First GPU hardware TM: word-granularity, value-based conflict detection scaled to tens of thousands of small transactions.
Mascar / owner warp: When MSHRs/miss queues saturate, one owner warp keeps memory priority while others' compute overlaps; EP↔MAP mode switch.
recency bloom filter: Time-ordered bloom filter compressing committing write-sets so hundreds of non-conflicting transactions commit in parallel (few kB, on-chip).
region coherence: Coarse-grained CPU/GPU coherence: acquire permissions per region, route most traffic over an incoherent direct bus, spare the directory.
span / split: Nested-parallel-pattern knobs: how many collection elements one thread handles; split(n) shards span(all) across n threads + a combiner kernel.
temporal conflict detection (TCD): Implicitly synchronized on-chip timers timestamp reads so conflict-free read-only transactions commit without value checks.
threadblock scheduling: Assigning CTAs to cores in bulk (round-robin, resource-limited); no preemption — a CTA runs to completion before its resources free up.
TLP-aware LLC management: Sample GPU cores (bypass vs MRU-insert) to detect cache sensitivity; correct utility partitioning for the GPU's 5–10× access rate.
value-based conflict detection: Validate by re-reading a transaction's read-set from memory and comparing values — no global metadata, finest granularity.
victim tag array: Per-warp record of that warp's evicted L1 tags; a miss matching it signals lost intra-warp locality to the scoring system.
WarpTM: Warp-level transaction management: two-phase intra-warp conflict resolution enabling coalesced validation/commit memory accesses.