Efficient Temporal Graph Network Training via Unified Redundancy Elimination

Temporal Graph Networks learn from graphs that change over time — social interactions, transactions, clicks arriving as a stream — but training them wastes enormous effort re-doing the same computations, re-storing data that could be regenerated, and re-shipping the same features between CPU and GPU; this paper (Beihang University — Yiqing Wang, Hailong Yang, and colleagues) presents PULSE, which attacks all three kinds of waste under one "keep only the minimal unit" principle, defining a minimal input unit, a minimal storage unit, and a minimal reuse unit, and reports up to 6.67 times higher training throughput than the state of the art.

The argument in one glance

Problem

Training on graphs that change over time wastes effort everywhere

A Temporal Graph Network (TGN) is a neural network for graphs that evolve over time — think of a social network where friendships form, a payment network where transactions arrive, or a shopping site where clicks stream in. Unlike a static graph, the data is a sequence of timestamped events (interactions between nodes), and the model keeps a per-node memory of what has happened so far, updates it as new events arrive, and computes embeddings by looking at each node's recent temporal neighbors. This makes TGNs powerful for recommendation, fraud detection, and other tasks where when things happened matters. But training them is inefficient, because the pipeline is riddled with redundancy of three kinds. There is computational redundancy: the same nodes and the same neighbors recur across many events and batches, so the model recomputes messages and embeddings it has effectively already produced. There is storage redundancy: the pipeline stores messages and state for many nodes, even when a lot of that could be regenerated from other data rather than kept around. And there is data-loading redundancy: the same node features get shuttled from CPU memory to the GPU over and over, because the pipeline does not exploit the fact that batches close in time touch overlapping data. Each kind of redundancy hurts a different resource — wasted compute, bloated memory, and a saturated CPU-GPU link — and together they throttle training.
What people did

Optimize the TGN training pipeline one bottleneck at a time

Existing TGN training frameworks improve efficiency by attacking individual pain points — speeding up a particular operator, caching some data to cut transfers, or managing memory for one stage. Each optimization targets one symptom of the redundancy, in one part of the pipeline.
Why it fell short

Piecemeal fixes miss that the redundancies share a root cause

Attacking the bottlenecks one at a time leaves gains on the table, because the three kinds of redundancy are not independent problems to be patched separately — they are all manifestations of the same underlying issue: the pipeline works on more data than it fundamentally needs to. A framework that only deduplicates some computation still stores and transfers redundant data; one that only caches transfers still recomputes and over-stores; one that only manages memory for a single stage does nothing for the compute and loading waste. Without a unifying principle, each optimization is local and partial, and the redundancy that one fix removes reappears in the parts it does not touch. What was missing was a single, principled way to identify, across computation, storage, and data loading, the minimal amount of data and work the pipeline actually requires, and to eliminate everything above that minimum in a coordinated fashion.
This paper's idea

Eliminate all three redundancies under one minimal-unit principle

PULSE is an end-to-end TGN training framework that systematically eliminates redundancy across computation, storage, and data loading, guided by a single unifying idea the authors call the minimal-unit principle: for each resource, work with the smallest unit of data and computation that is actually irreducible, and derive everything else from it rather than storing or recomputing it wastefully. PULSE realizes this with three synergetic units. The Minimal Input Unit (MIU) attacks computational redundancy through component-wise deduplication and operator-level reconstruction: it identifies the repeated inputs (recurring nodes and neighbors) and reorganizes the operators so redundant computations are done once, not many times. The Minimal Storage Unit (MSU) attacks storage redundancy through dependency-guided message reconstruction: instead of storing everything, it keeps only the entries that cannot be reproduced from other data, and recovers the reproducible ones on demand by recomputing them from their dependencies. The Minimal Reuse Unit (MRU) attacks data-loading and memory redundancy through GPU memory management: it combines a BlockPool-based buffer allocator, which reduces memory fragmentation, with a bipartite temporal reuse strategy that exploits the overlap between batches close in time so data already on the GPU is reused rather than re-transferred. Because the three units share the minimal-unit principle and reinforce one another, PULSE cuts waste across the whole pipeline at once, and the authors report it improves training throughput by up to 6.67 times over state-of-the-art baselines.

Why this problem is worth solving

Temporal graph learning is where many high-value prediction tasks are heading, and training cost is what stands between a good model and a deployable one.

Many of the most valuable prediction problems are fundamentally about dynamics: which product a user will click next given their recent behavior, whether a sequence of transactions signals fraud, how influence spreads through a network as it changes. Temporal Graph Networks capture exactly this — the structure of a graph together with the timing of events on it — which is why they are increasingly adopted. But their power comes from machinery that is expensive to train: maintaining and updating per-node memory, sampling temporal neighbors, and computing time-aware embeddings, all over a stream of events. When training is slow and memory-hungry, it limits how large a graph, how long a history, or how many model variants a team can afford to explore, which directly caps model quality and iteration speed. So making TGN training efficient is not a narrow systems concern; it widens what temporal graph models can be applied to and how quickly they can be improved.

The reason the inefficiency is stubborn is that the waste is spread across three different resources but springs from one source, so fixing it demands a unified view.

TGN training stresses three different resources, and redundancy shows up in each. Computationally, because events reuse the same nodes and neighbors, the model repeats work it has effectively already done. In storage, the pipeline holds messages and state that could often be regenerated, inflating memory footprint. In data movement, the same features cross the CPU-GPU boundary repeatedly, because batches near each other in time touch overlapping data that is not reused. The trap is that these look like three separate engineering problems, and a fix aimed at one does nothing for the others — deduplicating computation leaves the storage and transfer waste intact, and vice versa. But they share a root: the pipeline is manipulating more data than is truly necessary at each step. That shared root is what makes a unifying principle valuable: if you can define, for each resource, the minimal irreducible unit and derive the rest from it, you can remove redundancy coherently across the whole pipeline rather than chasing it from one bottleneck to the next. Finding that principle and making it practical across such different resources — compute, memory, and the interconnect — is the hard part.

What people did before, and where it broke

Speed up individual operators. Optimize a specific expensive computation in the TGN pipeline. Where it broke. It addresses one slice of computational redundancy while leaving the storage and data-loading waste untouched.
Cache some data to cut transfers. Keep frequently used features on the GPU to reduce CPU-GPU traffic. Where it broke. Without a principled view of what is reusable, caching is partial, and the pipeline still recomputes and over-stores.
Manage memory for a single stage. Improve allocation or reuse within one part of the training loop. Where it broke. Fragmentation and reuse opportunities span the whole pipeline, so a single-stage fix leaves memory pressure elsewhere.
Store everything to avoid recomputation. Keep all messages and state so they never have to be regenerated. Where it broke. This trades one waste for another, inflating memory footprint with entries that could have been cheaply reconstructed.
The gap. No framework identified, across computation, storage, and data loading together, the minimal unit the pipeline truly needs and eliminated everything above it in a coordinated way.

How PULSE works

One minimal-unit principle across compute, storage, and data movement

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Do repeated work once by deduplicating the inputs. The Minimal Input Unit’s job is to notice that the same nodes and neighbors recur across events and batches, and to make the pipeline compute their contributions a single time rather than re-deriving them. Recognizing that identical or overlapping computations are being performed repeatedly and collapsing them so the work happens once is the same undertaking the work on coarse-grained deduplication to accelerate computation (Don’t Repeat Yourself) covered elsewhere on this site pursues, eliminating repeated work by identifying what is truly the same. PULSE applies this to the recurring inputs of temporal graph training, deduplicating components and reconstructing operators so redundant computation disappears.

Keep only what you cannot regenerate, and recompute the rest on demand. The Minimal Storage Unit refuses to store data that can be reproduced; it holds only the irreproducible entries and reconstructs the others from their dependencies when needed, trading a little recomputation for a much smaller memory footprint. Choosing to regenerate values on demand instead of storing them, to relieve memory pressure, is the same principle the work on adaptive recomputation in pipeline parallelism (AdaPipe) covered elsewhere on this site is built on, spending compute to save memory where the trade pays off. PULSE guides this trade with a dependency analysis, so it stores exactly the minimal set that cannot be recovered any other way.

Manage GPU memory so buffers are reused, not fragmented or re-fetched. The Minimal Reuse Unit treats GPU memory as a resource to be pooled and reused: its BlockPool allocator fights fragmentation, and its temporal reuse strategy keeps data that neighboring batches share resident so it is not fetched again. Carefully pooling and reusing GPU memory to cut fragmentation and avoid redundant transfers is the same idea the work on efficient GPU memory defragmentation (GMLake) covered elsewhere on this site develops, reclaiming memory and bandwidth lost to poor allocation. PULSE adds the temporal twist that batches close in time overlap, so reusing their shared data across the CPU-GPU boundary is a large, structured win.

Following a training batch through PULSE

Processing a batch of temporal events with minimal waste

An illustrative sketch, based on the paper's abstract, of how PULSE's three units cooperate on a batch. It is not a detailed mechanism from the paper; specifics should be read there.

Step 1 of 5

A batch of timestamped events arrives

The trainer takes a batch of interactions from the temporal graph. Many of the nodes and neighbors in this batch also appeared in nearby batches, and some appear multiple times within the batch itself.

This recurrence is the source of the pipeline's redundancy.

Why it works

Deduplicating inputs removes repeated computation at its source. The same nodes and neighbors recur across events → but computing each distinct contribution once and reusing it → so the redundant messages and embeddings are never recomputed.
Storing only the irreproducible shrinks the footprint. Keeping every message and state entry inflates memory → but a dependency analysis identifies what can be regenerated → so only the truly irreducible entries are stored and the rest are recomputed on demand.
Regenerating on demand trades cheap compute for scarce memory. Recomputation is not free → but reconstructing a value from its dependencies is often cheaper than the memory it would occupy → so the pipeline relieves memory pressure where the trade pays off.
Temporal reuse avoids re-fetching shared data. Batches close in time touch overlapping nodes and features → but keeping that shared data resident on the GPU → so it is reused instead of re-transferred across the CPU-GPU link.
A pooled allocator keeps the reuse from fragmenting memory. Reusing buffers ad hoc would fragment GPU memory → but a BlockPool allocator manages the buffers coherently → so reuse and low fragmentation are achieved together rather than trading off.

What it costs (trade-offs and scope)

Recomputation trades compute for memory. The Minimal Storage Unit saves memory by regenerating reproducible entries on demand, which spends extra computation to do so; this pays off when memory is the binding constraint, but the balance depends on how expensive the reconstructions are relative to the storage they replace.
The gains depend on how much redundancy the workload has. PULSE’s benefit comes from recurring nodes and neighbors and from overlap between temporally close batches; a workload with little such recurrence or overlap would offer less to eliminate, so the speedup varies with the graph and access pattern.
It adds coordination machinery across three units. Running deduplication, dependency-guided reconstruction, and pooled temporal reuse together, and keeping them consistent, is more complex than a single-stage optimization, which is the price of the unified approach.
It is specialized to Temporal Graph Network training. The design is built around the TGN pipeline’s structure — per-node memory, temporal neighbor sampling, and event streams; other model families would need the minimal-unit principle re-derived for their own redundancies.
This explainer is drawn from limited source material. The full paper text was not accessible for this write-up; the description here is based on the paper’s abstract, so the precise mechanisms of the three units, the dependency analysis, and the detailed results should be confirmed in the paper.

How it compares

Eliminating redundancy in Temporal Graph Network training

Property	Piecemeal single-bottleneck fixes	Store-everything baseline	PULSE
Computational redundancy
Storage redundancy
CPU-GPU transfers
GPU memory fragmentation
Guiding principle

A qualitative comparison, based on the paper's abstract, of ways to make TGN training efficient. PULSE unifies redundancy elimination across computation (Minimal Input Unit), storage (Minimal Storage Unit), and data loading and memory (Minimal Reuse Unit) under one minimal-unit principle, and the authors report up to 6.67 times higher training throughput than state-of-the-art baselines. Specific per-benchmark figures should be read from the paper.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is based on the paper’s abstract, not its full text, which was not accessible for this write-up. The system and claims described are the authors’: PULSE is an end-to-end Temporal Graph Network training framework that systematically eliminates redundancy in computation, storage, and data loading, guided by a unified minimal-unit principle, via three synergetic units. The Minimal Input Unit (MIU) targets computational redundancy through component-wise deduplication and operator-level reconstruction; the Minimal Storage Unit (MSU) targets storage redundancy through dependency-guided message reconstruction, preserving only irreproducible entries and enabling on-demand recovery of the rest; and the Minimal Reuse Unit (MRU) targets GPU memory management, combining a BlockPool-based buffer allocator with a bipartite temporal reuse strategy to mitigate fragmentation and exploit inter-batch locality. The authors report that on representative benchmarks PULSE improves training throughput by up to 6.67 times over state-of-the-art baselines. The 6.67 times figure is the authors’ reported headline result; no other quantitative results, figure contents, or precise mechanisms are asserted here, because the full paper was unavailable. Exact algorithms for the three units, the dependency analysis, the experimental setup, and full results should be read from the paper. This page is marked as a draft for that reason.

Background

Why eliminating redundant work is a perennial systems lever

It helps to see PULSE within a recurring theme in systems: much of the performance left on the table comes not from doing hard things slowly but from doing the same thing more than once. Across domains, the biggest wins often come from spotting redundancy — repeated computation, duplicated storage, re-fetched data — and doing the work only once. What makes this hard is that redundancy hides in different resources and different layers, so it is tempting to attack each occurrence locally and miss that they share a cause. Machine-learning training on structured data is especially prone to this, because the data has overlap by nature: graphs reuse the same nodes, batches near each other in time share features, and layers recompute intermediate values. The productive move, repeated in many efficient training systems, is to find a principle that identifies the minimal necessary work and data and to remove everything above it coherently, rather than patching bottlenecks one by one. Graph learning sharpens the challenge because the computation is irregular and data-dependent, so the redundancy is not obvious to a generic optimizer. The broad effort to run graph neural network computation efficiently by exploiting the structure of the workload is the same terrain the work on high-performance graph operator computation via a unified abstraction (uGrapher) covered elsewhere on this site occupies, extracting efficiency from how graph computation is actually organized. PULSE contributes a unifying redundancy-elimination principle for temporal graphs specifically, arguing that computation, storage, and data movement should all be pared down to their minimal units together.

If you remember only three things

1. Temporal Graph Networks learn from graphs that evolve as a stream of timestamped events, maintaining per-node memory and computing time-aware embeddings — powerful for recommendation and fraud detection, but expensive to train. The training pipeline is plagued by three kinds of redundancy that hurt three different resources: computational (recurring nodes and neighbors are recomputed), storage (state is kept that could be regenerated), and data loading (the same features are re-transferred between CPU and GPU). Existing frameworks fix these piecemeal, missing that they share one root: the pipeline works on more data than it needs.
2. PULSE eliminates all three under a single minimal-unit principle, with three cooperating units. The Minimal Input Unit deduplicates recurring inputs and reconstructs operators so redundant computation is done once. The Minimal Storage Unit uses a dependency analysis to store only entries that cannot be regenerated and recomputes the rest on demand, shrinking memory. The Minimal Reuse Unit manages GPU memory with a BlockPool allocator to cut fragmentation and a bipartite temporal reuse strategy that reuses data shared by temporally close batches instead of re-transferring it.
3. The authors report that PULSE improves TGN training throughput by up to 6.67 times over state-of-the-art baselines. The trade-offs: the storage unit spends recomputation to save memory (worthwhile when memory is the constraint), the gains depend on how much recurrence and inter-batch overlap the workload has, the unified approach adds coordination complexity, and it is specialized to the TGN pipeline. This explainer is a draft based on the abstract; the precise mechanisms and full results should be confirmed in the paper.

Check yourself

Check your understanding

1. What three kinds of redundancy plague Temporal Graph Network training, and why do they matter?
2. What is PULSE's unifying idea, and how do its three units embody it?
3. Why does the Minimal Storage Unit store only "irreproducible" entries instead of everything?
4. Predict: two consecutive training batches in a temporal graph touch many of the same nodes and features. How does PULSE's Minimal Reuse Unit exploit this, and what would happen without it?

Sources

The paper is “Efficient Temporal Graph Network Training via Unified Redundancy Elimination” by Yiqing Wang, Hailong Yang, Kejie Ma, Enze Yu, Pengbo Wang, Xin You, Qingxiao Sun, Chenhao Xie, Zhongzhi Luan, Yi Liu, and Depei Qian (Beihang University), published at ASPLOS 2026, DOI 10.1145/3779212.3790157.

This explainer is written from the paper’s abstract, because the full paper text was not accessible for this write-up (the ACM version was not retrievable and no author preprint was found). PULSE is an end-to-end Temporal Graph Network training framework that systematically eliminates redundancies in computation, storage, and data loading, guided by a unified minimal-unit principle. It defines three synergetic units: the Minimal Input Unit (MIU) for component-wise deduplication and operator-level reconstruction of redundant computations; the Minimal Storage Unit (MSU) for dependency-guided message reconstruction, preserving only irreproducible entries while enabling on-demand recovery of the others; and the Minimal Reuse Unit (MRU) for GPU memory management, combining a BlockPool-based buffer allocator with a bipartite temporal reuse strategy to mitigate fragmentation and exploit inter-batch locality. The authors report that on representative benchmarks PULSE improves training throughput by up to 6.67 times over state-of-the-art baselines. The 6.67 times figure is the authors’ reported headline result; no other quantitative results or precise mechanisms are asserted here, and they should be confirmed in the paper. This page is marked draft because the full text was unavailable.