HEPIC: Private Inference over Homomorphic Encryption with Client Intervention

Running a neural network on encrypted data lets a cloud serve predictions without ever seeing the client's input, but the server has to keep "refreshing" the encryption (bootstrapping) and converting between encoding schemes, and doing all of that on the encrypted data alone is astronomically slow; this paper (Seoul National University and UNIST — Kevin Nam, Hyungon Moon, Yunheung Paek, and colleagues) presents HEPIC, which lets the client — who holds the decryption key — quickly do that ciphertext housekeeping in short round-trips, overlapping it with the server's work to run private inference far faster with no loss in accuracy.

The argument in one glance

Problem

Running a model on encrypted data protects privacy but is punishingly slow

In Machine Learning as a Service, a client sends its data to a cloud server that runs a neural network and returns a prediction. The privacy problem is obvious: the client does not want to reveal its input (a medical image, a private document), and the server does not want to reveal its trained model. Homomorphic Encryption (HE) offers a beautiful answer — the client encrypts its input, the server computes on the ciphertext directly without ever decrypting it, and the client decrypts the encrypted result. Both the client's input and the server's parameters stay protected, and this is called Private Inference (PI). The catch is cost. Computing on encrypted data is orders of magnitude slower than on plaintext, and a large part of that cost is not the neural-network math itself but the "ciphertext management" that HE forces on you. Every homomorphic operation adds noise to a ciphertext, and once the noise budget runs low the ciphertext must be refreshed by an operation called bootstrapping — which is spectacularly expensive, because it amounts to homomorphically evaluating the decryption function itself. On top of that, good HE-based inference switches between different encryption schemes and parameter settings to balance accuracy and speed, and those scheme and parameter conversions are costly too. So the promise of private inference runs headlong into a performance wall built largely out of ciphertext management.
What people did

Do everything on the server, and switch schemes to tune the trade-off

The dominant HE-based approach is "fire-and-forget": the client encrypts its input once, sends it off, and the server performs the entire computation — including all the bootstrapping and all the scheme and parameter conversions — purely on the encrypted data, then returns the encrypted answer. The client is not involved again until it decrypts the result. Recent systems make this smarter with a layer-wise scheme, choosing per layer which encryption scheme and parameters to use, and switching between them, to improve the accuracy-versus-performance trade-off. The appeal of fire-and-forget is that it is simple and non-interactive: one message out, one message back.
Why it fell short

Fire-and-forget makes the server do the most expensive work the hardest way

The problem with fire-and-forget is that it forces the server to perform ciphertext management the most expensive way possible — entirely under encryption. Bootstrapping on the server is a giant homomorphic circuit; scheme and parameter conversions on encrypted data are heavy operations; and because only the server acts, all of this sits on the critical path with nothing to overlap it. Yet there is a striking asymmetry the fire-and-forget model ignores: the client holds the secret key. Refreshing a ciphertext, which costs the server an enormous homomorphic bootstrapping circuit, is something the client could do almost trivially — decrypt the ciphertext and re-encrypt it fresh, with a full noise budget and in whatever scheme or parameters are wanted next. The other established family of private inference, MPC-based methods, is interactive with the client, but it makes the client participate throughout the neural-network computation, with its own heavy communication. What was missing was a design that keeps the client out of the model computation but brings it in for exactly the ciphertext management that it can do cheaply — and that makes those extra round-trips fast enough to be worth it.
This paper's idea

Let the client handle ciphertext management, and hide the round-trips

HEPIC is an HE-based private-inference system that explores a new design point: it leverages client interventions for ciphertext management. Like MPC-based methods it is interactive with the client, but unlike them the client only intervenes for the ciphertext management that HE operations require — bootstrapping and scheme or parameter conversions — never for the neural-network computation itself. The key observation is that ciphertext management has identical semantics whether done on the client or the server: refreshing or converting a ciphertext produces the same result either way, but the client, holding the key, can do it by cheap re-encryption instead of an expensive homomorphic circuit. Because the operation is interchangeable between the two sides, HEPIC lets developers decide where and how often to perform it, opening up fine-grained trade-offs among computation (on the server), communication (round-trips to the client), and ciphertext configuration. The obvious risk is that shuttling ciphertexts to the client and back could stall everything, so HEPIC makes it practical by overlapping client re-encryption, server computation, and communication through dependency-aware pipelining and streaming-based transfers. It adds two schedulers: a cache-aware task allocator (CATA) to exploit ciphertext-level parallelism, and a cost-aware client intervention scheduler (CACIS) to decide when to invoke the client so the fast side does not stall waiting on the slow side, given that client and server have very different performance. The result, the paper reports, is up to a 2.20 to 41.93 times speedup over state-of-the-art fire-and-forget HE-based private inference, with zero loss in inference accuracy.

Why this problem is worth solving

Private inference could let people use powerful cloud models on sensitive data without surrendering that data, but only if it is fast enough to be practical.

A great deal of the data people would most benefit from running through machine-learning models is exactly the data they cannot safely hand over: medical scans, financial records, private messages, proprietary documents. Homomorphic-encryption private inference is one of the few techniques that squares this circle, letting a cloud run its model on a client’s input while neither side reveals its secret — the client’s data stays encrypted throughout, and the server’s model never leaves its hands. That is a genuinely valuable capability. The obstacle has always been speed: computing under encryption can be thousands of times slower than in the clear, which relegates private inference to a research curiosity rather than a deployable service. Closing that gap is what turns the privacy guarantee from a theoretical possibility into something a business could actually offer. Because the dominant cost is concentrated in a few expensive operations — above all ciphertext refreshing — attacking those operations specifically is the highest-leverage way to make private inference practical, which is exactly what makes a better way to handle them worth pursuing.

The deep insight is that the most expensive operation under encryption is nearly free for whoever holds the key.

Bootstrapping is the defining cost of homomorphic computation: as operations accumulate noise, a ciphertext eventually must be refreshed, and doing so without the secret key means homomorphically evaluating the entire decryption procedure — an enormous, slow circuit. But this expense exists only because the server does not have the key. The client does. For the client, refreshing a ciphertext is trivial: decrypt it, then encrypt it again with a clean noise budget and any scheme or parameters desired. The same is true of scheme and parameter conversions. So the single most costly thing the server does is something the client can do almost instantly. Fire-and-forget throws this asymmetry away by insisting the server do everything alone. HEPIC’s premise is that the ciphertext management, and only that, should be handed to the side that can do it cheaply — turning a giant homomorphic circuit into a quick decrypt-and-re-encrypt plus a network round-trip. The catch, and the reason this needs real engineering rather than just an observation, is that the round-trips introduce communication and dependencies that could stall the server; making the interactive design actually faster than fire-and-forget is what the rest of the system is for.

What people did before, and where it broke

Fire-and-forget HE, all on the server. The client encrypts once; the server does the entire computation, including all ciphertext management, under encryption. Where it broke. Bootstrapping and conversions done purely homomorphically are enormously expensive and sit on the critical path with nothing to overlap them.
Layer-wise schemes and parameter switching. Choose the best encryption scheme and parameters per layer to balance accuracy and speed. Where it broke. It improves the trade-off but still runs fire-and-forget, so the server alone pays the costly scheme and parameter conversions on encrypted data.
MPC-based interactive private inference. Have the client and server interact throughout using secret sharing. Where it broke. The client participates in the whole neural-network computation, incurring heavy, pervasive communication rather than a targeted one.
Bootstrap on the server whenever noise runs out. Refresh ciphertexts with the homomorphic bootstrapping circuit. Where it broke. It ignores that the client, holding the key, could refresh the same ciphertext almost for free by decrypting and re-encrypting.
The gap. No HE-based system brought the client in for just the ciphertext management it can do cheaply, while keeping it out of the model computation and hiding the resulting round-trips behind server work.

How HEPIC works

Offloading ciphertext management to the client, and overlapping the round-trips

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Give the expensive encrypted work to the party who can do it cheaply. HEPIC’s founding move is to notice that refreshing a ciphertext, ruinously expensive for the keyless server, is trivial for the client who holds the key — so the client should do it. Optimizing encrypted computation by offloading the parts that are cheap for the client to the client, rather than forcing the server to do everything under encryption, is the same approach the work on client-optimized algorithms and acceleration for encrypted compute offload covered elsewhere on this site takes, using the client’s special position to sidestep the server’s worst costs. HEPIC applies this specifically to ciphertext management — bootstrapping and conversions — which is where the private-inference cost concentrates.

Treat private inference as a whole-system trade-off among compute, communication, and encryption settings. HEPIC does not just move one operation; it exposes where and how often to do ciphertext management as a tunable knob, balancing server computation against client round-trips and ciphertext configuration. Optimizing private inference by reasoning about the end-to-end system — how encryption choices, computation, and data movement interact — rather than any single kernel, is the same lens the work on characterizing and optimizing end-to-end systems for private inference covered elsewhere on this site applies, treating the full pipeline as the thing to tune. HEPIC’s developer-chosen intervention points are exactly such an end-to-end trade-off.

Hide the cost of interaction by overlapping it with computation. The danger of making the client intervene is that the round-trips stall the server, so HEPIC pipelines client re-encryption, server computation, and communication so they proceed at once. Overlapping communication with the computation that surrounds it, so the data movement is masked rather than paid for serially, is the same principle the work on overlapping communication with dependent computation covered elsewhere on this site is built on, turning a blocking exchange into a pipeline. HEPIC’s dependency-aware pipelining and streaming transfers are what let the interactive design beat fire-and-forget instead of being dragged down by its own round-trips.

Following a bootstrapping through HEPIC

Refreshing a ciphertext by client intervention instead of homomorphic bootstrapping

An illustrative walkthrough, based on the paper's described mechanism (this explainer is written from the paper's abstract; exact details are in the full paper).

Step 1 of 5

The server computes until noise runs low

The server evaluates neural-network layers on the encrypted data. Each homomorphic operation adds noise, and after enough of them a ciphertext's noise budget is nearly exhausted, so it must be refreshed before computation can continue.

In fire-and-forget, this would trigger an expensive homomorphic bootstrapping.

Why it works

The key-holder refreshes ciphertexts almost for free. Server-side bootstrapping is a giant homomorphic circuit → but the client, holding the key, just decrypts and re-encrypts → so the single most expensive HE operation is replaced by a near-trivial one.
Identical semantics make the operation freely relocatable. Ciphertext management gives the same result on either side → but only the client can do it cheaply → so developers can place it wherever it costs least and tune how often it happens.
Pipelining hides the round-trips behind server work. Sending ciphertexts to the client could stall the server → but overlapping client re-encryption, server computation, and streaming transfers → so the interaction is masked and the interactive design still runs fast.
Scheduling copes with the client-server performance gap. The client and server run at very different speeds, risking stalls → but a cost-aware scheduler choosing when to intervene, plus a cache-aware allocator exploiting ciphertext parallelism → so the faster side keeps busy rather than waiting.
Re-encryption is exact, so accuracy is untouched. Offloading work to the client could risk approximation → but decrypt-and-re-encrypt reproduces the ciphertext exactly → so HEPIC keeps zero loss in inference accuracy while going faster.

What it costs (trade-offs and scope)

It requires an interactive client, not fire-and-forget. HEPIC gives up the simple one-message-out, one-message-back model; the client must be online and responsive during inference to perform its interventions, which suits some deployments and not others.
It adds communication and round-trips. Trading homomorphic bootstrapping for client re-encryption introduces network traffic and latency; the design only wins because the round-trips are overlapped with computation, so it depends on there being enough parallel server work to hide them and on adequate bandwidth.
The benefit hinges on scheduling under a performance disparity. Client and server can differ greatly in speed, so if the schedulers (CATA and CACIS) misjudge when to intervene, one side stalls; the gains depend on getting that cost-aware timing right for the given deployment.
Security rests on what the client is allowed to see and do. Bringing the client into the loop to decrypt and re-encrypt intermediate ciphertexts is a change to the execution model; the precise privacy guarantees — what the client learns during intervention and how the server’s model stays protected — are established by the paper’s security analysis, which this summary does not reproduce.
Reported speedups are workload- and baseline-dependent. The 2.20 to 41.93 times range is wide, reflecting that the advantage varies a great deal across networks and settings; the exact gain in any deployment depends on how much ciphertext management dominates and how well the round-trips can be hidden.

The evaluation

Approaches to private neural-network inference

Property	Fire-and-forget HE	MPC-based interactive	HEPIC
Who does ciphertext management
Client involvement
Dominant cost
Tunability of the trade-off
Reported outcome

A comparison of private-inference approaches. By having the client perform ciphertext management (bootstrapping and conversions) via cheap re-encryption instead of the server doing it under encryption, and overlapping the round-trips with server computation through pipelining and scheduling, HEPIC reports up to a 2.20 to 41.93 times speedup over state-of-the-art fire-and-forget HE-based private inference with zero accuracy loss. This explainer is written from the paper's abstract; exact figures and mechanisms should be read from the paper.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is written from the paper’s abstract (the full text was not accessible at the time of writing — the ASPLOS 2026 paper is open access under CC BY, but the copy was reachable only through a publisher page that could not be retrieved, and no author-hosted copy or preprint was found). The described system is the paper’s: HEPIC is an HE-based private-inference system that leverages client interventions for ciphertext management. It shares interactivity with MPC-based private inference but differs in that the client intervenes only for the ciphertext management that HE operations require — bootstrapping and scheme or parameter conversions — not for the neural-network computation. The key premise is that ciphertext management has identical semantics on client and server, so the client (holding the key) can perform it by re-encryption, and developers can decide where and how often to execute it, enabling fine-grained trade-offs among computation, communication, and ciphertext configuration. HEPIC makes this practical by overlapping client re-encryption, server computation, and communication via dependency-aware pipelining and streaming-based transfers, and adds a cache-aware task allocator (CATA) to exploit ciphertext-level parallelism and a cost-aware client intervention scheduler (CACIS) to mitigate stalls under client-server performance disparity. The paper reports up to a 2.20 to 41.93 times speedup over state-of-the-art fire-and-forget HE-based private inference while maintaining zero loss in inference accuracy. The background on bootstrapping, noise budgets, and scheme conversion given above is standard homomorphic-encryption context provided to make the approach intuitive; the paper’s own security analysis and mechanism details are the authority. The step-by-step walkthrough is illustrative, not a claim about the paper’s exact implementation.

Background

Why making encrypted computation practical keeps coming down to its single worst operation

It helps to see HEPIC within a broad pattern: homomorphic encryption is one of the most powerful privacy tools known — it lets a party compute on data it cannot read — but its practicality has always been gated by a small number of dominant costs, above all bootstrapping, the operation that refreshes a ciphertext so computation can continue. Almost every advance in making encrypted computation usable is, at heart, an attack on that cost, whether by building specialized hardware to run the homomorphic circuits faster, by restructuring algorithms to bootstrap less often, or, as here, by moving the operation somewhere it is cheaper to perform. What makes HEPIC’s angle distinctive is that it does not try to make bootstrapping faster under encryption at all; it observes that the party holding the key never needed to bootstrap the hard way, and rearranges the protocol to use that fact, then does the systems work to hide the communication this introduces. This effort to make homomorphic encryption practical by driving down the cost of its heaviest operations is the same terrain the work on algorithm-hardware co-design for efficient fully homomorphic encryption (Falcon) covered elsewhere on this site occupies, attacking the bootstrapping bottleneck so encrypted computation can be fast enough to use. HEPIC’s contribution is to show that a well-scheduled interactive design, in which the client handles ciphertext management and the round-trips are overlapped with server work, can substantially outrun fire-and-forget private inference without sacrificing any accuracy.

If you remember only three things

1. Homomorphic-encryption private inference lets a cloud run a neural network on a client's encrypted input, protecting both the client's data and the server's model, but it is punishingly slow — and much of the cost is ciphertext management, not the network math: every homomorphic operation adds noise, so ciphertexts must periodically be refreshed by bootstrapping (a spectacularly expensive homomorphic evaluation of decryption), and good systems also convert between encryption schemes and parameters. Existing HE systems are fire-and-forget: the client encrypts once and the server does all the computation and all the ciphertext management under encryption, so those costly operations sit on the critical path with nothing to overlap them.
2. HEPIC explores a different design point: it lets the client intervene for ciphertext management only — bootstrapping and conversions — while the neural-network computation stays on the server. The key insight is that ciphertext management has identical semantics on either side, but the client holds the secret key, so it can refresh or convert a ciphertext by cheap decrypt-and-re-encrypt instead of the server's giant homomorphic circuit. Because the operation is interchangeable, developers can choose where and how often to do it, trading off computation, communication, and ciphertext configuration. HEPIC makes the round-trips worthwhile by overlapping client re-encryption, server computation, and communication via dependency-aware pipelining and streaming transfers, plus a cache-aware task allocator (CATA) and a cost-aware client intervention scheduler (CACIS) to exploit parallelism and avoid stalls under the client-server speed gap.
3. The paper reports HEPIC achieves up to a 2.20 to 41.93 times speedup over state-of-the-art fire-and-forget HE-based private inference while maintaining zero loss in inference accuracy (because client re-encryption is exact). The trade-offs: it requires an interactive, online client rather than fire-and-forget; it adds communication that only pays off when overlapped with enough parallel server work; its gains depend on the cost-aware scheduler getting intervention timing right under the client-server performance disparity; and the exact privacy guarantees rest on the paper's security analysis. This explainer is based on the abstract; full details are in the paper.

Check yourself

Check your understanding

1. In homomorphic-encryption private inference, what is "ciphertext management," and why is it so expensive in fire-and-forget systems?
2. What is the key observation behind HEPIC's use of "client intervention"?
3. Bringing the client into the loop adds network round-trips. How does HEPIC keep those from making it slower than fire-and-forget?
4. What does the paper report for HEPIC, and what caveats should be kept in mind?

Sources

The paper is “HEPIC: Private Inference over Homomorphic Encryption with Client Intervention” by Kevin Nam, Youyeon Joo, Seungjin Ha, Hyungon Moon, and Yunheung Paek (Seoul National University and Ulsan National Institute of Science and Technology, UNIST), published at ASPLOS 2026, DOI 10.1145/3779212.3790170. The paper is open access under CC BY.

This explainer is written from the paper’s abstract; the full text was not accessible at the time of writing (the open-access copy was reachable only through a publisher page that could not be retrieved, and no author-hosted copy or preprint was found). HEPIC is an HE-based private-inference system that leverages client interventions for ciphertext management. It is interactive with the client like MPC-based methods, but the client intervenes only for the ciphertext management that HE operations require — bootstrapping and scheme or parameter conversions — not for the neural-network computation. Because ciphertext management has identical semantics on client and server, and the client holds the key, the client can perform it cheaply by re-encryption, and developers can decide where and how often to execute it, enabling fine-grained trade-offs among computation, communication, and ciphertext configuration. HEPIC overlaps client re-encryption, server computation, and communication via dependency-aware pipelining and streaming-based transfers, and adds a cache-aware task allocator (CATA) and a cost-aware client intervention scheduler (CACIS) to exploit ciphertext-level parallelism and mitigate stalls under client-server performance disparity. The paper reports up to a 2.20 to 41.93 times speedup over state-of-the-art fire-and-forget HE-based private inference while maintaining zero loss in inference accuracy. The background on bootstrapping and scheme conversion is standard homomorphic-encryption context; exact mechanisms, the security analysis, and full quantitative results should be confirmed in the paper.