An operating system kernel makes constant decisions — which memory page to reclaim, which processor should take a task, how far ahead to read from disk — using hand-tuned heuristics that aim at the average case and cannot specialize to your machine or your workload; machine learning could decide better, and earlier papers showed it can, but nobody had solved the SYSTEMS problems of actually running ML inside a kernel; this paper (UT Austin) builds LAKE, which gives kernel code access to a graphics processor by REMOTING the accelerator interface out to a trusted user-space helper rather than porting the closed-source vendor stack into the kernel, adds an in-kernel feature registry so developers can collect training data asynchronously across module boundaries, and — because a graphics processor only pays off above a certain batch size and must not starve the user programs sharing it — lets developers install small policies that switch between processor and accelerator while the system runs; inference time falls by up to 96 percent, but the paper's most valuable result is honest and negative — the benefit of ML in a kernel depends on the subsystem, the workload AND the hardware, and on modern storage devices one previously published ML win had largely evaporated.
The argument in one glance
-
Problem
Kernels decide with hand-tuned heuristics that cannot specialize
An operating system kernel makes constant decisions that determine how fast your machine runs. Which memory page should it reclaim? Which processor core should steal a waiting task? How far ahead should it read from disk? Today it answers with HEURISTICS — rules a kernel developer wrote by observing system behaviour and encoding their experience. Heuristics are cheap, and they give reasonable AVERAGE-case behaviour, which is exactly their weakness: they are one-size-fits-all. An I/O-heavy server and a compute-heavy server run the same kernel and get the same rules, even though a decision specialized to each workload would perform better. Hardware keeps widening these trade-off spaces — core counts grow, memory becomes tiered and non-uniform, new storage and network devices arrive — so the decisions get harder while the rules stay rigid.
-
What people did
Replace one heuristic with a machine-learned model
Researchers proposed replacing individual heuristics with machine learning, which can learn a system's real observed behaviour rather than encoding a developer's guess. Published work covered I/O latency prediction, page placement in tiered memory, processor load balancing, file-system read-ahead, and intrusion detection. Each showed a genuine decision-quality improvement for its own subsystem, so the premise was demonstrated.
-
Why it fell short
Nobody solved the systems problems of running ML inside a kernel
Each of those papers proved that ML COULD decide better; none addressed what it takes to actually run ML in a kernel. Three problems block it. First, ML needs an accelerator such as a graphics processor to absorb its computational cost — but accelerator software stacks are proprietary and user-space only, so kernel code cannot reach a graphics processor at all, and reverse-engineering an undocumented, frequently-changing stack is impractical. Second, acceleration is not always profitable: sending data to an accelerator costs time that only pays off on large enough batches, and how large depends on the subsystem, the workload AND the hardware. Third, the features a model needs are scattered across different kernel modules and abstraction layers with different locking rules, so simply collecting the input data fights the kernel's own structure. On top of that, a kernel using the graphics processor CONTENDS with the user programs that also want it — the paper measures a user program losing up to 68 percent of its throughput to an ML-assisted kernel.
-
This paper's idea
Remote the accelerator interface, and make profitability a runtime policy
This paper (UT Austin — Fingler, Tarte, Yu, Szekely, Hu, Akella, Rossbach) builds LAKE, a framework for the systems problems rather than a new model. For accelerator access it uses API REMOTING: kernel code calls a function that looks like the vendor's interface, but the call is serialized and sent to a trusted user-space helper that runs it against the real, unmodified vendor library and returns the result — so nothing proprietary is ported into the kernel and no stack is reverse-engineered. A shared memory region mapped into both sides moves bulk data with no copy across the boundary. For variable profitability, LAKE lets a developer install a small policy that runs while the system executes and chooses, per call, between the accelerator and the processor based on the current batch size and how contended the accelerator is — the same mechanism solves both profitability and contention. For feature collection, an in-kernel feature registry lets developers capture values ASYNCHRONOUSLY wherever the data already lives, using lock-free structures, and assemble them into batches for inference. Inference time falls by up to 96 percent. But the most valuable result is honest and negative: the benefit of ML in a kernel is subsystem-, workload- AND hardware-dependent, and on modern storage devices one previously published ML win had largely evaporated.
Why this problem is worth solving
Every program you run is shaped by decisions the kernel makes on your behalf,
and today those decisions come from rules that were tuned for nobody in
particular — so it is worth being precise about what a heuristic is and why
learning could beat it.
KERNEL
HEURISTICS are the hand-written rules an operating system uses to resolve
complex trade-offs quickly. Which memory page to reclaim, which processor
core should steal a waiting task, how far ahead to read from disk — each is
an expensive question to answer optimally (some are mathematically
intractable), so a kernel developer writes a rule that reaches a
good-enough answer cheaply, based on observing system behaviour and their own
experience. The weakness is that these rules aim at the AVERAGE case and are
one-size-fits-all: an I/O-heavy server and a compute-heavy server run the
same kernel version and get exactly the same rules, even though a decision
specialized to each workload would perform better. MACHINE LEARNING FOR
SYSTEMS proposes replacing such a rule with a model trained on the system’s
real, dynamically observed behaviour, so the decision can specialize to the
actual machine and workload instead of encoding one developer’s guess about
the average. Published work has shown decision-quality gains this way for I/O
latency prediction, page placement, load balancing, read-ahead and intrusion
detection
The obstacle is not the model but the machine underneath it. Running a model
costs computation, and a kernel cannot afford to spend processor time it owes
to applications, so the natural answer is an accelerator. That is exactly
where the trouble starts.
AN ACCELERATOR is
specialized hardware — most commonly a GRAPHICS PROCESSOR (GPU), originally
built for graphics and now widely used for machine learning — that performs
many arithmetic operations in parallel, far faster and more
energy-efficiently than a general processor for suitable work. The problem is
purely one of software access. Accelerator vendors ship a proprietary
user-space library (for graphics processors, CUDA) plus a kernel driver, and
the high-level interfaces programmers actually use live in that closed-source
user-space library. Kernel code cannot call it. Reverse-engineering the stack
is impractical because the internal interfaces change frequently and are
undocumented, and even after a vendor open-sourced part of its driver, the
driver still does not expose the high-level interfaces the kernel would need.
There is a second cost too: an accelerator sits behind an input/output bus,
so using it means TRANSFERRING data there and back, and that transfer only
pays for itself if enough work is batched together to amortize it
What people did before, and where it broke
- Hand-tuned kernel heuristics. Cheap, predictable, and the reason Linux performs acceptably everywhere. But they target the average case and cannot specialize, and one study found the load-balancing heuristic contained performance-critical bugs producing non-optimal choices.
- Per-subsystem ML proposals. Separate papers applied ML to I/O latency prediction, page warmth, load balancing, read-ahead and malware detection, each demonstrating better decisions. But each stopped at showing the potential benefit for one subsystem, leaving the integration problems open.
- Hand-built up-calls to user space. Earlier kernel-acceleration systems reached user-space accelerator services through custom up-calls. They work, but each introduces its own workload-specific interface at significant developer effort, and they copy kernel data structures into user space, paying for redundant transfers across the boundary.
- Accelerator virtualization by API remoting. Existing virtualization systems already remote accelerator interfaces between processes. But their communication transports are either unavailable inside a kernel or too slow for kernel-to-user transfer, so they cannot be used as they are.
- An in-kernel ML library. One recent attempt brought ML into the operating system directly, but it offers limited functionality next to mainstream ML frameworks and has no accelerator support at all.
- The gap. No one provided the shared infrastructure an ML-assisted kernel needs — accelerator access, a way to decide when acceleration actually pays, protection for user programs sharing the accelerator, and a practical way to collect features across module boundaries. Without it, every ML subsystem reinvents its own ad-hoc plumbing. That is the gap LAKE fills.
How LAKE gets a kernel to a graphics processor
Remote the vendor interface, share memory without copying, and decide at runtime whether acceleration pays
-
The kernel is locked out of the accelerator
ML in a kernel needs an accelerator to absorb its computational cost, but the vendor high-level interfaces are closed-source and user-space only. Kernel code cannot call them, reverse-engineering them is impractical, and porting a framework such as TensorFlow into the kernel is out of the question.
-
Expose the vendor interface as kernel symbols new
A kernel module (lakeLib) defines a function with the same name as each interface the kernel wants — so kernel code calls what looks like the ordinary vendor function. The call serializes an identifier plus its arguments into a command and sends it out for execution, then waits for the reply. It behaves like a remote procedure call.
-
Execute it in a trusted user-space helper new
A daemon (lakeD) receives each command, deserializes it, and runs it against the real, unmodified vendor library, then sends the return value back. Nothing proprietary is ported or reverse-engineered, and the same path serves both low-level accelerator calls and whole high-level frameworks such as TensorFlow and Keras.
-
Move bulk data without copying it new
Commands travel over Netlink sockets, chosen because they are fast and do not waste processor time spinning. Bulk data instead uses lakeShm, which reserves one large contiguous region at load time and maps it into the daemon, so kernel modules and the daemon share the same memory and no copy crosses the boundary.
-
Decide per call whether acceleration pays new
An accelerator only wins above a crossover batch size, and that size differs per subsystem. A developer installs a small eBPF policy with two callbacks — one accelerator path, one processor path — and the kernel runs it automatically, choosing based on the current batch size.
-
Yield the accelerator when user programs need it new
The same policy handles contention. It samples accelerator utilization (through a monitoring interface that is itself remoted), keeps a moving average, and when user programs are competing it routes kernel work back to the processor — then reclaims the accelerator once they finish.
The idea
Do not port the stack, and do not reverse-engineer it — remote its interface. The accelerator problem looks like it demands one of two painful options: reverse-engineer an undocumented, frequently-changing vendor stack so the kernel can drive the hardware directly, or port an enormous user-space framework into kernel space. LAKE refuses both. It keeps the vendor library exactly where it is, in user space, and moves only the CALL: kernel code invokes a function with the vendor’s name, the arguments are serialized and handed to a trusted daemon, and the real library runs untouched. Reusing a mature, proprietary component through a thin bridge rather than reimplementing or relocating it is the same compose-what-exists discipline covered elsewhere on this site — the high-level-synthesis super-optimizer that orchestrates existing compiler passes as rewrites instead of re-expressing them, and the FPGA-compilation page that keeps the vendor synthesizer as a backend. The bridge is where all the engineering goes, and the thing being bridged stays untouched.
Treat profitability as a runtime question, not a design-time assumption. It is tempting to decide once that ML decisions run on the accelerator. LAKE shows that is wrong: an accelerator wins only above a CROSSOVER batch size, and that threshold varies enormously by subsystem — one input for the page-warmth model, 8 for I/O latency prediction, 64 for read-ahead, 256 for load balancing. Below it the transfer costs more than it saves and the processor is faster. So LAKE makes the choice a small policy the kernel evaluates per call, with an accelerator path and a processor path, deciding from the batch size actually present. Refusing to commit at design time and instead choosing between two implementations with live information is the same decide-at-runtime, keep-a-fallback instinct covered elsewhere on this site in the pages that pick a fast path with a safe slower path behind it, and in the autotuning pages that learn where a threshold really lies rather than assuming one. The elegant part is that the same mechanism solves a second problem: when user programs contend for the accelerator, the policy simply routes kernel work to the processor instead.
Make feature collection a first-class facility that tolerates the kernel’s own structure. The data a model needs is scattered. Predicting I/O latency needs the number of pending requests and the completion times of recent ones — values that live in different functions, in different modules, touched by different threads under different locking rules. Gathering them synchronously just before inference fights all of that. So LAKE inverts it: developers place small capture calls exactly where each value already exists, and a lock-free registry assembles them into feature vectors over time, holding them in a circular buffer until a batch is ready. Turning a scattered, ad-hoc concern into a named, first-class object with its own interface is the same reify-the-representation move covered elsewhere on this site, where promoting something from hand-rolled plumbing to a proper type with proper operations is what makes the rest of the system tractable.
Predicting I/O latency inside a running kernel
From scattered kernel state to an accelerated decision
LAKE's own case study. A model predicts whether a disk request will be slow, so a slow one can be re-sent to another device. Numbers from the paper.
Step 1 of 5
The decision
In storage systems with several devices, predicting that a request will be slow lets the kernel re-issue it to a different device and avoid the delay. Earlier work showed a small neural network can classify each request as fast or slow, using the number of pending requests and the completion times of recent ones.
the original model is deliberately tiny — two layers of 256 and 2 neurons — to keep processor cost and latency low one inference on the processor takes about 15 microseconds
Why it works
- Vendor accelerator interfaces are closed-source and user-space only → LAKE serializes the call and runs it in a trusted user-space daemon against the unmodified library → the kernel gets full access to both low-level accelerator calls and high-level frameworks with nothing ported or reverse-engineered.
- Crossing the kernel-user boundary normally means copying buffers twice → lakeShm reserves one contiguous region mapped into both the kernel modules and the daemon → bulk data moves with no copy, and only small commands travel over the socket, whose cost grows sharply with message size.
- Acceleration pays only above a crossover batch size that differs per subsystem → a small policy evaluates the real batch size per call and picks the accelerator or the processor → each subsystem gets acceleration exactly where it helps and avoids it where it would hurt.
- A kernel and user programs contend for one accelerator with no arbitration → the same policy samples utilization and routes kernel work back to the processor while user programs are busy → the user program keeps its throughput instead of losing up to 68 percent of it.
- Model features are scattered across modules with different locking rules → developers place capture calls where each value already lives and a lock-free registry assembles vectors over time → feature collection stops fighting the kernel’s structure, and batches are ready for inference when needed.
What it costs (trade-offs and scope)
- The honest headline is conditional, and sometimes negative. The paper finds the benefit of in-kernel ML depends on the subsystem, the workload AND the hardware. Re-running a published I/O-prediction win on modern NVMe drives, the benefit had largely evaporated — the model made average read latency worse on the original workloads, on processor and accelerator alike, because faster drives with larger caches had absorbed the variance the model existed to predict.
- It adds a trusted, privileged user-space component. The daemon runs as root and kernel-private data passes through it. The authors argue this matches other tightly-integrated daemons and that address-space separation still protects against leakage, and suggest sandboxing it, but it is genuinely more attack surface than a self-contained kernel. Side-channels are explicitly left to future work.
- Remoting is not free, and neither is the boundary. Every remoted call pays serialization and a round trip. It is small relative to real ML work — about 4 percent overhead on average and at most 6 percent for one benchmark — but message cost grows sharply with size, which is precisely why bulk data must use the shared region instead.
- Batching to make the accelerator profitable fights latency goals. Waiting for enough work to amortize the transfer adds delay, which is at odds with a kernel subsystem that wants to answer immediately. For fine-grained decisions the crossover may simply never be reached.
- This is infrastructure, not a demonstration that ML kernels win. The title says “towards” for good reason. LAKE relies on earlier literature for the claim that ML decisions are better, contributes the framework for navigating the trade-offs, and the authors note that a mechanism to skip ML entirely — even on the processor — is likely still needed.
The evaluation
LAKE was built on Linux 6.0 and measured on a server with two 16-core Intel Xeon Gold 6226R processors, 376 gigabytes of memory, two NVIDIA A100 graphics processors and three Samsung 980 Pro PCIe 4.0 NVMe drives. Five ML kernel subsystems were ported to it — I/O latency prediction, page warmth classification, load balancing, file-system read-ahead and malware detection — plus one non-ML use, file-system encryption. The comparisons are against the same models running on the processor, against the unmodified kernel, and for encryption against the processor and its built-in AES-NI encryption instructions.
Running machine learning inside a kernel
| Property | Hand-tuned heuristic | ML on the processor only | ML with LAKE (this work) |
|---|---|---|---|
| Specializes to workload | |||
| Accelerator access | |||
| Inference cost | |||
| Effect on user programs | |||
| When it helps |
Check yourself
Check your understanding
1. Kernel code cannot call the proprietary user-space accelerator library, and reverse-engineering it is impractical. Predict how LAKE gets a kernel access to a graphics processor anyway.
2. LAKE measures crossover batch sizes of 1 input for page warmth, 8 for I/O latency, 64 for read-ahead and 256 for load balancing. Predict what this variation implies for the design.
3. The features an I/O latency model needs are produced in one kernel function and completed in another, on different threads under different locking rules. Predict how LAKE handles feature collection.
4. Re-running a published ML I/O-prediction result on newer NVMe drives, LAKE found the model made average read latency WORSE. Predict what general lesson this carries.
Sources
- Towards a Machine Learning-Assisted Kernel with LAKE (ACM DL, DOI 10.1145/3575693.3575697) — ASPLOS 2023.
- Author camera-ready PDF (UT Austin) — full 16-page paper, read in full for this explainer.
- LAKE source code (GitHub, utcs-scea/LAKE) — the framework and the modified Linux 6.0 kernel, released under GPLv3.