FleetIO: Managing Multi-Tenant Cloud Storage with Multi-Agent Reinforcement Learning

Jinghan Sun, Benjamin Reidys, Daixuan Li, Jichuan Chang, Marc Snir and Jian Huang

Cloud providers slice one flash SSD among many tenants, and for decades they have been stuck choosing between two bad options — give each tenant its own private flash channels for strong isolation but waste the idle capacity, or let everyone share for high utilization but suffer long tail latency from interference; this work from Illinois puts a small reinforcement-learning agent in charge of each virtual SSD, letting them harvest each other's idle bandwidth on the fly while a shared reward keeps them honest, delivering up to 1.4 times better utilization and 1.5 times lower tail latency at once.

The argument in one glance

Problem

Sharing one SSD forces a choice between isolation and utilization

Cloud platforms slice a physical flash SSD into virtual SSDs shared by many tenants. For decades this has meant a losing trade-off: to prevent tenants from interfering with each other, give each its own private flash channels — strong isolation, but idle capacity is stranded; or let tenants share channels — high utilization, but interference inflates tail latency and breaks latency targets. Their study finds hardware isolation wastes up to 1.5 times the utilization of sharing, while sharing suffers up to 2 times the tail latency.
What people did

Pick one side, or learn a static allocation up front

Providers commit to strong isolation or to sharing with throttling. More recent work learns from workloads: a pre-trained neural network decides a fixed number of channels to give each tenant, or a heuristic reallocates channels each interval in proportion to the last interval's bandwidth use, trying to hand idle resources to low-priority spot instances.
Why it fell short

Static allocations cannot track fast-changing storage dynamics

These schemes decide an allocation up front or once per interval, and cannot keep pace with workload patterns and storage state that shift with every read, write, and garbage-collection event, so they land at suboptimal efficiency. Worse, they mostly optimize for utilization alone, without folding the performance-isolation goal — meeting tail-latency targets — into their decisions, so they cannot achieve both objectives at once.
This paper's idea

Give each virtual SSD a reinforcement-learning agent that harvests idle bandwidth

FleetIO puts one reinforcement-learning agent in charge of each virtual SSD. Each agent watches its own storage state and takes actions — prioritize its I/O to hit its latency target, harvest idle bandwidth from neighbors, or offer its own idle bandwidth — with a reward that scores both bandwidth used and latency-target violations. A coefficient blends each agent's reward with its neighbors' so they cooperate, and a ghost-superblock abstraction makes the fine-grained harvesting clean. The result is near-best utilization and near-strongest isolation together.

Why this problem is worth solving

The tension at the heart of this paper is structural, not a matter of tuning.

A flash SSD is built from multiple independent flash channels, each with its own controller that can serve I/O in parallel. To share one SSD among cloud tenants, the platform slices it into virtual SSDs and maps each to some channels. There are two ways to enforce isolation, and they sit at opposite ends of a trade-off. Hardware isolation gives each virtual SSD its own private channels, so a tenant’s I/O never collides with a neighbor’s — the strongest possible isolation, and reliably low tail latency. But if that tenant leaves its channels idle, the bandwidth is simply lost, because no collocated tenant is permitted to touch channels it does not own; utilization suffers badly. Software isolation does the opposite: all tenants share the channels, and the platform throttles them with rate limiting to divide the bandwidth. Now no bandwidth is stranded and utilization is high, but the read-write interference between tenants sharing the same flash inflates tail latency and causes latency-target (SLO) violations. The authors’ measurements make the trade-off concrete: hardware isolation’s utilization can be 1.5 times worse than sharing, while sharing’s tail latency can be 2 times worse than hardware isolation. Neither can deliver strong isolation and high utilization at the same time, and that is precisely what a cloud provider wants.

What makes a genuinely better answer possible is that the situation is dynamic in a way learning is built for.

The reason static allocations fall short is that the environment never holds still. A tenant’s workload shifts between bursts and lulls, and the SSD’s internal state changes with every operation: each read and write consumes bandwidth, fills blocks, and eventually triggers garbage collection, which reclaims space but steals bandwidth of its own. Any allocation fixed up front, or even recomputed once per interval from the last interval’s statistics, is always reacting to stale information. Reinforcement learning is designed for exactly this kind of problem: an agent observes the current state of a dynamic environment, takes an action, receives a reward measuring how well the action served its goal, and gradually learns a policy that maximizes long-run reward. Applied here, an agent can watch a virtual SSD’s live bandwidth, latency, latency-target violations, queue backlog, and garbage-collection status, and decide moment to moment whether to lend out spare bandwidth, grab some from a neighbor, or reprioritize its own I/O. Crucially, the goal itself — balancing utilization against isolation — can be written directly into the reward, so the learner optimizes the actual objective rather than a proxy that ignores half of it. That fit is why FleetIO reaches for RL rather than another heuristic.

What people did before, and where it broke

Hardware isolation. Give each virtual SSD its own private flash channels for the strongest isolation. Where it broke. Any channel a tenant leaves idle is stranded, because collocated tenants cannot use channels they do not own, so storage utilization is low — up to 1.5 times worse than sharing.
Software isolation with throttling. Let all tenants share the channels and use rate limiting to divide bandwidth. Where it broke. Read-write interference between tenants on the shared flash inflates tail latency by up to 2 times and violates latency targets, so isolation is weak.
Learned static partitioning (SSDKeeper). Train a neural network to pick a fixed number of channels per tenant that minimizes average latency. Where it broke. A static partition, however well chosen, cannot track workloads and storage state that change with every I/O and garbage-collection event, and it targets utilization without folding in the isolation goal.
Interval-based adaptive allocation. Reallocate channels each interval in proportion to the previous interval’s bandwidth use, harvesting idle resources for spot instances. Where it broke. Deciding from last interval’s statistics still lags fast-changing demand, and it too optimizes utilization without directly protecting tail latency.
The gap. No approach achieved strong isolation and high utilization together, because none could adapt to the storage’s fast dynamics while explicitly optimizing both objectives at once. A per-tenant learner that harvests idle bandwidth on the fly, with isolation written into its reward, is what closes that gap.

How FleetIO works

From a fixed isolation choice to per-tenant learned harvesting

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Decide online with a learner, because the storage never holds still. Every read, write, and garbage-collection event changes the picture, so any allocation fixed in advance is stale the moment it is made; FleetIO instead has each agent read the live state and act, learning a policy that tracks the dynamics rather than guessing them up front. Committing decisions to runtime because the governing conditions are only knowable then, and static plans are doomed to be wrong, is the same conviction the dynamic-deep-learning work covered elsewhere on this site is built on, optimizing for the shapes and conditions that appear only when the computation actually runs rather than baking in assumptions that the real workload will violate.

Turn the rigid trade-off into an opportunistic harvest. Strong isolation strands idle bandwidth and sharing causes interference, but FleetIO reframes the choice: lend out only the bandwidth a tenant is not currently using, and take it back when it is needed, so a borrower gains utilization without a lender losing its guaranteed performance. Applying the beneficial-but-costly action only where and when it actually pays — here, only to genuinely idle resources — rather than committing to it globally, is exactly the discipline the page-table replication work covered elsewhere on this site follows, duplicating structures only when the access pattern shows the move will earn its keep instead of paying the cost everywhere by default.

Add a clean abstraction so the messy sharing is manageable. Fine-grained harvesting across flash channels could be a tangle of ad-hoc block bookkeeping, so FleetIO introduces the ghost superblock — a first-class unit that packages a tenant’s idle blocks for lending, with metadata tracking who owns it and who borrows it — turning harvesting into simple allocate-and-return operations. Introducing an operating-system-managed abstraction that makes fine-grained resource sharing tractable, rather than leaving each client to juggle the raw hardware, is the same move the direct-memory-translation work covered elsewhere on this site makes, having the OS manage a purpose-built structure so the fine-grained virtual-memory operation becomes a clean, cheap action.

Following two collocated tenants through FleetIO

A batch job and a database sharing one SSD

Tracing two tenants shows harvesting, prioritization, and cooperation in action.

Step 1 of 5

Two virtual SSDs share the drive, each with an agent

One virtual SSD runs a bandwidth-hungry batch job, the other a latency-sensitive database. Each has its own reinforcement-learning agent watching its bandwidth, tail latency, latency-target violations, and queue backlog.

By default each starts hardware-isolated, owning a private set of channels.

Why it works

A learner tracks what static plans cannot. Workloads and storage state change with every read, write, and garbage collection → but an RL agent decides from the current measured state → so it tracks dynamics that a static or once-per-interval allocation always lags.
Harvesting lends only what is idle. Strong isolation strands idle capacity and sharing causes interference → but fine-grained harvesting lends only the bandwidth a tenant is not using → so a borrower gains utilization while the lender keeps its guaranteed performance.
The reward optimizes the real balance. Utilization and isolation pull in opposite directions → but the reward scores both bandwidth used and latency-target violations under a tunable weight → so the agent optimizes the actual objective instead of one metric at the other’s expense.
Blended rewards make agents cooperate. Independent agents would each hoard their resources → but mixing each reward with its neighbors’ gives every agent a stake in overall efficiency → so they lend and harvest without any central controller.
Per-type tuning fits diverse tenants. Different workloads care about different metrics → but clustering them by I/O pattern and fine-tuning the reward weight per type → so a latency-sensitive tenant and a bandwidth-hungry one each get a policy suited to them.

What it costs (trade-offs and scope)

It is best-effort, not ideal. FleetIO does not reach perfect isolation or perfect utilization; it lands near both — about 93 percent of the best achievable utilization and within 1.2 times of the strongest isolation — a genuine improvement on the frontier, not a free lunch.
It needs offline pre-training and periodic tuning. The model takes about two hours to pre-train, and new or uncategorized workload types fall back to a generic reward until an hourly offline tuning learns a policy suited to them.
It requires a programmable SSD. FleetIO is built on an open-channel SSD where the host directly manages the flash channels; commodity black-box SSDs without software-defined flash cannot support channel-level harvesting or the ghost-superblock abstraction.
Coordination is a simple blend, not true negotiation. Agents cooperate through a single reward-mixing coefficient rather than explicitly negotiating, which works well here but is a heuristic, with no guarantee of global optimality.
Per-tenant agents add modest, if off-path, overhead. Each virtual SSD runs its own agent on a CPU core, with periodic fine-tuning and per-window inference and a couple of megabytes of model state; these sit off the I/O critical path but still consume compute and memory.
The results are tied to one board and workload set. Evaluation used a single 16-channel SSD and five cloud workloads at a superblock harvesting granularity, and the provider policies that gate harvesting would shift the picture elsewhere.

The evaluation

Three ways to share a flash SSD among tenants

Property	Hardware isolation	Software isolation
How flash channels are assigned		Shared and throttled
Performance isolation	Strongest
Storage utilization		High
Adapts to changing workloads	No	No

Three ways to share a flash SSD among tenants, evaluated on a real 16-channel open-channel SSD across bandwidth-intensive (TeraSort, ML-prep, PageRank) and latency-sensitive (VDI-Web, YCSB) cloud workloads. By giving each virtual SSD an RL agent that harvests idle bandwidth while protecting latency targets, FleetIO improves utilization over hardware isolation by up to 1.4 times (1.3 times average) while cutting tail latency versus software isolation by up to 1.58 times (1.47 times average) — reaching about 93 percent of the best utilization and staying within 1.2 times of the strongest isolation, where every prior scheme wins only one objective. The gains grow as more virtual SSDs are collocated, and the overheads (offline pre-training, per-window inference, ghost-superblock management) stay off the I/O critical path.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

FleetIO is a well-built systems result: it takes a decades-old trade-off in cloud storage — strong isolation or high utilization, pick one — and dissolves much of it by giving each virtual SSD a reinforcement-learning agent that harvests genuinely idle bandwidth while writing the isolation goal directly into the reward, coordinating agents with a lightweight reward blend and a clean ghost-superblock abstraction. It is validated on real programmable-SSD hardware and reaches near-best utilization and near-strongest isolation at once, with gains that grow as more tenants share the drive. Read it with its scope. This is a best-effort improvement on the frontier, not perfection at both ends; it needs offline pre-training and periodic tuning, with new workloads temporarily on a generic policy; it depends on a programmable open-channel SSD that commodity black-box drives do not provide; its multi-agent cooperation is a heuristic reward blend rather than true negotiation; and the numbers come from one 16-channel board and five workloads. The durable lesson is that many rigid systems trade-offs are rigid only because the controller was static — replace it with a learner that watches the live state and optimizes the real, two-sided objective, and the frontier itself can move.

Background

Why the cloud keeps fighting the isolation-utilization tussle

It helps to see FleetIO as one front in a war the cloud fights everywhere: multi-tenancy is what makes cloud economics work, but every resource shared for efficiency becomes a source of interference that threatens the isolation tenants are promised. Storage bandwidth is one instance; the same tension plays out in CPU caches, memory bandwidth, network links, and accelerator time, and the recurring prize is to reclaim the slack that strict isolation strands without letting tenants trample each other. The work on cloud serving with tight latency targets covered elsewhere on this site pursues the same balance from the compute side, squeezing utilization out of expensive shared hardware while still meeting each request’s service-level objective, exactly the two-sided goal FleetIO writes into its reward. The broad lesson is that in a multi-tenant system the scarce resource is not raw capacity but the ability to safely share it — and that the systems which win are the ones that treat isolation and utilization as a single objective to be optimized together, dynamically, rather than a dial to be set once and lived with.

If you remember only three things

1. Cloud platforms share a flash SSD by slicing it into virtual SSDs, and face a fundamental tussle between performance isolation and storage utilization. Hardware isolation gives each tenant private flash channels (strongest isolation, low tail latency) but strands idle capacity — up to 1.5 times worse utilization than sharing. Software isolation shares channels with throttling (high utilization) but read-write interference inflates tail latency up to 2 times and breaks latency targets. Existing learned or heuristic schemes decide a static or once-per-interval allocation that cannot track workloads and storage state changing with every read, write, and garbage collection, and mostly optimize utilization without folding in the isolation goal.
2. FleetIO manages virtualized SSDs with multi-agent reinforcement learning: one RL agent per virtual SSD, each observing its live storage state (bandwidth, latency, latency-target violations, queue delay, capacity, garbage-collection status) and taking three actions — set its I/O priority to meet its latency target, harvest idle bandwidth from neighbors, and make its own idle bandwidth harvestable. The reward scores bandwidth used minus latency-target violations under a tunable weight, fine-tuned per workload type via k-means clustering on I/O patterns, and a coefficient blends each agent's reward with its neighbors' so independent agents cooperate without a central controller. A ghost-superblock abstraction packages idle blocks for clean lending and return, and an admission-control layer batches and orders harvest actions under provider policy, all off the I/O critical path.
3. On a real 16-channel open-channel SSD across bandwidth-intensive and latency-sensitive workloads, FleetIO improves utilization over hardware isolation by up to 1.4 times (1.3 average) while cutting tail latency versus software isolation by up to 1.58 times (1.47 average), reaching about 93 percent of the best utilization and staying within 1.2 times of the strongest isolation — a Pareto point no prior scheme reaches — with gains growing as more virtual SSDs are collocated. Its limits: it is best-effort not ideal, needs offline pre-training (about two hours) and hourly tuning with new workloads on a generic policy, requires a programmable SSD, coordinates via a heuristic reward blend, adds per-tenant agent overhead (off the critical path), and is validated on one board and five workloads.

Check yourself

Check your understanding

1. What is the fundamental trade-off FleetIO addresses in shared SSD storage?
2. Why is reinforcement learning a natural fit for managing shared SSDs, and how does FleetIO structure it?
3. How does FleetIO get independent per-tenant agents to cooperate, and how does it keep harvesting clean?
4. Which statement best reflects FleetIO's results and limitations?

Sources

The paper is “FleetIO: Managing Multi-Tenant Cloud Storage with Multi-Agent Reinforcement Learning” by Jinghan Sun, Benjamin Reidys, Daixuan Li, Jichuan Chang, Marc Snir, and Jian Huang, published at ASPLOS 2025, DOI 10.1145/3669940.3707229. The authors are at the University of Illinois Urbana-Champaign and Google.

This explainer is written from the full text of the paper (the author-hosted open-access version from the authors’ Illinois PlatformX lab, matching the ASPLOS proceedings). Everything above comes from the paper’s own sections: the introduction and background on SSD virtualization, software-defined flash, and the software- versus hardware-isolation trade-off, with the measured 1.5-times utilization and 2-times tail-latency gaps; the motivation for reinforcement learning over static and heuristic schemes (SSDKeeper and interval-based adaptive allocation); the RL formulation with its state space (bandwidth, IOPS, latency, SLO violations, read-write ratio, queue delay, capacity, garbage-collection status, priority), its three actions (set priority, harvest, make harvestable), and its reward function balancing bandwidth against SLO violations with a tunable weight; the multi-agent reward blending via a coefficient, the per-workload-type fine-tuning by k-means clustering on I/O features (with 98.4 percent clustering accuracy), and the admission-control mechanism; the ghost-superblock abstraction and its management with lock-free lists; the implementation on a real open-channel SSD (3.5K lines of C/C++ plus 2K of Python, RLlib and PyTorch, a 2.2 MB model); and the evaluation against hardware isolation, SSDKeeper, adaptive allocation, and software isolation — the up-to-1.4-times utilization and up-to-1.58-times tail-latency improvements, the reach to 93 percent of best utilization and within 1.2 times of strongest isolation, the scalability to 4-8 virtual SSDs, the reward-function ablations, the mixed-isolation and robustness studies, and the overhead measurements (about two-hour pre-training, 1.1-millisecond inference, under-one-microsecond ghost-superblock creation).