FreePart: Hardening Data Processing Software via Framework-based Partitioning and Isolation

Machine-learning and image-processing programs lean on big frameworks like TensorFlow and OpenCV, so a memory-corruption bug in one framework function lets an attacker who feeds it a crafted input take over the entire application — because the framework and the app share one address space; this paper (Maryland, Illinois, Columbia, Ohio State, UT Dallas — Ali Ahad, Yonghwi Kwon, and colleagues) builds FreePart, which notices that a data-processing app's execution flows through framework API calls in phases (load, process, visualize, store), runs each phase's APIs in its own privilege-restricted process, and uses a lazy data-copy trick to keep the isolation cheap, stopping all 18 real-world CVE attacks at 3.68 percent overhead.

The argument in one glance

Problem

A bug in a shared framework can hand an attacker the whole application

Machine-learning and image-processing programs are built on a few enormous frameworks — TensorFlow, OpenCV, PyTorch, Caffe — that they call for nearly everything: loading an image, running a model, saving a result. Those frameworks are huge, widely used, and, like all large software, contain vulnerabilities. The danger is that a framework runs inside the very same process as the application that calls it, sharing one address space with all the application's code and data. So a memory-corruption bug in, say, OpenCV's image-reading function is not a contained problem: an attacker who feeds that function a maliciously crafted image can corrupt the application's critical data, execute arbitrary code, or crash the program, regardless of what algorithm or model the application itself uses. The framework you depend on becomes the way in.
What people did

Isolate parts of the program with existing software partitioning

The classic defense is software isolation: split the program into separate compartments so that a compromise of one cannot reach the others, confining an exploited vulnerability to a small blast radius. Prior partitioning tools do this by analyzing the program to decide where to draw boundaries and which data each compartment may touch, then enforcing the split with the operating system.
Why it fell short

Existing isolation is either too hard to analyze or too slow

The existing approaches stumble in practice for one of two reasons. Some rely on statically analyzing the program's dependencies to figure out how to partition it, but data-processing code with its tangled data flows through a giant framework is extremely hard to analyze precisely, so the partitioning is either unsound or incomplete. Others enforce isolation with heavy machinery — frequent inter-process communication to move data across boundaries — which imposes so much runtime overhead that it is impractical for performance-sensitive ML and image pipelines. Neither gives effective isolation at a cost people will actually pay.
This paper's idea

Partition by framework API phase, isolate each phase, and copy data lazily

FreePart observes that a data-processing application's execution naturally flows through framework API calls in a predictable temporal order — first loading APIs, then processing APIs, then visualizing and storing APIs — and that this sequence tells you what data should be in use at each moment. So instead of statically carving up the code, FreePart partitions the execution by framework API type: it categorizes the framework's hundreds of APIs into four phases, runs each phase's APIs in its own isolated process, and restricts each process to only the system calls that phase actually needs. A vulnerability exploited in one phase is trapped there, unable to corrupt the host's critical data. To keep this cheap, a lazy data-copy technique passes results directly between the phase processes instead of routing everything through the host. FreePart stops all 18 real-world CVE attacks across 23 applications at only 3.68 percent overhead.

Why this problem is worth solving

The threat is severe precisely because it does not depend on anything the application did wrong.

A software vulnerability in a framework — a memory-corruption bug such as a buffer overflow inside an OpenCV or TensorFlow function — is one of the most dangerous kinds of flaw, because the framework executes in the same process and address space as the application that calls it. There is no boundary between them: the framework’s code can read and write all of the application’s memory, both its code and its data. So when an attacker supplies a crafted input that triggers the bug during a framework call, the exploit runs with the application’s full authority and can do almost anything — silently corrupt the application’s critical data so it computes wrong results, hijack control to run the attacker’s code, or crash the process for denial of service. This is categorically worse than the machine-learning-specific attacks people often worry about, like adversarial examples that fool a model’s decision, because a software vulnerability sidesteps the algorithm entirely and compromises the whole program. And it is not the application developer’s fault: OpenCV alone has tens of millions of downloads and is embedded in countless critical systems, so a single framework bug is a shared liability across all of them, exploitable regardless of the model or logic the host application runs. That breadth and severity is why hardening the framework boundary is worth a dedicated system.

The reason it has stayed unsolved is that the obvious defense collides with the nature of this software.

Software isolation, or compartmentalization, is a well-established idea: divide a program into parts that run in separate protection domains so a compromise of one part cannot reach another, shrinking what an exploit can touch. The difficulty is deciding where to draw the boundaries and enforcing them cheaply, and data-processing software is uniquely hostile to both. Deciding the boundaries usually means analyzing the program’s dependencies — which code touches which data — but ML and image pipelines run their data through a massive, opaque framework with deeply tangled internal data flows, so static analysis of those dependencies is imprecise or intractable, and a partition based on it is unreliable. Enforcing the boundaries usually means moving data between isolated processes whenever it crosses a boundary, but these pipelines are throughput-sensitive and pass large data objects (images, tensors) constantly, so naive inter-process communication for every crossing imposes crippling overhead. The two failure modes are linked: to keep overhead low you want few boundaries, but to isolate effectively you want boundaries in the right places, and you cannot find those places precisely. FreePart’s contribution is a way to draw the boundaries without hard dependency analysis and to enforce them without paying for constant data copying, which is exactly the pair of obstacles that made prior isolation impractical here.

What people did before, and where it broke

Run the framework in the same process as the application. The default: call OpenCV or TensorFlow functions directly, sharing one address space. Where it broke. A memory-corruption bug in any framework API gives an attacker full reach over the application’s code and data, so one framework vulnerability compromises the entire program.
Static-analysis-based program partitioning. Analyze the program’s dependencies to decide how to split it into compartments and which data each may access. Where it broke. The tangled data flows of ML and image pipelines through a huge framework are extremely hard to analyze precisely, so the resulting partition is unsound or incomplete and misses real paths.
Privileged-data isolation with per-crossing IPC. Isolate the sensitive data or components into a separate domain and communicate across the boundary as needed. Where it broke. Routing every data crossing through inter-process communication imposes excessive runtime overhead on throughput-sensitive pipelines that constantly pass large images and tensors, making it impractical.
Shared-memory optimizations over ad-hoc boundaries. Reduce the IPC cost by sharing some variables through shared memory. Where it broke. These reduce overhead only for specific hand-chosen cases and still rest on the hard problem of deciding correct boundaries, so they do not generalize to arbitrary data-processing applications.
The gap. No approach both chose isolation boundaries without relying on brittle static dependency analysis — using the runtime structure of the computation instead — and enforced them cheaply enough for performance-sensitive data pipelines, which is what practical framework hardening requires.

How FreePart works

Turning a framework's API-call sequence into isolated, least-privilege execution phases

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Draw the boundaries from what the program does at runtime, not from static code. The insight that unlocks everything is that a data-processing application’s execution is legible through its framework API calls. These programs follow a stereotyped arc — load the data, pre-process it, run the algorithm, then visualize or store the output — and that arc is expressed directly as a temporal sequence of framework calls, which in turn tells you what critical data is being touched at each moment. So FreePart does not need to solve the intractable static problem of tracing tangled data dependencies through a giant framework; it lets the runtime sequence of API invocations define the partition. Each framework API is sorted into one of four phase types, and the execution is split along those phase boundaries. This is isolation from the perspective of data accesses over time rather than from the perspective of static program structure, which is why it sidesteps the analysis that defeated prior tools. Deciding how to compartmentalize a system by the shape of its actual execution rather than a brittle up-front analysis is the same pragmatic stance the flexible OS-isolation work covered elsewhere on this site takes, choosing isolation boundaries to fit how the software really runs instead of forcing a single static decomposition.

Follow the data, because the data is what needs protecting. FreePart’s target is not abstract code regions but the application’s critical data — the values a corrupted framework call could silently falsify to derail the whole computation. By tracking which type of API is executing, FreePart infers what data permissions are appropriate at that instant and confines each phase to a process that can only touch what that phase legitimately needs. This is a data-centric view of security: the question is never merely “what code is running” but “what data can this running code reach, and should it be able to.” An exploit in the loading agent cannot reach into the coordinates or scores the application computes later, because those live behind a process boundary the loading phase has no business crossing. Organizing a defense around the flow and reachability of the sensitive data itself, rather than around control flow alone, is the same data-flow-centered reasoning the taint-source inference work for vulnerability detection covered elsewhere on this site is built on, following where sensitive or attacker-controlled data can travel to decide what must be watched or contained.

Isolate with least privilege, and make it cheap enough to keep. Splitting into processes is only half the defense; the other half is denying each process the power to do harm even if it is compromised. FreePart profiles the system calls each API phase genuinely uses and restricts each agent to exactly that set, so a processing agent that only crunches numbers cannot open files or sockets, and a compromised loading agent cannot spawn a shell — the exploit lands in a box with almost no tools. Because a few security-critical calls are needed only during one-time initialization, FreePart runs the APIs once to initialize and then clamps the restrictions down, keeping the steady-state privilege minimal. The remaining risk to practicality is the cost of moving data across all these new boundaries, which is exactly what sank prior systems; FreePart neutralizes it with lazy data copy, passing outputs directly between agents over shared memory and deferring the copy until the data is actually used, so the vast majority of transfers avoid a round trip through the host. Reducing each component to the minimum authority it needs while engineering away the overhead that makes isolation unaffordable is the same discipline the lightweight fault-isolation work covered elsewhere on this site pursues, delivering strong sandboxing at a cost low enough that developers will actually deploy it.

Following an attack through FreePart

Containing a malicious image that exploits an OpenCV bug in an auto-grader

A trace, drawn from the paper's motivating example, of how FreePart stops a framework exploit from corrupting an application's critical data.

Step 1 of 5

An auto-grader reads student answer sheets with OpenCV

The application scans optical-mark-recognition images, using OpenCV to read each image and then comparing recognized marks against an answer key to compute scores it writes to a file. A key piece of critical data holds the coordinates of the answer regions on the sheet.

In a normal run OpenCV's image-reading and processing functions execute inside the same process as the grader, sharing all its memory.

Why it works

API phases give correct boundaries without static dependency analysis. Tangled framework data flows defeat precise static partitioning → but the runtime sequence of framework API calls already reveals the load-process-visualize-store structure → so FreePart draws sound boundaries from execution instead of from intractable code analysis.
Process isolation contains a framework exploit to one phase. In a shared address space a bug reaches all of the application’s memory → but running each API type in its own OS-isolated process walls off the others → so an exploited vulnerability cannot corrupt the host’s critical data or other phases.
Least privilege disarms a compromised agent. An exploit normally inherits the whole application’s authority → but each agent is confined to only the system calls its phase needs → so even a hijacked agent cannot open files, sockets, or shells it was never granted.
Lazy data copy removes the overhead that sank prior isolation. Routing every cross-boundary transfer through the host costs a copy each time → but passing outputs directly between agents and deferring copies until data is used avoids most round trips → so overhead falls from about 9.7 percent to 3.68 percent.
Data-centric tracking protects what actually matters. Watching code regions alone misses which data an exploit can falsify → but inferring data permissions from the current API phase confines each phase to its legitimate data → so the specific critical values an attack targets stay out of reach.

What it costs (trade-offs and scope)

It defends the framework boundary, not the application’s own bugs. FreePart’s threat model is an attacker exploiting a vulnerability inside a framework API; vulnerabilities in the host application’s own code are explicitly out of scope, so a bug the developer wrote is not contained by this partitioning.
Machine-learning-specific attacks are orthogonal and unaddressed. Attacks on the model itself — adversarial examples that fool a classifier’s decision — are a different problem FreePart does not touch; it hardens against software exploitation, not against a model being tricked by valid-looking inputs.
Correct categorization of hundreds of APIs is essential. The isolation is only as good as the assignment of each framework API to the right phase type; the hybrid static-and-dynamic profiler automates this, but a mis-categorized API could place work in the wrong privilege domain, so the profiling’s accuracy is a real dependency.
The overhead, though low, is not zero, and depends on data flow. Lazy data copy works because outputs usually feed the next framework API directly; the 3.68 percent average rests on that pattern holding, and an application whose data crossings do not fit it (more transfers routed through the host) would see higher cost, closer to the 9.7 percent measured without the optimization.
It trusts the OS and its own runtime, and targets specific frameworks. FreePart relies on the operating system’s process isolation and on its own protected runtime, and is demonstrated on four frameworks (OpenCV, Caffe, PyTorch, TensorFlow) across 23 applications; a compromised OS is out of scope, and applying it to a new framework requires categorizing that framework’s APIs.

The evaluation

Hardening a data-processing application against framework vulnerabilities

Property	Same-process framework	Static-analysis partitioning	FreePart
How boundaries are decided
Contains a framework exploit
Privilege of the compromised part
Runtime overhead
Reported outcome

A comparison drawn from the paper. FreePart is evaluated on 23 applications using OpenCV, Caffe, PyTorch, and TensorFlow, against attacks composed of 18 real-world CVE vulnerabilities; it prevents all of them at an average runtime overhead of 3.68 percent (about 9.7 percent without the lazy-data-copy optimization).Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

The full paper text was available (the authors’ own copy from an author’s site), so this account of the mechanism — the framework-vulnerability threat model, the observation that a data-processing app’s execution flows through framework API calls in temporal phases, the four API types (loading, processing, visualizing, storing), the per-phase isolated agent processes, the hybrid static-and-dynamic profiling that categorizes APIs, the least-privilege system-call restriction, and the lazy-data-copy optimization over shared memory — follows the paper’s own description. The headline numbers are the paper’s reported results: prevention of all attacks composed of 18 real-world CVE vulnerabilities across 23 applications on OpenCV, Caffe, PyTorch, and TensorFlow, at an average runtime overhead of 3.68 percent (about 9.7 percent without lazy data copy, which handles roughly 95 percent of data transfers). The scope excludes vulnerabilities in the host application’s own code and machine-learning model attacks such as adversarial examples, and trusts the operating system and FreePart’s runtime. Exact per-application overheads, the API-categorization details, and the system-call profiles should be read from the paper itself.

Background

Why the framework became the weak point

It is worth seeing how the modern software supply chain created this exposure. A decade ago much data-processing code was written from scratch; today almost none is. A computer-vision product calls OpenCV, a machine-learning product calls PyTorch or TensorFlow, and those frameworks are millions of lines of C and C++ maintained by others, embedded identically into thousands of downstream applications. This is enormously productive — nobody should reimplement image decoding — but it means a single memory-corruption bug in a popular framework function is simultaneously a vulnerability in every application that calls it, and the application developer has neither written nor can easily audit that code. The framework thus becomes a shared, high-value attack surface that the application inherits by dependence alone, which is why its severity is rated so highly and why containing it at the boundary, rather than hoping the framework is bug-free, is the pragmatic response. This concern with shrinking what an untrusted or bug-prone dependency is allowed to reach is the same attack-surface-reduction goal the on-demand code-mapping work covered elsewhere on this site pursues, exposing only the code and capability that are actually needed so that the inevitable flaws have less to work with. FreePart applies that principle specifically to the framework calls at the heart of every data-processing pipeline.

If you remember only three things

1. Data-processing and machine-learning software depends on huge frameworks (TensorFlow, OpenCV, PyTorch, Caffe) that run in the same process and address space as the application, so a memory-corruption vulnerability in a framework API is a critical attack surface: an attacker who calls it with a crafted input can corrupt the host application's critical data, run code, or crash it, regardless of the app's own logic. Existing software isolation either relies on brittle static analysis of tangled framework data flows or imposes excessive IPC overhead, making it impractical.
2. FreePart partitions by the runtime structure of the computation instead of static code: it observes that a data-processing app's execution flows through framework API calls in temporal phases, categorizes the framework's APIs into four types (loading, processing, visualizing, storing) with a hybrid static-and-dynamic profiler, runs each phase's APIs in its own OS-isolated agent process, and confines each agent to only the system calls that phase needs (least privilege). A lazy-data-copy technique passes outputs directly between agents over shared memory, deferring copies until data is used, so isolation stays cheap.
3. Across 23 applications on OpenCV, Caffe, PyTorch, and TensorFlow, FreePart prevents every attack built from 18 real-world CVE vulnerabilities at an average runtime overhead of just 3.68 percent (about 9.7 percent without lazy data copy, which covers roughly 95 percent of transfers). Its scope is framework vulnerabilities, not the application's own bugs or machine-learning model attacks like adversarial examples, and it trusts the OS and its runtime; applying it to a new framework requires categorizing that framework's APIs.

Check yourself

Check your understanding

1. Why is a memory-corruption vulnerability inside a framework like OpenCV so dangerous to an application that uses it?
2. What is FreePart's key insight for deciding how to partition the program?
3. How does FreePart both contain an exploit and keep the isolation cheap?
4. What are FreePart's results and the boundaries of its protection?

Sources

The paper is “FreePart: Hardening Data Processing Software via Framework-based Partitioning and Isolation” by Ali Ahad and Yonghwi Kwon (University of Maryland), Gang Wang (University of Illinois at Urbana-Champaign), Chung Hwan Kim (University of Texas at Dallas), Suman Jana (Columbia University), and Zhiqiang Lin (Ohio State University), published at ASPLOS 2023 (Volume 4), DOI 10.1145/3623278.3624760. It is licensed CC BY.

This explainer is based on the paper’s full text, obtained from an author’s copy (yonghwi-kwon.github.io). The account of the mechanism — the framework-vulnerability threat model in which an attacker exploits a memory-corruption bug in a framework API via crafted input; the observation that a data-processing application’s execution flows through framework API invocations in a temporal order that reveals which critical data is in use; the categorization of framework APIs into four types (loading, processing, visualizing, storing); the execution of each API type in a separate OS-isolated agent process; the hybrid static-and-dynamic profiling that automatically categorizes hundreds of APIs by their data-dependency patterns; the least-privilege restriction of each agent to only the system calls its phase requires (with initialization-only calls permitted once then locked down); and the lazy-data-copy optimization that passes data directly between agents over shared memory to reduce inter-process-communication overhead — follows the paper’s own description. The reported results cited here are the paper’s own: prevention of all attacks composed of 18 real-world CVE vulnerabilities across 23 applications on OpenCV, Caffe, PyTorch, and TensorFlow, at an average runtime overhead of 3.68 percent (about 9.7 percent without lazy data copy, which accounts for roughly 95 percent of data transfers). The scope excludes host-application vulnerabilities and machine-learning model attacks such as adversarial examples, and trusts the operating system and FreePart’s runtime. Exact per-application overheads, the API-categorization methodology, and the system-call profiles should be read from the paper itself.