Java programs constantly turn in-memory objects into bytes to store or send them (serialization), but the fast way needs a hand-written schema for every type while the flexible way uses one slow general-purpose serializer — and under heavy load neither is good enough; this paper (Shanghai Jiao Tong University, Ant Group, Alibaba, and colleagues — Chaokun Yang, Yuting Chen, and others) presents JOSer, which generates specialized serialization code for each class at runtime and lets the Java Virtual Machine's own just-in-time compiler keep optimizing it, getting both flexibility and speed.
The argument in one glance
-
Problem
Serializing Java objects is everywhere, and under heavy load it is a bottleneck
Serialization is the act of turning an in-memory object into a stream of bytes, so it can be written to disk or sent across the network to another machine, and later turned back into an object (deserialization). In Java it is pervasive and critical: distributed data systems, caches, remote calls, and big-data frameworks constantly serialize and deserialize enormous numbers of objects. When a workload is serialization-heavy — a stream processor like Flink shuffling records between operators, or a search-and-recommendation service moving objects between stages — the time spent serializing becomes a first-order cost, consuming CPU and adding latency to every request. So the efficiency of the serializer directly shapes the performance of the whole system. The trouble is that the serialization techniques Java systems rely on today are, under this kind of load, either inflexible or inefficient. The two dominant styles each fail in their own way, and the failure is not a minor tax — the paper reports order-of-magnitude gaps between what is used today and what is achievable. For systems that spend a large fraction of their time moving objects around, that gap is a lot of wasted hardware and a lot of avoidable latency.
-
What people did
Use a hand-written schema, or a single general-purpose serializer
Java serialization comes in two flavors. The schema-based style (as in formats requiring an interface definition) has the developer define an explicit schema for each type; from that schema, fast, specialized serialization code can be produced. The omni-functional style (as in Java's built-in serialization, and general-purpose libraries) instead uses one all-purpose serializer that can handle any object without a schema, by inspecting objects at runtime through reflection. The first is fast; the second is convenient and flexible.
-
Why it fell short
Schemas are a burden; the one-size-fits-all serializer is slow
Each style trades away exactly what the other has. Schema-based serialization is fast but inflexible: a developer must manually define and maintain a schema for every type that will be serialized, which is a real burden and does not fit the idiomatic domain objects a Java program naturally uses; adding or changing a type means changing a schema. The omni-functional serializer is flexible but inefficient: because a single serializer must work for every possible object, it leans on reflection and generic, branchy logic to discover an object's fields at runtime and handle every case. That generic code is slow, and crucially it is opaque to the Java Virtual Machine's just-in-time (JIT) compiler — the JVM cannot specialize or deeply optimize a one-size-fits-all routine the way it optimizes ordinary hot, type-specific code. So under heavy load you are forced to choose between the maintenance cost of hand-written schemas and the runtime cost of a slow universal serializer. What was missing was a way to get the flexibility of handling any object automatically together with the speed of specialized, optimizable code.
-
This paper's idea
Generate class-specific serializers at runtime and let the JIT optimize them
JOSer (Just-in-time Object SERializer) resolves the dilemma by generating the specialized code automatically, at runtime, rather than making the developer write a schema or accepting a generic serializer. Its founding observation is a serialization-specific optimization opportunity in the Java runtime: instead of one omni-functional serializer, JOSer generates a set of class-specific serializers — actual serialization code tailored to each class's exact fields and layout — and, critically, generates them in a JIT-friendly form. Because these serializers are ordinary, type-specific Java methods with direct field access rather than reflection, the JVM's just-in-time compiler can treat them like any other hot code path: it compiles them to fast native code and continuously optimizes them as they run, inlining and specializing them over time. This is the key move — JOSer does not build its own optimizer; it shapes the generated code so the JVM's existing, highly-tuned JIT compiler does the optimizing. JOSer also shares metadata across the process: the metadata of the objects being serialized, and of the serializers being optimized, is shared and reused rather than recomputed for each object, cutting redundant work. The result is both flexible (it handles idiomatic Java objects automatically, with no hand-written schema) and fast (the generated code is specialized and JIT-optimized). Evaluated against six Java serialization techniques including OpenJDK's built-in serialization, JOSer improves throughput by up to 20 to 83 times for serialization and 43 to 229 times for deserialization. It has been deployed in real products — reducing Flink's serialization CPU usage by roughly 35 to 41 percent and cutting search-recommendation latency by more than 30 milliseconds — and it grounds Apache Fory, an open-source serialization framework.
Why this problem is worth solving
Serialization is a hidden but enormous cost in the data-intensive systems that dominate modern computing, so making it faster pays off broadly and directly. In distributed data systems, stream processors, microservices, and caches, objects are constantly being turned into bytes and back — every time data crosses a network link, a process boundary, or a storage layer. This work is invisible in the application logic but very real in the profile: for serialization-heavy systems it can consume a large fraction of CPU and add latency to every operation. Because it sits on the critical path of so much data movement, even a proportional improvement in serialization throughput translates into meaningful savings in hardware and responsiveness across a huge range of software. The paper’s deployment numbers make this concrete: cutting a major stream processor’s serialization CPU usage by more than a third, and shaving tens of milliseconds off recommendation latency, are the kind of gains that matter at production scale. And since serialization is a general-purpose primitive used by countless systems, a technique that improves it is not a niche optimization but one that compounds across the whole ecosystem that depends on it.
The heart of the matter is a false choice between flexibility and speed that the runtime itself can dissolve. The reason serialization has been stuck choosing between schemas and generic serializers is a belief that flexibility requires generality, and generality requires slow, reflective, one-size-fits-all code. JOSer’s insight is that this is false in a managed runtime like the JVM, because the runtime can generate specialized code on the fly. Once you accept that the serializer can be produced per class at runtime, the flexibility problem dissolves — no human writes a schema, because the code is generated automatically from the class — and the speed problem dissolves too, but only if the generated code is shaped so the JIT compiler can optimize it. That last condition is the subtle, decisive part. A general serializer is opaque to the JIT: its behavior depends on runtime type inspection, so the compiler cannot specialize it. Class-specific generated code, by contrast, looks to the JVM exactly like ordinary type-specific Java, full of direct field accesses and monomorphic calls, which the JIT is superb at compiling and continuously reoptimizing. So the whole design hinges on generating not just specialized code, but JIT-friendly specialized code, and then getting out of the way and letting the JVM’s decades-tuned optimizer do the heavy lifting. Sharing metadata across objects and serializers then removes the remaining redundant setup cost.
What people did before, and where it broke
- Schema-based serialization with an interface definition. Define a schema per type and generate fast specialized code from it. Where it broke. It requires manually defining and maintaining a schema for every type, a burden that does not fit idiomatic Java domain objects and changes whenever a type does.
- Omni-functional serialization (Java built-in, general libraries). Use one all-purpose serializer that handles any object via reflection. Where it broke. The generic, reflective logic is slow, and it is opaque to the JVM JIT, which cannot specialize a one-size-fits-all routine the way it optimizes type-specific code.
- Rely on reflection to discover fields at runtime. Inspect each object’s structure dynamically during serialization. Where it broke. Reflection is costly and repeated per object, and it prevents the compiler from generating tight, specialized field-access code.
- Recompute per-object setup each time. Handle each object’s metadata independently. Where it broke. It repeats work — type descriptors, layout information — that is shared across objects of the same class and across the serializers being optimized.
- The gap. No technique delivered the flexibility of handling any object automatically together with the speed of specialized code the JVM JIT could continuously optimize.
How JOSer works
Generating specialized serializers the JVM JIT can keep optimizing
-
-
new
-
new
-
new
-
new
-
new
The idea
Generate specialized code at runtime instead of interpreting a general one. JOSer’s core move is to replace one reflective, all-purpose serializer with per-class serialization code produced on the fly, so each class gets a routine tailored exactly to it. Turning a slow, general runtime activity into fast specialized code generated for the specific case at hand, rather than interpreting a one-size-fits-all path, is the same approach the work on domain-specific run-time optimization for software data planes covered elsewhere on this site takes, specializing the hot path at runtime to the actual work it does. JOSer specializes the serializer to each class, which is what removes the reflection and generic branching that made the universal serializer slow.
Ride the platform’s existing compiler rather than building your own. JOSer deliberately shapes its generated serializers to be JIT-friendly and then hands them to the JVM’s just-in-time compiler, inheriting all of its optimization instead of reimplementing it. Getting high performance by producing code in a form a powerful existing compiler already optimizes well, rather than constructing a bespoke optimizer, is the same principle the work on simplifying and optimizing sparse GPU kernels by lowering to a dense compiler (Insum) covered elsewhere on this site is built on, borrowing a mature compiler’s power. JOSer borrows the JVM JIT’s decades of tuning by making its serializers look like ordinary hot Java code.
Make moving structured data between systems fast. At bottom JOSer is about turning objects into bytes and back quickly, the workhorse operation whenever structured data crosses a boundary. Driving down the cost of parsing and processing a structured data representation at high throughput is the same goal the work on streaming semi-structured data with bit-parallel fast-forwarding (JSONSki) covered elsewhere on this site pursues, attacking the per-record cost of a data format. JOSer does it for Java object serialization, where the format is the object graph and the win is order-of-magnitude throughput.
Following an object through JOSer
Serializing objects of a class for the first time and then at speed
An illustrative walkthrough, based on the paper's abstract and the Apache Fory project (this explainer is written without the full paper text; exact details are in the paper).
Step 1 of 5
A class is serialized for the first time
When JOSer first encounters objects of a class, it inspects the class's fields and layout once and generates a serializer specialized to that class — code that reads and writes exactly those fields in order, without reflection.
No developer-written schema is needed; the serializer is derived from the class.
Why it works
- Specialized code drops the reflection and generic branching. A universal serializer inspects each object with reflection → but generating per-class code with direct field access → so the slow, generic runtime inspection that dominated cost is eliminated.
- JIT-friendly code lets the JVM do the optimizing. A general serializer is opaque to the JIT and cannot be specialized → but class-specific code looks like ordinary hot Java → so the JVM’s just-in-time compiler compiles and continuously reoptimizes it.
- Runtime generation preserves flexibility without a schema. Fast serialization used to require hand-written schemas → but generating the serializer automatically from the class at runtime → so any idiomatic Java object is handled with no developer schema burden.
- Sharing metadata removes repeated setup. Handling each object independently repeats type and layout work → but sharing metadata across objects and across the serializers being optimized → so redundant per-object setup is cut.
- The result gets both flexibility and speed. Prior techniques forced a choice between the two → but automatic generation gives flexibility while specialization and the JIT give speed → so JOSer reaches order-of-magnitude throughput gains on any object.
What it costs (trade-offs and scope)
- Generation and warm-up have a first-time cost. Producing a class-specific serializer and letting the JIT compile it takes some initial work, so the biggest wins accrue to heavy, repeated serialization of the same classes; a workload that serializes each class only a handful of times amortizes less.
- It is tied to the managed runtime and its JIT. JOSer’s advantage comes from generating code the JVM just-in-time compiler can optimize; the approach is specific to a runtime that offers runtime code generation and a strong JIT, so it is a Java-and-JVM technique rather than a universal one.
- Generated serializers and metadata consume memory. Keeping class-specific serializers and shared metadata resident uses memory and management overhead that a single universal serializer does not, a trade the design makes to gain speed.
- The gains are largest under heavy serialization workloads. The technique targets systems that serialize enormously; where serialization is a small part of the workload, the order-of-magnitude throughput improvements translate into smaller end-to-end wins.
- Reported numbers are workload- and deployment-specific. The up-to-20-to-83-times and 43-to-229-times figures, and the Flink CPU and recommendation-latency reductions, come from specific benchmarks and production deployments; the exact improvement depends on the objects, the baseline serializer, and the system.
The evaluation
Serializing Java objects under heavy workloads
| Property | Schema-based serializer | Omni-functional serializer | JOSer |
|---|---|---|---|
| Handles any object automatically | |||
| Developer burden | |||
| Optimizable by the JVM JIT | |||
| Redundant per-object work | |||
| Throughput under heavy load |
Check yourself
Check your understanding
1. Why are the two dominant styles of Java serialization inadequate under heavy workloads?
2. What is JOSer's core idea for getting both flexibility and speed?
3. Why is it important that JOSer's generated serializers are "JIT-friendly," and what else does JOSer share to cut work?
4. What does JOSer achieve, and what should be kept in mind about its scope?
Sources
The paper is “JOSer: Just-In-Time Object Serialization for Heavy Java Serialization Workloads” by Chaokun Yang, Pengbo Nie, Ziyi Lin, Weipeng Wang, Qianwei Yu, Chengcheng Wan, He Jiang, and Yuting Chen (Shanghai Jiao Tong University, Ant Group, Alibaba Group, East China Normal University, and Dalian University of Technology), published at ASPLOS 2026, DOI 10.1145/3779212.3790179. The paper is open access under CC BY, and JOSer grounds the open-source Apache Fory serialization framework at fory.apache.org.
This explainer is written from the paper’s abstract and the open-source Apache Fory project; the full paper text was not accessible at the time of writing (the open-access copy was reachable only through a publisher page that could not be retrieved, and no author-hosted copy or preprint was found). JOSer (Just-in-time Object SERializer) is a JIT object serialization technique that, at runtime, generates a set of class-specific, JIT-friendly object serializers and continuously optimizes them with the JVM’s just-in-time compiler, and shares the metadata of objects under serialization and of serializers under optimization. It exploits a serialization-specific optimization opportunity in Java to overcome the inflexibility of schema-based serializers and the inefficiency of omni-functional serializers under heavy serialization workloads. The paper reports evaluating JOSer against six Java serialization techniques including OpenJDK’s built-in serialization, improving throughput by up to 20 to 83 times in serialization and 43 to 229 times in deserialization, and reports production deployment reducing Flink’s serialization CPU usage by roughly 35.32 to 41.14 percent and search-recommendation latency by more than 30 milliseconds. Numbers are the paper’s own; exact mechanisms and full results should be confirmed in the paper.