cuJSON: A Highly Parallel JSON Parser for GPUs

Parsing JSON is a workload everyone assumed GPUs were bad at, because parsing is full of branches and each character's meaning can depend on the ones before it, which fights the GPU's do-the-same-thing-in-lockstep design; this paper (UC Riverside — Ashkan Vedadi Gargary, Soroosh Safari Loaliyan, Zhijia Zhao) presents cuJSON, which redesigns the whole parsing algorithm to be nearly branch-free and to break the sequential dependencies, so all three phases (checking the text encoding, splitting it into tokens, and recovering the nesting structure) run massively in parallel on the GPU and outrun the best CPU and GPU parsers.

The argument in one glance

Problem

JSON parsing is a bottleneck, and GPUs were assumed to be bad at it

JSON is the format the modern web and data pipelines run on: configuration, logs, API responses, and huge analytics datasets are all JSON. Before a program can use that data, it must parse it — read the raw text and turn it into a structured form the machine understands. Parsing is deceptively expensive, and for large datasets it becomes a real bottleneck. A natural thought is to throw a GPU at it, since GPUs have thousands of cores and enormous memory bandwidth. But conventional wisdom says GPUs are the wrong tool for parsing, for two reasons. First, parsing code is branch-heavy: at every character the parser asks questions (is this a brace, a quote, a digit, whitespace?) and takes different paths, and GPUs run their cores in lockstep, so when cores in the same group take different branches they stall waiting for each other. Second, parsing is inherently sequential: what a character means can depend on everything before it — most sharply, whether you are currently inside a quoted string, because a brace inside a string is just text, not structure. That running dependency seems to force you to scan the file left to right, one character at a time, which is the opposite of what a GPU is good at. So the intuition is that JSON parsing simply does not fit the GPU.
What people did

Parse fast on the CPU, or port a parser to the GPU without redesigning it

One line of work makes CPU parsing as fast as possible, most famously simdjson, which uses the CPU's vector (SIMD) instructions to process many bytes at once, and parallel-indexing tools like Pison. Another line moves parsing onto the GPU — systems like cuDF and GPJSON — but largely by mapping existing parsing structure onto GPU threads rather than rethinking the algorithm for the GPU's constraints.
Why it fell short

Branch divergence and sequential dependencies cap the GPU approaches

The CPU parsers, however clever, are ultimately limited by the CPU's core count and memory bandwidth; they cannot tap the GPU's scale. The existing GPU parsers, meanwhile, inherit the very features that make parsing GPU-hostile: if the algorithm still branches heavily, cores in a warp diverge and serialize, wasting the parallel hardware; and if the algorithm still carries the left-to-right dependency (the running "am I inside a string?" state, and the brace-and-bracket matching that builds the nesting structure), then a single large JSON file cannot be split cleanly across thousands of threads, because each chunk's meaning depends on what came before it. The result is that porting a branchy, sequential parser to the GPU leaves most of the hardware idle. What was missing was a parsing algorithm redesigned from the ground up to be nearly branch-free and to break the sequential dependencies, so the GPU's parallelism could actually be used on a single file.
This paper's idea

Redesign the whole parsing pipeline to be branch-free and dependency-broken

cuJSON is a JSON parser built on a new parsing algorithm designed specifically for the GPU: minimal branching and maximal parallelism. It offloads all three key phases of parsing to the GPU — (i) validating that the input is well-formed UTF-8 text, (ii) tokenizing it, meaning splitting the byte stream into the meaningful pieces (structural characters like braces and colons, strings, numbers), and (iii) recognizing the nesting structure, meaning matching the braces and brackets to recover which values are inside which objects and arrays. Each phase is powered by its own highly parallel algorithm that leans on GPU intrinsic functions (fast per-thread and per-warp bit operations) and high-performance CUDA libraries (for parallel primitives like scans) instead of data-dependent branches. Crucially, cuJSON breaks the key dependencies in the parsing process — the running string-state and the structural matching that normally force a sequential scan — so that even a single large JSON file can be parsed in parallel across the whole GPU rather than one character at a time. It also designs its output in a non-conventional way (compact structural and position arrays) so that the parsed result can be queried and navigated efficiently afterward. According to the authors, cuJSON outperforms highly optimized CPU parsers like simdjson and Pison and also surpasses existing GPU parsers like cuDF and GPJSON, in both functionality and performance.

Why this problem is worth solving

JSON parsing sits in front of an enormous amount of computing, so shaving its cost pays off broadly, and GPUs are where the spare throughput is.

JSON is everywhere data moves: web APIs return it, logs are written in it, and analytics systems ingest terabytes of it. Every one of those uses must first parse the text into a usable structure, and because JSON is verbose and irregular, parsing routinely dominates the time to load and process a dataset — it is a classic hidden bottleneck. Meanwhile, machines increasingly have powerful GPUs sitting alongside the CPU, with an order of magnitude more parallel throughput and memory bandwidth. If parsing could be moved onto the GPU effectively, it would both remove a bottleneck and put otherwise-idle hardware to work, and it would let parsing keep pace with GPU-based analytics that already run there (so the data would not have to bounce back to the CPU just to be parsed). The prize is thus not a niche speedup but faster ingestion for a huge class of data-processing pipelines.

The obstacle is that parsing violates the two things a GPU needs — uniform control flow and independent work — so the fix has to be an algorithmic redesign, not a port.

A GPU gets its speed from two assumptions: that its many cores execute the same instruction at the same time (so control flow should be uniform), and that the work splits into independent pieces (so thousands of threads can run without waiting on each other). Parsing breaks both. It is branch-heavy — every byte triggers case analysis — so cores in the same group diverge onto different paths and end up running one after another instead of together, a penalty called warp divergence. And it is sequential — the meaning of a byte can depend on the running state, above all whether you are inside a quoted string, since structural characters lose their meaning inside strings; likewise, recovering the nesting requires matching each closing brace to its opening one, which looks like a strict left-to-right process. That dependency is what really blocks parallelism: if you cannot decide what a chunk in the middle of the file means without first scanning everything before it, you cannot hand different chunks to different threads. So the real crux is not just avoiding branches but breaking the sequential dependency, so that the file can be carved into independent pieces that the GPU parses at once. Achieving that while still producing a correct parse is the hard, worthwhile problem.

What people did before, and where it broke

Optimize CPU parsing with vector instructions (simdjson). Use the CPU’s SIMD units to process many bytes per instruction. Where it broke. However well tuned, it is bounded by the CPU’s limited core count and memory bandwidth, and cannot reach the GPU’s scale of parallelism.
Build parallel indexing parsers on the CPU (Pison). Parse and index JSON with multiple CPU threads for analytics. Where it broke. Same ceiling: CPU parallelism is modest compared to a GPU, so throughput on very large datasets is limited.
Port a parser onto the GPU (cuDF, GPJSON). Map existing parsing work onto GPU threads. Where it broke. Keeping the branch-heavy, sequential structure means warps diverge and the left-to-right dependency remains, so a single large file cannot be split across threads and much of the GPU sits idle.
Split the file into chunks and parse each independently. Divide the input and assign chunks to threads. Where it broke. Naive splitting is unsafe because a chunk boundary may fall inside a string or in the middle of a nested structure, so a chunk cannot be interpreted without knowing the state carried in from before it.
The gap. No JSON parser was redesigned so that all of parsing — encoding validation, tokenization, and nesting recovery — is branch-free and dependency-broken enough to parse even a single large file in parallel on a GPU.

How cuJSON works

Turning JSON parsing into three parallel, branch-free GPU passes

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Make a task believed GPU-unfriendly run massively parallel by redesigning it. cuJSON’s premise is that JSON parsing is not inherently unsuited to GPUs — the standard parsing algorithm is, and a different algorithm can fit. Taking a workload that looks hostile to the GPU’s lockstep, throughput-oriented hardware and reshaping it so the hardware’s parallelism can actually be applied is the same undertaking the work on boosting the scalability of deformable convolution operations on GPUs (DeFT) covered elsewhere on this site pursues, bending an irregular computation to fit the accelerator. cuJSON does this for parsing, replacing branchy, sequential logic with parallel, branch-light phases.

Compute with bit-level and vector-style parallelism instead of per-character branches. The heart of cuJSON’s tokenization is doing many bytes’ worth of decisions at once with GPU intrinsic bit operations, echoing how the fastest CPU parsers use vector instructions — trading a stream of conditional branches for a few wide, uniform operations. Exploiting a processor’s wide, special-purpose parallel instructions to replace slow scalar, branchy work is the same principle the work on efficient VNNI utilization for linear algebra covered elsewhere on this site is built on, extracting speed from the machine’s vector units. cuJSON applies this idea on the GPU, using warp-level bit primitives to find structure without diverging.

Design the output as an index that makes later queries cheap. cuJSON does not just emit a parse tree; it emits compact structural and position arrays purpose-built so a downstream iterator can skip over nested structures and jump straight to the field it needs. Shaping the stored form of data specifically so that the queries run over it later are fast is the same idea the work on an analytics object store optimized for query pushdown (Fusion) covered elsewhere on this site develops, co-designing the layout with the access pattern. cuJSON co-designs its parsed output with its query iterator, so parsing and querying reinforce each other.

Following a JSON file through cuJSON

Parsing one large JSON document on the GPU

An illustrative trace, drawn from the paper's description and the authors' code, of cuJSON's phases. Exact algorithmic details should be read in the paper.

Step 1 of 5

The raw JSON bytes are loaded onto the GPU

A large JSON document is transferred to GPU memory as a byte stream. The goal is to parse the whole thing in parallel, not to scan it left to right on one core.

Filling the GPU normally needs independent work; parsing seems to have none.

Why it works

A branch-free redesign avoids warp divergence. Per-character case analysis makes GPU cores diverge and serialize → but recasting each phase as branch-light bit and parallel operations keeps cores in lockstep → so the GPU’s cores actually run together instead of waiting on each other.
Breaking the sequential dependency unlocks single-file parallelism. The left-to-right “am I in a string?” and brace-matching dependencies force a serial scan → but cuJSON computes these states in parallel → so one large file can be split across thousands of threads rather than needing many files.
GPU intrinsics and CUDA libraries replace scalar work. Character-by-character logic underuses the hardware → but warp-level bit operations and parallel primitives (like scans) do the same work in wide, uniform steps → so tokenization and structure recovery run at the GPU’s throughput.
All three phases run on the GPU, avoiding round-trips. Splitting parsing between CPU and GPU would add transfers → but validation, tokenization, and nesting recovery all run on the GPU → so the data stays put and the pipeline is not bottlenecked on the CPU.
A query-friendly output makes parsing pay off downstream. A parse that is expensive to navigate would shift the cost to later → but emitting compact structural and position arrays lets an iterator skip nested structures → so queries over the parsed data are fast too.

What it costs (trade-offs and scope)

It requires a GPU and CUDA-specific engineering. cuJSON is built on GPU intrinsic functions and CUDA libraries and targets NVIDIA GPUs; its benefits are unavailable without such hardware, and the algorithm is tuned to that execution model rather than being portable in the way a plain CPU parser is.
The output is non-conventional. To maximize speed and query efficiency, cuJSON emits compact structural and position arrays rather than a conventional parse tree or document object; consumers must use its iterator interface, so integrating it means adopting its representation.
The wins are largest on large inputs. GPU parsing amortizes data-transfer and launch overheads over the work done, so the approach is aimed at large JSON documents and datasets; for tiny inputs, moving data to the GPU may not pay off.
Correctness rests on the redesigned algorithms. Breaking the sequential dependencies in string-state resolution and brace matching is subtle, and the parser must still handle the full range of valid JSON (escapes, nested structures, encodings) correctly; the authors report full functionality against prior parsers, which the paper details.
This explainer is drawn from limited source material. The full paper text was not accessible for this write-up; the description here is based on the abstract and the authors’ public repository, so specific quantitative results and the precise algorithms should be confirmed in the paper itself.

How it compares

Approaches to fast JSON parsing

Property	CPU SIMD parsers (simdjson, Pison)	Ported GPU parsers (cuDF, GPJSON)	cuJSON
Hardware used
Control flow
Single large file
Output form
Reported standing

A qualitative comparison of JSON parsing approaches, based on the paper's abstract and the authors' repository. cuJSON redesigns the parsing algorithm to be branch-light and to break the sequential dependencies so all three phases run in parallel on the GPU, and the authors report it outperforms both leading CPU parsers (simdjson, Pison) and existing GPU parsers (cuDF, GPJSON) in functionality and performance. Specific speedup figures should be read from the paper.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is based on the paper’s abstract and the authors’ public code repository, not its full text, which was not accessible for this write-up. The system and claims described are the authors’: cuJSON is a JSON parser built on a redesigned, GPU-tailored parsing algorithm with minimal branching and maximal parallelism. It offloads all three key phases of JSON parsing to the GPU — UTF-8 validation, tokenization, and nesting-structure recognition — each powered by a highly parallel algorithm that leverages GPU intrinsic functions and high-performance CUDA libraries. cuJSON breaks the key dependencies in the parsing process so that even a single large JSON file can be parsed in parallel, and it designs its output in a non-conventional way (compact structural and position arrays consumed by a query iterator) to speed up parsing and downstream querying. The authors state that cuJSON outperforms highly optimized CPU parsers (simdjson, Pison) and existing GPU parsers (cuDF, GPJSON) in both functionality and performance. No specific speedup numbers, figure contents, or precise algorithmic details are asserted here because the full paper was unavailable; the exact algorithms, the experimental setup, the datasets, and all quantitative results should be read from the paper and the repository. This page is marked as a draft for that reason.

Background

Why parsing keeps resisting parallel hardware

It helps to see cuJSON within a long struggle to make text and structure processing keep up with everything else. Parsing — turning a flat stream of bytes into structured meaning — is one of the oldest tasks in computing, and it has always been awkward to parallelize, because meaning accumulates left to right: what a byte signifies can depend on the state built up from all the bytes before it. As processors gained wide vector units and then massive GPU parallelism, most numeric workloads sprinted ahead, while parsing lagged, still shaped by inherently sequential state machines. Progress has come from a repeated move: rather than parallelizing the sequential algorithm, invent a new algorithm that computes the same result while exposing independent work — using bit tricks to find structure without branches, and clever reformulations to break the running dependencies so chunks can be processed at once. The broad effort to make tokenization and text ingestion efficient and parallel is the same terrain the work on static analysis for efficient streaming tokenization covered elsewhere on this site occupies, attacking the front end of data processing where bytes become tokens. cuJSON pushes this onto the GPU, arguing that even a task as branchy and sequential-seeming as JSON parsing can be redesigned to run at the throughput of massively parallel hardware.

If you remember only three things

1. JSON parsing is a common bottleneck, and GPUs were assumed to be a poor fit because parsing is branch-heavy (per-character case analysis makes GPU cores diverge and serialize) and inherently sequential (a byte's meaning can depend on prior state, such as whether it sits inside a quoted string, and nesting recovery looks like a strict left-to-right brace-matching pass). Porting existing parsers to the GPU keeps these properties and leaves most of the hardware idle.
2. cuJSON is a JSON parser built on a GPU-tailored algorithm with minimal branching and maximal parallelism. It runs all three parsing phases on the GPU — UTF-8 validation, tokenization, and nesting-structure recognition — each as a highly parallel, branch-light algorithm using GPU intrinsic functions and CUDA parallel libraries. Its key move is breaking the sequential dependencies in the parsing process, so that even a single large JSON file can be split across thousands of threads and parsed at once, and it emits a non-conventional compact output (structural and position arrays) designed for a fast query iterator.
3. According to the authors, cuJSON outperforms highly optimized CPU parsers (simdjson, Pison) and existing GPU parsers (cuDF, GPJSON) in both functionality and performance. The trade-offs: it requires an NVIDIA GPU and CUDA-specific engineering, uses a non-conventional output that consumers must adopt via its iterator, and pays off most on large inputs. This explainer is a draft based on the abstract and the authors' repository; the exact algorithms, datasets, and quantitative results should be confirmed in the paper.

Check yourself

Check your understanding

1. Why was JSON parsing considered a poor fit for GPUs?
2. What is cuJSON's core strategy for making parsing run in parallel on a GPU?
3. Why does cuJSON produce its parsed output in a "non-conventional" form?
4. Predict: a user runs cuJSON on one enormous JSON document on a GPU, versus on a single very small JSON snippet. Where should cuJSON help most, and why?

Sources

The paper is “cuJSON: A Highly Parallel JSON Parser for GPUs” by Ashkan Vedadi Gargary, Soroosh Safari Loaliyan, and Zhijia Zhao (University of California, Riverside), published at ASPLOS 2026, DOI 10.1145/3760250.3762222 (pages 85 to 100). The code is available at github.com/AutomataLab/cuJSON.

This explainer is written from the paper’s abstract and the authors’ public repository, because the full paper text was not accessible for this write-up (the ACM version was not retrievable and no author preprint was found). cuJSON is a JSON parser built on a redesigned parsing algorithm tailored for GPU architectures, with minimal branching and maximal parallelism, challenging the conventional wisdom that GPUs are ill-suited to branch-heavy parsing. It offloads all three key phases of JSON parsing to the GPU — UTF-8 validation, tokenization, and nesting-structure recognition — each powered by a highly parallel algorithm that leverages GPU intrinsic functions and high-performance CUDA libraries. cuJSON breaks the key dependencies in the parsing process so that a single large JSON file can be parsed in parallel, and it designs its output in a non-conventional way (compact structural and position arrays) to accelerate both parsing and downstream querying via a query iterator. The authors state that cuJSON outperforms highly optimized CPU-based parsers such as simdjson and Pison and surpasses existing GPU-based parsers such as cuDF and GPJSON, in both functionality and performance. No specific quantitative results, figure contents, or precise algorithms are asserted here; they should be confirmed in the paper and repository. This page is marked draft because the full text was unavailable.