LightRidge: An End-to-end Agile Design Framework for Diffractive Optical Neural Networks

Yingjie Li, Ruiyang Chen, Minhan Lou, Berardi Sensale Rodriguez, Weilu Gao and Cunxi Yu

A diffractive optical neural network computes by literally shining light through stacked panels whose etched patterns act as the weights — inference happens at the speed of light with almost no energy — but designing one demands rare expertise across optics, fabrication, and machine learning, and the physics is so slow to simulate that training a small model took days; this work from Maryland and Utah builds LightRidge, an end-to-end framework whose fast, differentiable optical-physics engine, design-space explorer, and simple programming language turn a months-long, expert-only process into hours, verified on real optical hardware.

The argument in one glance

Problem

Optical neural networks are efficient but brutally hard to design

A diffractive optical neural network (DONN) computes with light: an image is encoded on a laser beam that diffracts through a stack of panels whose etched phase patterns are the weights, and a detector reads the output light pattern. Inference runs at the speed of light and, since the panels are passive, at almost no energy — a huge efficiency win over digital accelerators. But designing one is extremely hard, demanding combined expertise in optics, fabrication, and machine learning that few people have.
What people did

Simulate the optics with existing optical emulation tools

Because the model must be trained digitally before it is built in glass, designers simulate the light physics with existing optical emulation engines — commercial tools like BeamLab and open-source ones like LightPipes — which model how light diffracts and phase-modulates through the system.
Why it fell short

Those tools are not differentiable, not fast, and hardware-blind

Existing optical simulators focus on modeling the physics but lack what training a DONN needs. Their core functions are not differentiable, so backpropagation cannot train the phase-pattern weights; they are not runtime-optimized (no tensor operators or operator fusion), so simulating a small five-layer network took three to four days; and they are hardware-blind, so a model trained in simulation can lose at least thirty percent of its accuracy when deployed to real optical devices with their limited, noisy phase levels.
This paper's idea

An end-to-end, differentiable, hardware-aware optical ML framework

LightRidge is the first end-to-end optical ML compilation framework. It provides precise, differentiable optical-physics kernels (FFT-based scalar diffraction) that let backpropagation train DONNs, accelerated on CPU and GPU so emulation runs in minutes to hours; a design-space-exploration engine and hardware-aware codesign that pick good optical parameters automatically and map the model onto the device's real quantized response; and a simple domain-specific language that carries a design from exploration through training to a fabrication file, cutting a months-long expert process to hours.

Why this problem is worth solving

To see why a design tool is the bottleneck, it helps to understand how the machine computes.

In a conventional neural network, electrons move between transistors to do the arithmetic. A diffractive optical neural network replaces the electrons with light. The input — say an image — is encoded onto the intensity of a coherent laser beam. That beam then passes through a sequence of diffractive layers stacked in space; as the light travels between layers it spreads out and interferes, and this natural light diffraction plays the role of the connections between neurons, moving and mixing information. At each layer sits a pattern of phase modulators — one per pixel — that shift the phase of the light passing through, and these phase patterns are the learnable weights, the optical analog of a weight matrix. After the light has propagated through all the layers, a detector (a camera) at the end captures the resulting intensity pattern, and the region of the detector that lights up most brightly is read out as the prediction. The remarkable thing is that the entire computation happens physically, as light propagates, at the speed of light and with no energy spent on the arithmetic itself, because the diffractive layers are passive — the only energy goes into the laser and, crucially, into the detector that converts the light pattern back to digital numbers. That combination of speed and near-zero compute energy is why optical neural networks are so attractive as the energy cost of digital deep learning becomes unsustainable.

The obstacle is that all the difficulty moves from running the network to designing it.

Although a DONN’s inference is all-optical, the design that produces it is done entirely on a digital computer, by simulating the optical physics and training the phase patterns with backpropagation just like an ordinary network. That places enormous weight on the emulation engine, and it must satisfy several demands at once that existing optical simulators do not. First, it must be differentiable: to train the weights by backpropagation, gradients have to flow through the whole physics simulation, but classic optical tools compute the light field with functions that are not differentiable, so they cannot be trained against. Second, it must be fast: simulating light diffraction is computationally heavy, and without runtime optimization — tensor representations, operator fusion, accelerated kernels — training even a small five-layer network took three to four days. Third, it must be hardware-aware: real optical devices have only a limited number of discrete phase levels and suffer fabrication noise and non-uniform response, so a model trained against idealized physics degrades badly when built, with reported accuracy drops of thirty percent or more. On top of all this sits a fourth barrier: designing a DONN at all requires rare, combined expertise in optics, fabrication, and machine learning, and a large design space of optical and fabrication parameters to search. A framework that removes these barriers is what makes DONNs practical.

What people did before, and where it broke

Commercial and open-source optical simulators (BeamLab, LightPipes). Model how light diffracts and phase-modulates through an optical system. Where it broke. They focus on simulating the physics, not on training: their core functions are not differentiable, so backpropagation cannot optimize the weights, and they lack runtime optimization, so emulation is extremely slow.
Custom PyTorch or TensorFlow implementations. Hand-code the optical physics inside a deep-learning framework to get differentiability. Where it broke. This takes enormous programming effort (tens of times more lines of code), has no design-space exploration, and still runs slowly and ignores the real device, so the hardware-deployment gap remains.
Hardware-in-the-loop adaptive training with manual calibration. Train against the physical system directly and calibrate by hand to close the deployment gap. Where it broke. It is slow and labor-intensive — days to weeks per design with manual hardware calibration — and does not lower the expertise barrier.
The gap. No framework provided precise, differentiable, fast, hardware-aware optical emulation together with automatic design-space exploration and an easy programming interface — everything needed to go from a task to a deployed optical chip without rare cross-disciplinary expertise. Building that end-to-end framework is what LightRidge contributes.

How LightRidge works

From a slow, hardware-blind simulation to an end-to-end optical design flow

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Compute in an unconventional physical substrate — light — for near-zero-energy inference. LightRidge exists to make practical a machine that does its arithmetic not with switching transistors but with light diffracting through passive glass, where the computation is free and instantaneous and only the readout costs energy. Harnessing an unconventional physical substrate whose natural behavior does the computation at a fraction of digital energy is the same bet the temporal-arithmetic work covered elsewhere on this site makes, computing in the analog timing domain of ordinary circuits precisely because that physical behavior is nearly free where digital switching is not.

Turn the physics into a precise, differentiable intermediate representation that bridges ML to hardware. The heart of LightRidge is an optical-physics kernel that is at once accurate to the real device, fast, and differentiable — an intermediate representation that lets a high-level ML training loop compile down to a physical optical system. Building a precise intermediate representation that spans the gap between a high-level program and diverse hardware, so the same design can be optimized and lowered to a real device, is exactly the role the fixed-point instruction-selection work covered elsewhere on this site gives its IR, a shared, well-defined layer that makes compiling to the target both possible and efficient.

Choose the approximate-but-scalable model over the exact-but-intractable one. Solving the full electromagnetic equations is exact but requires a grid that explodes exponentially with size, so LightRidge deliberately uses an FFT-based scalar-diffraction approximation that is precise enough for these systems and scales, trading unnecessary exactness for tractability. Accepting a controlled approximation because the exact method is exponentially expensive and the approximation is verified good enough is the same discipline the fermion-to-qubit encoding work covered elsewhere on this site follows, dropping the exponentially costly part of its formulation once it is shown to almost never matter, so the problem becomes solvable at scale.

Following a design through LightRidge

From a task to a fabricated optical chip

Tracing one design shows exploration, differentiable training, and codesign at work.

Step 1 of 5

State the task and the hardware constraints

A user specifies a target machine-learning task (say handwritten-digit recognition) and hardware constraints — chip size, pixel size, laser wavelength — without needing to be an optics expert.

The optical design space is broad, spanning optics and fabrication parameters.

Why it works

The approximate physics scales where the exact one cannot. Solving Maxwell’s equations exactly needs a fine grid that blows up exponentially with system size → so it is infeasible for large networks → but FFT-based scalar diffraction is precise enough and scales, so large DONNs can actually be emulated.
Casting the physics as tensors makes it trainable. Backpropagation needs gradients through the whole model → so the physics emulation must be differentiable → but expressing diffraction as complex FFTs and matrix multiplies makes it fully differentiable, so standard autodiff trains the optical weights.
Accelerating the dominant kernel dominates the speedup. The diffraction modeling is over ninety percent of the runtime → so accelerating those kernels is what matters → so complex-tensor libraries and operator fusion cut the design cycle from days to hours, up to twelve times faster than the commercial tool.
Codesign closes the deployment gap. A trained model normally loses accuracy on real hardware because the device has limited, noisy phase levels → but folding the device’s measured response into differentiable quantization-aware training maps the weights onto what the hardware can do → so the model deploys with almost no calibration and matches the emulation.
Automation removes the expertise barrier. Designing a DONN needs rare cross-disciplinary expertise → but a learned design-space explorer and a simple language choose the parameters and automate the flow → so a non-expert reaches a fabricated design in hours, and the compute runs on passive optics at near-zero energy.

What it costs (trade-offs and scope)

DONNs still trail digital networks in accuracy. The optical prototype lands about one percent below a comparable digital network on simple datasets, and optical neural networks remain an early-stage technology that struggles with more complex tasks.
The physics is an approximation. FFT-based scalar diffraction is chosen for scalability over the exact but intractable full electromagnetic solution, so it applies where the scalar approximations hold, not in regimes where they break down.
Only the optical compute is free. The passive diffractive layers cost no energy, but the laser source and especially the detector and its analog-to-digital readout do — so practical efficiency is bounded by the readout, not the ideal zero-energy compute.
The hardware is bulky or hard to fabricate. Free-space DONNs are physically large; compact on-chip integration needs advanced 3D nanofabrication, and once fabricated an on-chip DONN’s weights are fixed unless it is rebuilt.
It is a design and emulation framework, not a guarantee. Physical accuracy still depends on device quality and noise; the codesign closes much of the gap, but the framework models the hardware rather than replacing careful physical setup.
It targets diffractive optical networks specifically. The kernels and flow are built for diffraction-and-phase-modulation systems, other optical-computing styles would need their own modeling, and very large systems still hit single-GPU resource limits.

The evaluation

Three ways to design a diffractive optical neural network

Property	Custom PyTorch / TensorFlow	Optical simulators (LightPipes)
Differentiable optical physics for training	Hand-rolled, hard
Design-space exploration	None	None
Emulation runtime	Days
Hardware correlation on deployment	Manual; up to 30 percent loss	Manual calibration

Three ways to design a diffractive optical neural network. Prior tools either cannot be trained against (optical simulators are not differentiable) or take tens of times more code and days of runtime (custom PyTorch/TensorFlow), and both leave a large hardware-deployment gap. LightRidge provides differentiable, accelerated optical-physics kernels (up to 6.4 times faster on CPU and 12 times on GPU than LightPipes), a learned design-space explorer, and hardware-aware codesign — validated on a real 532-nanometer optical prototype whose measurements matched the emulation, reaching 0.98 and 0.89 accuracy on MNIST and FashionMNIST at roughly 995 frames per second per watt, about two orders of magnitude more energy-efficient than desktop CPUs and GPUs, and demonstrating the first all-optical image segmentation.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

LightRidge is a genuinely enabling piece of systems work: it takes diffractive optical neural networks — a promising path to near-zero-energy inference that was previously buildable only by rare experts over months — and turns their design into an automated, hours-long flow. Its differentiable, accelerated FFT-based physics kernels make optical networks trainable by ordinary backpropagation and up to twelve times faster to emulate than a commercial tool; its learned design-space explorer removes the optics-expertise barrier; and its hardware-aware codesign closes the deployment gap that used to cost thirty percent of accuracy, all validated on a real optical prototype and an on-chip integration case study, and used to demonstrate the first all-optical image segmentation. Read it with its scope. Optical networks are early-stage and still about one percent behind digital ones on simple tasks; the physics is a scalable approximation, not the exact electromagnetic solution; the energy win is for the passive optical compute, while the laser and especially the detector readout still cost power; and the hardware is bulky or needs advanced nanofabrication, with on-chip weights fixed once built. The durable lesson is that as the energy ceiling of digital computing bites, unconventional substrates like optics become worth their strangeness — but only if the tooling exists to design for them, and building that differentiable, hardware-aware, automated design stack is often the harder and more valuable half of the problem.

Background

Why unconventional substrates need a whole software stack to be usable

It helps to see LightRidge against the pressure pushing computing off its digital default. The energy cost of large-scale machine learning has grown unsustainable, and conventional accelerators, however clever, keep bumping against the same efficiency frontier — which is exactly what makes radically different substrates, computing with light, with analog physics, with memory itself, worth the trouble despite their unfamiliarity. But a novel substrate is only as usable as the software that lets people design for it, and historically that tooling lags the physics by years. The retrospective on deep-learning accelerators covered elsewhere on this site frames the same moment from the hardware side, arguing that the era of simply adding digital arithmetic is ending and the field must look to new architectures and substrates to move the efficiency frontier at all. The broad lesson is that a promising physical substrate is not enough on its own: it becomes a real option only when someone builds the differentiable models, the design-space search, the hardware-aware codesign, and the programming interface that let a non-specialist turn an idea into a working device — so that the exotic hardware inherits the mature engineering practices that made digital computing productive.

If you remember only three things

1. A diffractive optical neural network (DONN) computes with light: an input is encoded on a laser beam that diffracts through stacked layers whose per-pixel phase patterns are the learnable weights (diffraction is the connections, phase modulation the weights), and a detector reads the output intensity. Inference runs at the speed of light with near-zero compute energy because the diffractive layers are passive. But the design is done digitally by simulating the optics and training with backpropagation, and existing optical simulators are unfit for it: their physics is not differentiable (so it cannot be trained against), not runtime-optimized (a small five-layer network took three to four days to emulate), and hardware-blind (deployed models lose thirty percent or more accuracy), while designing a DONN at all demands rare expertise across optics, fabrication, and machine learning.
2. LightRidge is the first end-to-end optical ML compilation framework. It models light diffraction with FFT-based scalar approximations (scalable, unlike the exponentially expensive exact Maxwell solution) expressed as differentiable complex-tensor operators (2-D FFTs, inverse FFTs, complex matrix multiplies) so backpropagation trains the phase-pattern weights; it accelerates those kernels — over ninety percent of runtime — on CPU and GPU with complex-tensor libraries and operator fusion; it adds a learned design-space-exploration engine that picks good optical parameters without deep optics expertise, a physics-aware complex-valued regularization that stabilizes training, and a differentiable quantization-aware codesign that maps the model onto the device's real discrete, noisy phase levels to close the deployment gap; and it wraps everything in a simple domain-specific language that generates fabrication files.
3. On a real 532-nanometer optical prototype (spatial light modulators as diffractive layers, a CMOS camera detector), LightRidge's emulation matched the measured light patterns with no manual calibration, and it accelerates emulation up to 6.4 times on CPU and 12 times on GPU over the commercial LightPipes tool, cutting the design cycle from days to hours. The DONN reached 0.98 and 0.89 accuracy on MNIST and FashionMNIST — about one percent below comparable digital networks — at roughly 995 frames per second per watt, about two orders of magnitude more energy-efficient than desktop CPUs and GPUs, and the framework enabled an on-chip integration case study and the first all-optical image segmentation. Its limits: DONNs still trail digital accuracy and are early-stage; the physics is a scalable approximation; only the passive optical compute is free (the readout costs energy); and the hardware is bulky or needs advanced nanofabrication.

Check yourself

Check your understanding

1. How does a diffractive optical neural network compute, and why is it so energy-efficient?
2. Why were existing optical simulators unusable for designing DONNs?
3. What are the main pieces of LightRidge's solution?
4. Which statement best reflects LightRidge's results and scope?

Sources

The paper is “LightRidge: An End-to-end Agile Design Framework for Diffractive Optical Neural Networks” by Yingjie Li, Ruiyang Chen, Minhan Lou, Berardi Sensale-Rodriguez, Weilu Gao, and Cunxi Yu, published at ASPLOS 2024, DOI 10.1145/3623278.3624757. The authors are at the University of Maryland and the University of Utah, and LightRidge is released as an open-source project.

This explainer is written from the full text of the paper (the authors’ arXiv version, 2306.11268, matching the ASPLOS proceedings). Everything above comes from the paper’s own sections: the introduction motivating optical neural networks by the energy cost of digital deep learning and describing DONNs (light diffraction as connections, phase modulation as weights, detector readout); the three challenges (multi-disciplinary expertise and design-space breadth; the algorithm-hardware miscorrelation gap with its reported thirty-percent-plus accuracy degradation; and the high computational cost, three to four days to train a five-layer network); the DONN emulation and training background including the choice of FFT-based scalar diffraction (Rayleigh-Sommerfeld, Fresnel, Fraunhofer approximations) over FDTD because the latter’s fine grid expands exponentially, and the differentiable formulation with the MSE loss on detector-region intensities; the LightRidge framework with its physics kernels, the physics-aware complex-valued regularization (a factor balancing amplitude and phase gradients) and the Gumbel-Softmax-based differentiable quantization-aware codesign, and the DSL; the LightRidge-DSE design-space-exploration engine using a learned analytical model; and the evaluation — the physical prototype (532-nanometer laser, HOLOEYE spatial light modulators, Thorlabs CMOS detector) whose emulation matched measurements, the runtime speedups (up to 6.4 times on CPU and 12 times on GPU over LightPipes, with per-kernel FFT2, iFFT2, and complex-matrix-multiply breakdowns), the accuracy results (0.98 and 0.89 on MNIST and FashionMNIST, about one percent below a comparable MLP and CNN) and energy efficiency (roughly 995 frames per second per watt, about two orders of magnitude better than CPUs and GPUs), the on-chip integration case study, and the advanced architectures (a multi-channel RGB DONN improving Places365 accuracy and the first all-optical image segmentation).