I’ve been watching the AI hardware space long enough to know that we’re currently living in a state of managed crisis. We’re building massive data centers, burning through megawatts of power, and acting like the current GPU-centric model can scale forever. It can’t. The physics of moving electrons through copper and silicon is hitting a wall, and that’s exactly why the industry is suddenly looking at light.
Optical computing isn’t new. The idea of using photons instead of electrons to do math has been kicking around for 70 years. But it’s finally moving out of academic white papers and into the market. We’ve seen the sector’s valuation jump from $400 million to over $2.2 billion in just two years. As a founder, you have to ask: is this just another hype cycle, or is the underlying architecture of AI about to shift?
The Energy Wall
The problem is simple: AI inference—the actual act of running a model to answer a user—is a power hog. Specifically, it relies on one mathematical operation called matrix-vector multiplication. Today’s GPUs do this by shuffling electrons, which generates heat and consumes massive amounts of energy. As models get bigger, the energy cost scales exponentially.
Optics changes the math. In theory, once you encode a matrix into a light-modulating device, the actual multiplication happens passively as light passes through it. The energy cost doesn't scale with the size of the matrix in the same way it does with silicon. We’re talking about a potential 100,000x reduction in energy per operation. Even if you account for the overhead of converting data from digital to light and back, startups are still targeting a 10x to 80x efficiency gain over Nvidia’s best hardware. For anyone building infrastructure, those numbers are impossible to ignore.
Free-Space vs. On-Chip
The builders in this space are split into two camps. The first is the "free-space" crowd, like Lumai and Neurophos. They let light travel through air or specialized membranes, using lenses to manipulate it. It’s an elegant, 3D approach that offers the best theoretical scaling. Lumai is focusing on the "prefill" phase of inference—the heavy lifting where the query is first processed—claiming they can do it at a tenth of the energy a GPU requires.
The second camp is integrated photonics. Companies like Q.ANT and Lightelligence are keeping the light on the chip using waveguides. They’re trading some of that 3D efficiency for something more important to founders: manufacturability. They want to use existing semiconductor fabrication lines. Their argument is that if you can’t mass-produce it, it doesn’t matter how fast it is. These chips are already finding their way into supercomputing centers in Europe, proving that the hardware is real, even if it’s not yet in every rack.
The Amdahl Trap
Here is the skeptical founder perspective: Amdahl’s Law. If you accelerate 90% of a process but the other 10% is stuck on old tech, your total speedup is capped at 10x. Optical computing is brilliant at linear algebra (the matrix stuff), but it struggles with the nonlinear parts of AI models. You still need traditional electronics to handle the logic and the memory shuffling.
There’s also the "I/O tax." Converting digital bits to light pulses and back again costs energy. If your vector size is too small—say, under 1,000 elements—the energy you save on the math is eaten up by the conversion process. This means optical computing isn't a silver bullet for small, nimble models; it’s a specialized tool for the massive, agentic workloads that are currently breaking the grid.
Thinking Beyond the Matrix
Some of the most interesting work isn’t trying to mimic GPUs at all. Companies like LightSolver are using lasers to solve differential equations by letting light reach a "steady state" in a cavity. They aren't just doing math faster; they're doing it differently. They’ve shown they can solve certain complex equations thousands of times faster than a GPU by leveraging the natural behavior of light waves.
Then there’s Optalysys, which is taking a "computing in transit" approach. Since data centers already use fiber optics to move data between racks, they’re putting the processing power directly into the networking path. Why wait for data to reach a GPU to process it if you can do the math while the photons are already moving through the cable? It’s a brilliant way to tackle the memory-bound bottlenecks of modern AI.
What This Means for Builders
Don't expect to swap your H100s for optical processors next month. The software ecosystem is still heavily optimized for CUDA and silicon. But the move toward heterogeneous computing is inevitable. The future data center isn't one giant slab of GPUs; it's a mix of CPUs for logic, GPUs for general acceleration, and photonic units for the heavy-duty math and networking.
For founders, the takeaway is clear: the energy ceiling of AI is driving a radical reinvention of the stack. Keep an eye on the prefill and networking startups. If they can solve the I/O bottleneck, the economics of running large-scale AI could drop by an order of magnitude, opening up applications that are currently too expensive to even consider.
The industry is betting that light can save us from a power crisis, but the real winner will be the architecture that plays nice with the silicon we already have.
Read the original at IEEE Spectrum →