German chips that compute AI at the speed of light

English translation of the original German article, published under licence from the Frankfurter Allgemeine Zeitung. The German original is available via the language switch (DE).

Some things are simply easier with light than with electricity. Finding your way in a dark room, for instance. But also performing certain calculations. While every conventional computing system has worked with electricity for decades, the Stuttgart-based company Q.ant is betting on light.

Since the 1940s, virtually all computers have been based on electricity. They perform their calculations with billions of tiny electronic switches that are either in the “on” or the “off” position. This binary on or off, zero or one, is therefore the foundation of every modern computer. But it has a few weaknesses.

A computer based on zeros and ones cannot carry out many complex calculations directly. Instead, the machine has to take detours. And the more complex the calculation, the longer the detour. The problem resembles trying to draw a curve with a ruler. Instead of simply drawing freehand, the curve has to be broken down into a great many small strokes that are drawn one after another. That creates extra work to coordinate the individual steps and takes a lot of time.

The path to enlightenment

This is exactly the problem Q.ant’s processors address. Light waves can carry far more information than just zeros and ones. Through the distances between the peaks and troughs of the wave, its height and its direction, as well as the on and off of the light, a lot of information is transported at once, in the fastest way possible. Light waves like these reach each of us countless times a day as carriers of data, through the fibre-optic cables that move global internet traffic. No other medium can move more data or do so faster than light.

When two light waves meet, however, even more happens. Together, the two waves form a new wave, combine their forces and thereby perform a natural calculation. Literally at the speed of light and without any detour. These are exactly the processes Q.ant controls on its native processing cards. The card carries a chip that guides light through tiny, fixed paths. Small components split the light and bring it back together. Electrical signals control how the light waves are shifted against each other. When they are combined, they amplify or weaken each other accordingly. This precisely controlled behaviour can be used for calculations.

That is where the name Native Processing Unit (NPU) comes from, explains Michael Förtsch, founder and CEO of Q.ant. The photonic operations take place natively, that is, instantly and with the real physical quantities. That is more efficient than computing with zeros and ones, as the digital computer does. What 1,200 switches have to orchestrate there in a complex interplay is done by a single component in photonic computing.

The Nvidia strategy

The concept of the NPU becomes easier to understand in comparison with a GPU (graphics processing unit), such as those developed by market leader Nvidia. Its cards are currently the most sought-after resource for AI applications. Just like Q.ant’s cards, which are only now coming onto the market, they extend the functionality of an existing computer. Both are connected to conventional computers, but massively accelerate their computing speed for certain tasks.

Nvidia uses this above all to execute many simple computing steps at the same time. In the early days of its GPU cards, they were mainly used for video games. For other developers, it was very demanding and hardly worth the effort to design their calculations for Nvidia GPUs. Nvidia therefore invested in developing its own programming platform called Cuda. It is designed to do the difficult work in the background: bringing the calculations into a GPU-compatible format. Nvidia thus relieved developers of the laborious work of optimising their calculations for the GPUs. Instead, developers could simply give rough instructions and the Cuda programming platform did the rest. This very simplicity made Nvidia the market leader it is today.

For its novel photonic processors, it therefore makes sense for Q.ant to pursue a similar strategy. In September, the company plans to release a programming toolkit intended to bring photonic computing closer to developers. It offers an entry point for gaining first experience with photonic calculations, while doing a lot of preparatory work in the background to make it easy to use.

Q.ant’s vice president for software, Utz Bacher, previously worked at chipmaker IBM for many years and understands what matters. To unlock the full performance of photonic computing, the preparation of the photonic processor and the applications has to go hand in hand. His team at Q.ant has therefore done a great deal of groundwork to make linking existing applications to the new computing model seamless. This is meant to release the full efficiency of the photonic processor.

The same AI performance in a compact package

This efficiency can also come into play for AI applications. The photonic way of computing could also help make AI models smaller. What matters here are the parameters, the numerical values a model consists of and adjusts during training. They determine how a model processes the information it takes in. The more parameters a model has, the more memory it needs. At the same time, performance rises with the number of parameters. Precisely because of this demand for powerful models, the global supply chain for computer memory is currently experiencing a bottleneck and cannot deliver enough. Transferring ever larger models to the GPUs and running them there costs more and more time and energy. That in turn drives costs up further.

Q.ant’s hope is therefore to get more model performance out of every parameter with suitable mathematical functions. On the photonic NPU, certain computing steps can be executed directly that a conventional computer can only laboriously reproduce in many small individual steps. That is the difference between the freehand curve and the many small strokes with the ruler. With these non-linear computing steps, an AI model can represent complex relationships with fewer parameters. That would mean smaller models with consistently good performance.

So far, however, developers of AI models have largely done without this and relied on simpler computing methods that conventional chips execute more efficiently. They compensate for the lower capability with more parameters. More memory has simply been the lesser evil so far. If the photonic way of computing makes the more elaborate computing steps efficient, however, smaller models also pay off economically. That would ease AI’s hunger for resources. For this, Q.ant provides developers with exactly that toolkit, with which AI models can be tailored to the NPU’s way of computing.

The field test: computing on Q.ant’s chips

In our own (non-scientific) experiments with the toolkit on a server with an NPU provided by Q.ant, this works extremely well. A small language model that we train with the NPU’s way of computing is 17.2 percent smaller than conventional models in our tests, at the same performance. Another small AI model, which makes the best possible use of the wave-like computing of the photonic hardware to reconstruct a signal, does even better. With only 48 parameters and the complex way of computing, it achieves better results than a conventional model with more than 17,000 parameters.

Our experiments match Q.ant’s results. According to the company, an example model optimised for the photonic hardware with only 10,000 parameters delivers better results than the conventional reference model with around 20,000 parameters. The number of computing operations required is also lower. With the programming toolkit, these models can now show their full strength. So that users can start experimenting right away, the toolkit also contains a kind of simulator that lets them play through how efficient any calculation would be on the photonic hardware.

A look at the data center of the future

The programming toolkit also prepares the broader release of Q.ant’s NPUs. In the coming months, the photonic processors will become available to the wider public through the partnership with cloud provider Ionos. Then anyone should be able to compute with light. The programming toolkit thus also lays the groundwork for developers and companies to use the new technology in future.

The data center of the future will connect different specialised computers, says Förtsch. CPUs, GPUs and photonic NPUs would share the work, and in the longer term quantum processors could join them. Each of these technologies should take on the tasks it is particularly well suited for. The hope behind this: if calculations are executed on the most efficient hardware in each case, energy consumption and operating costs could fall significantly. Q.ant’s NPUs, for example, produce considerably less waste heat than GPUs.

Either way, the future of computing looks bright.