果冻传媒

Have you ever thought about how wonderful your brain really is? It鈥檚 a powerful computing machine, but it鈥檚 also fast, dynamic, adaptable, and very energy efficient. You should feel quite lucky!

The combination of these attributes has inspired researchers at 果冻传媒 such as Yoeri van de Burgt to mimic how the brain works in technologies where learning is important such as artificial intelligence (AI) systems in transport, communication, and healthcare.

The neural link

鈥淎t the heart of such AI systems you鈥檒l likely find a neural network,鈥� says Van de Burgt 鈥� associate professor at the Department of Mechanical Engineering at 果冻传媒.

Neural networks are brain-inspired computer software models. In the human brain, neurons talk to other neurons via synapses, and the more two neurons talk to each other, the stronger the connection between them becomes. In neural network models 鈥� which are made of nodes 鈥� the strength of a connection between any two nodes is given by a number called the weight.

鈥淣eural networks can help solve complex problems with large amounts of data, but as the networks get larger, they bring increasing energy costs and hardware limitations,鈥� says Van de Burgt. 鈥淏ut there is a promising hardware-based alternative 鈥� neuromorphic chips.鈥�

The neuromorphic catch

Like neural networks, neuromorphic chips are inspired by how the brain works but the imitation is taken to a whole new level. In the brain, when the electrical charge in a neuron changes it can then fire and send electrical charges to connected neurons. Neuromorphic chips replicate this process.

鈥淚n a neuromorphic chip there are memristors (which is short for memory resistors). These are circuit devices that can 鈥榬emember鈥� how much electrical charge has flowed through them in the past,鈥� says Van de Burgt. 鈥淎nd this is exactly what is required for a device modeled on how brain neurons store information and talk to each other.鈥�

But there鈥檚 a neuromorphic catch 鈥� and it relates to the two ways that people use to train hardware based on neuromorphic chips. In the first way, the training is done on a computer and the weights from the network are mapped to the chip hardware. The alternative is to do the training in-situ or in the hardware, but current devices need to be programmed one by one and then error-checked. This is required because most memristors are stochastic and it鈥檚 impossible to update the device without checking it.

鈥淭hese approaches are costly in terms of time, energy, and computing resources. To really exploit the energy-efficiency of neuromorphic chips, the training needs to be done directly on the neuromorphic chips,鈥� says Van de Burgt.

The masterful proposal

And this is exactly what Van de Burgt and his collaborators at 果冻传媒 have achieved and published in a new paper in Science Advances. 鈥淭his was a real team effort, and all initiated by co-first authors Tim Stevens and Eveline van Doremaele,鈥� Van de Burgt says with pride.

The story of the research can be traced back to the master鈥檚 journey of Tim Stevens. 鈥淒uring my master鈥檚 research, I became interested in this topic. We have shown that it鈥檚 possible to carry out training on hardware only. There鈥檚 no need to transfer a trained model to the chip, and this could all lead to more efficient chips for AI applications,鈥� says Stevens.

Van de Burgt, Stevens, and Van Doremaele 鈥� who defended her PhD thesis in 2023 on neuromorphic chips 鈥� needed a little help along the way with the design of the hardware. So, they turned to Marco Fattori from the Department of Electrical Engineering.

鈥淢y group helped with aspects related to circuit design of the chip,鈥� says Fattori. 鈥淚t was great to work on this multi-disciplinary project where those building the chips get to work with those working on software aspects.鈥�

For Van de Burgt, the project also showed that great ideas can come from any rung on the academic ladder. 鈥淭im saw the potential for using the properties of our devices to a much greater extent during his master鈥檚 research. There鈥檚 a lesson to be learnt here for all projects.鈥�

The two-layer training

For the researchers, the main challenge was to integrate the key components needed for on-chip training on a single neuromorphic chip. 鈥淎 major task to solve was the inclusion of the electrochemical random-access memory (EC-RAM) components for example,鈥� says Van de Burgt. 鈥淭hese are the components that mimic the electrical charge storing and firing attributed to neurons in the brain.鈥�

The researchers fabricated a two-layer neural network based on EC-RAM components made from organic materials and tested the hardware with an evolution of the widely used training algorithm backpropagation with gradient descent. 鈥淭he conventional algorithm is frequently used to improve the accuracy of neural networks, but this is not compatible with our hardware, so we came up with our own version,鈥� says Stevens.

What鈥檚 more, with AI in many fields quickly becoming an unsustainable drain of energy resources, the opportunity to train neural networks on hardware components for a fraction of the energy cost is a tempting possibility for many applications 鈥� ranging from ChatGPT to weather forecasting.

The future need

While the researchers have demonstrated that the new training approach works, the next logical step is to go bigger, bolder, and better.

鈥淲e have shown that this works for a small two-layer network,鈥� says van de Burgt. 鈥淣ext, we鈥檇 like to involve industry and other big research labs so that we can build much larger networks of hardware devices and test them with real-life data problems.鈥�

This next step would allow the researchers to demonstrate that these systems are very efficient in training, as well as running useful neural networks and AI systems. 鈥淲e鈥檇 like to apply this technology in several practical cases,鈥� says Van de Burgt. 鈥淢y dream is for such technologies to become the norm in AI applications in the future.鈥�

Full paper details

鈥溾��, Eveline R. W. van Doremaele, Tim Stevens, Stijn Ringeling, Simone Spolaor, Marco Fattori, and Yoeri van de Burgt, Science Advances, (2024).

Eveline R. W. van Doremaele and Tim Stevens contributed equally to the research and are both considered as first authors of the paper.

Tim Stevens is currently working as a mechanical engineer at , a company co-founded by Marco Fattori.