New Neural Tech Could Power Insect-Sized Intelligent Flying Robots

Researchers at Delft University of Technology have developed a groundbreaking neuromorphic vision-to-control system that enables autonomous drone flights. This new technology, which mimics the brain’s biological processing, could pave the way for creating intelligent flying robots as small as insects.

The Breakthrough: Neuromorphic Vision-to-Control Mechanism

The innovative neuromorphic system developed by the research team involves a spiking neural network (SNN) that processes data from event-based cameras. This network, consisting of 28,800 neurons, is designed to map incoming raw data to estimate the 3D motion of the drone within its environment, reports Interesting, Engineering. Unlike traditional vision-based drone flight systems that use artificial neural networks, this new approach is significantly more compact and energy-efficient.

“The neuromorphic processor consumes only 0.94 watts of idle power and an additional 7-12 milliwatts while running the network,” researchers stated. This efficient use of power is a crucial factor for small-scale autonomous drones.

SNN-Driven Drone Flight

Biological systems are known for their low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware, which includes event-based vision and spiking neural networks, aims to replicate these characteristics. However, current implementations are limited by the small network size of existing neuromorphic processors and the complexities involved in training spiking neural networks.

To showcase the potential of neuromorphic hardware, the researchers utilized a complete neuromorphic vision-to-control pipeline to operate a drone. The SNN, trained to process raw event camera data, enabled the drone to conduct autonomous vision-based ego-motion estimation and control at a frequency of about 200 Hz. This allowed the drone to hover, land, and maneuver sideways autonomously.

Efficient Neuromorphic Flight

The vision-to-control pipeline was implemented on Intel’s Loihi neuromorphic processor and integrated into a small flying robot. This setup included an event camera, a neuromorphic processor, a single-board computer, and a flight controller. Weighing 994 grams and with a diameter of 35 cm, the drone demonstrated efficient and smooth height reduction during landing experiments.

The system assumed a static, texture-rich, flat surface viewed by a DVS 240 camera to meet computational constraints. The vision processing pipeline reduced spatial resolution by focusing on corner regions of interest (ROIs), processing up to 90 events per ROI with a small SNN. This SNN, containing 7,200 neurons and 506,400 synapses across five layers, estimated optical flow for each ROI.

Trained using self-supervised contrast maximization for vision and an evolutionary algorithm for control, the system successfully navigated in various lighting conditions. Its energy consumption was also notably efficient compared to traditional GPU solutions.

Future Implications

The research team’s success with the neuromorphic vision-to-control pipeline demonstrates the potential for developing intelligent robots on a much smaller scale. These findings, published in the journal Science Robotics, could lead to further advancements in neuromorphic sensing and processing.

The drone’s ability to accurately control its ego-motion and perform complex maneuvers autonomously highlights the promise of this technology for future applications.

DroneXL’s Take

This breakthrough in neuromorphic technology represents a significant step forward in the field of robotics. By mimicking the brain’s processing capabilities, researchers are developing more efficient and compact systems that could revolutionize the use of drones and other autonomous devices. As this technology continues to evolve, we can expect to see increasingly intelligent and capable robots that are both energy-efficient and effective in a variety of environments.