SENSORS LEARN HOW TO THINK

Intelligent machines are increasingly becoming part of our everyday lives. Obvious examples are autonomous driving or modern robot vacuum cleaners in the home. But even where we do not come into direct contact with them, independently working robots are increasingly indispensable: as nimble and tireless helpers in logistics centers or as precise and strong partners in manufacturing environments.

In order to be able to work autonomously in these applications, i.e. as independently as possible without supervision by humans, these robots are peppered with sensors and electronics that enable them to perceive their environment and also cope with unforeseen situations independently. The more complicated the task, the more intelligent and agile the machine needs to be. As a result, more and more and different sensors are being combined, for example for distance measurement, for motion detection or for pressure determination in the event of contact, and the electronics and computer technology for recording and processing the data must be increasingly powerful.

In order to be able to work autonomously in these applications, i.e. as independently as possible without supervision by humans, these robots are packed with sensors and electronics that enable them to perceive their environment and also cope with unforeseen situations independently. The more complicated the task, the more intelligent and agile the machine needs to be. As a result, more and different sensors are being combined, for example for distance measurement, motion detection or pressure determination in the case of touch, and the electronics and computer technology for capturing and processing the data must be ever more powerful.
However, this trend toward mobile supercomputers is accompanied by a significant increase in energy consumption. Particularly in the case of mobile systems, this leads to shorter operating times or ranges and, according to current forecasts, will even push the limits of global energy production in the next few decades.

To counteract this escalation, the Fraunhofer researchers involved in NeurOSmart are relying on decentralized intelligence that is tailored to the respective sensor. The project has a duration of four years and a financial volume of eight million euros. The so-called neuromorphic electronics to be developed are modeled on the human brain, which is very energy-efficient when making decisions despite its enormous computing power.




»This kind of data processing, i.e. thinking, is realized by a novel analog computer memory technology that is also capable of performing computational operations when data is newly acquired in the system,« explains ISIT scientist and project leader Dr. Michael Mensing: "In practice, this is used to recognize objects and their behavior accurately and in real time." Until now, this mode of operation has required several separately developed components in computers and particularly energy-intensive communication between them.

The advantages of the new approach are underlined by the parallel development of particularly small and efficient models for object recognition and classification, which are specially adapted to the sensor, the new possibilities of directly integrated electronics and their applications. The result is fast response time, increased data protection and significant energy savings compared to the current trend of off-practice or cloud-based solutions that prefer to rely on ever larger, more energy-intensive models.

Over the next four years, this approach will be combined for the first time with a complex LiDAR (Light Detection And Ranging) system developed at Fraunhofer and tested in an application-oriented environment. This sensor system is a crucial component of autonomously operating systems, as it uses detailed distance information to recognize its surroundings even in bad weather and over a wide range of distances. As a first sample of the novel sensors, they will be integrated in the next few years into robotic systems that support their human colleagues in manufacturing environments, for example by moving heavy loads or reaching components.

www.ipms.fraunhofer.de