Social robots are autonomous machines designed to interact with humans using social cues such as voice, gestures, and facial expressions. They have remained uncommon in public spaces, despite expectations raised more than 25 years ago. Apart from basic robotic skills like safe navigation and obstacle avoidance, they need to facilitate natural verbal and nonverbal communication with multiple people in dynamic, noisy environments. Deployments in museums, airports, libraries, shopping malls, bars, and hospitals have reported positive results. However, many studies relied on a Wizard-of-Oz setup where researchers controlled the navigation and dialogue. Systems that operated independently were often stationary, limited to closed domains, or tuned for single-user interaction. These limitations did not align with the demands of public spaces.
The EU H2020 SPRING project addressed these challenges by developing software that allows social robots to operate effectively in complex, unstructured public environments. The team comprised eight groups, including a gerontology hospital with research facilities. The software, including the audio pipeline, was installed on the humanoid ARI robot from PAL Robotics, which stands 1.65 meters tall.
In this talk, Prof. Gannot will discuss the audio pipeline developed in SPRING and its role in supporting operations in public spaces. The pipeline includes noise reduction, direction-of-arrival estimation, multi-microphone beamforming, single-channel concurrent speaker separation, speaker extraction with enrollment, speaker identification, audio-visual concurrent speaker detection, emotion recognition from audio and video, and voice generation from video.
The talk will conclude with results from real deployments, demonstrating perceived usefulness and acceptance when robustness and adaptability are maintained.
Project page: https://spring-h2020.eu/
