Using cutting-edge AI technologies, we are exploring the untapped potential of environmental sounds for applications in the fields of bioacoustics, noise monitoring, logistics and traffic monitoring, as well as security surveillance at construction sites and public events.
Sounds and noises surround us everywhere in our daily lives – as disturbing noise, as the soothing rustle of leaves or as the warning sound of sirens on the street. Humans possess not only the ability to distinguish between important and unimportant sounds but also to derive crucial information about their surroundings through sound interpretation based on their experiences.
"Machine listening" is a subfield of artificial intelligence that aims to replicate this human capability by automatically capturing and interpreting information from environmental sounds. This involves combining signal processing techniques and machine learning and developing algorithms for the analysis, source separation, and classification of music, speech, and environmental sounds. Source separation allows for the decomposition of complex acoustic scenes into their components, i.e., individual sound sources, while classification identifies sounds and assigns them to predefined sound sources or classes.
The developed technologies and solutions find applications in various areas:
General challenges in the analysis of environmental sounds include robust recognition of individual sounds despite high acoustic variability within and between different sound classes. In simple terms, the algorithm must be able to recognize a Dachshund labrador and a Great Dane German shepherd as dogs based on their barking. The strong overlap of multiple static and moving sound sources in complex scenarios further complicates reliable recognition.
When deploying AI algorithms in acoustic sensors, various microphone characteristics and room acoustics effects such as reverberation and reflections can make classification challenging.
Our research also addresses the question of how compact AI models can be trained with minimal training data for deployment on resource-constrained hardware. This is necessary because many deployment locations often lack sufficient or consistent power supply, and long-term analyses may span several days or weeks. Therefore, the models must not be overly large and complex to function for real-time analysis on the devices.
Our aim is to use the technology for practically relevant issues such as the measurement and investigation of noise pollution, bio- and eco-acoustics as well as construction site and logistics monitoring.
The basic research in the areas of efficient AI models, explainable AI, training with little data and domain adaptation also has the potential to be used across domains in other audio research areas such as speech processing or music signal processing.
Additionally, we conduct research in the context of listening tests and citizen science applications involving participants to explore the subjective perception of noise and other perceptual sound attributes. The aim is to gain a better understanding of which sound sources in everyday situations have a particularly disruptive impact on our perception of noise (and, by extension, our health).
The following methods and procedures are used to analyze environmental noise:
Fraunhofer Institute for Digital Media Technology IDMT