Environmental Sound Analysis

AI-based analysis of complex acoustic scenes and sounds

Using cutting-edge AI technologies, we are exploring the untapped potential of environmental sounds for applications in the fields of bioacoustics, noise monitoring, logistics and traffic monitoring, as well as security surveillance at construction sites and public events.

News and upcoming events

 

Journal Article

Human and Machine Performance in Counting Sound Classes in Single-Channel Soundscapes

Der Artikel ist in der Dezember-Ausgabe Volume 71 Number 12 des Journal of the AES (JAES) erschienen.

 

Call for Papers

Inter-Noise 2024

Our audio expert Jakob Abeßer  will co-organize the technical session "Machine Learning for Acoustic Scene Understanding". Submit your abstract by February 9, 2024.

 

New project / 15.5.2023

Open Innovation Lab

Noise monitoring field test project has started as part of the City of Gelsenkirchen's "Open Innovation Lab".

Capturing information from environmental sounds

Sounds and noises surround us everywhere in our daily lives – as disturbing noise, as the soothing rustle of leaves or as the warning sound of sirens on the street. Humans possess not only the ability to distinguish between important and unimportant sounds but also to derive crucial information about their surroundings through sound interpretation based on their experiences.

"Machine listening" is a subfield of artificial intelligence that aims to replicate this human capability by automatically capturing and interpreting information from environmental sounds. This involves combining signal processing techniques and machine learning and developing algorithms for the analysis, source separation, and classification of music, speech, and environmental sounds. Source separation allows for the decomposition of complex acoustic scenes into their components, i.e., individual sound sources, while classification identifies sounds and assigns them to predefined sound sources or classes.

 

The developed technologies and solutions find applications in various areas:

  • Bioacoustics: Identifying animal species, studying behavioral patterns, or monitoring environmental impacts based on acoustic characteristics
  • Noise monitoring: recording noise data, identifying noise sources and planning noise protection measures
  • Logistics and traffic monitoring: Counting and classifying vehicles, analyzing traffic flows to improve emergency response planning, and implement traffic management measures
  • Safety surveillance (construction sites, public events): Detecting hazardous situations, vandalism, or break-ins acoustically

Robust recognition, energy-efficient implementation

General challenges in the analysis of environmental sounds include robust recognition of individual sounds despite high acoustic variability within and between different sound classes. In simple terms, the algorithm must be able to recognize a Dachshund labrador and a Great Dane German shepherd as dogs based on their barking. The strong overlap of multiple static and moving sound sources in complex scenarios further complicates reliable recognition.

When deploying AI algorithms in acoustic sensors, various microphone characteristics and room acoustics effects such as reverberation and reflections can make classification challenging.

Our research also addresses the question of how compact AI models can be trained with minimal training data for deployment on resource-constrained hardware. This is necessary because many deployment locations often lack sufficient or consistent power supply, and long-term analyses may span several days or weeks. Therefore, the models must not be overly large and complex to function for real-time analysis on the devices.

Learning to understand sounds

Our aim is to use the technology for practically relevant issues such as the measurement and investigation of noise pollution, bio- and eco-acoustics as well as construction site and logistics monitoring.

The basic research in the areas of efficient AI models, explainable AI, training with little data and domain adaptation also has the potential to be used across domains in other audio research areas such as speech processing or music signal processing.

Additionally, we conduct research in the context of listening tests and citizen science applications involving participants to explore the subjective perception of noise and other perceptual sound attributes. The aim is to gain a better understanding of which sound sources in everyday situations have a particularly disruptive impact on our perception of noise (and, by extension, our health).

How we proceed

The following methods and procedures are used to analyze environmental noise:

  • Audio signal processing
  • Deep learning
  • Perception of sound signals

 

Research project

"StadtLärm" (CityNoise)

Development of a noise monitoring system to support urban noise protection tasks

 

Field test project

Open Innovation Lab

Noise monitoring field test project as part of the City of Gelsenkirchen's "Open Innovation Lab"

 

Research project

BioMonitor4CAP

Acoustic animal species recognition and classification for improved biodiversity monitoring in agriculture

 

Research project

Construction-sAIt

Multi-modal AI-driven technologies for automatic construction site monitoring

 

Research project

ISAD 2

Development of explainable and comprehensible deep-learning models to enable a better understanding of the structural and acoustic properties of sound sources (music or environmental sounds)

 

Research project

vera.ai

Sound event detection and acoustic scene recognition for the development of trustworthy AI solutions for detecting advanced disinformation techniques in the media sector

 

Research project

news-polygraph

Sound event detection and acoustic landmark detection for the development of a multi-modal, crowd-supported technology platform for disinformation analysis

 

Research project

NeuroSensEar

Sound event detection and acoustic scene recognition for bio-inspired acoustic sensor technology for highly efficient hearing aids

 

Research project

Sound Surv:AI:llance

Acoustic Burglary Monitoring

Jahr
Year
Titel/Autor:in
Title/Author
Publikationstyp
Publication Type
2023 Investigations on the Implementation of an Acoustic Rain Sensor System
Hock, Kevin; Götz, Julian; Seideneck, Mario; Sladeczek, Christoph
Konferenzbeitrag
Conference Paper
2023 How Robust are Audio Embeddings for Polyphonic Sound Event Tagging?
Abeßer, Jakob; Grollmisch, Sascha; Müller, Meinard
Zeitschriftenaufsatz
Journal Article
2022 Classifying Sounds in Polyphonic Urban Sound Scenes
Abeßer, Jakob
Zeitschriftenaufsatz
Journal Article
2022 Analyzing Bird and Bat Activity in Agricultural Environments using AI-driven Audio Monitoring
Abeßer, Jakob; Wang, Xiaoyi; Bänsch, Svenja; Scherber, Christoph; Lukashevich, Hanna
Konferenzbeitrag
Conference Paper
2022 Construction-sAIt: Multi-modal AI-driven technologies for construction site monitoring
Abeßer, Jakob; Loos, Alexander; Sharma, Prachi
Konferenzbeitrag
Conference Paper
2021 DESED-FL and URBAN-FL: Federated Learning Datasets for Sound Event Detection
Johnson, David S.; Lorenz, Wolfgang; Taenzer, Michael; Mimilakis, Stylianos Ioannis; Grollmisch, Sascha; Abeßer, Jakob; Lukashevich, Hanna
Konferenzbeitrag
Conference Paper
2021 Investigating the influence of microphone mismatch for acoustic traffic monitoring
Gourishetti, Saichand; Abeßer, Jakob; Grollmisch, Sascha; Kátai, András; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2021 DESED-FL and URBAN-FL: Federated Learning Datasets for Sound Event Detection
Johnson, David S.; Lorenz, Wolfgang; Taenzer, Michael; Grollmisch, Sascha; Abeßer, Jakob; Lukashevich, Hanna; Mimilakis, Stylianos
Paper
2021 IDMT-Traffic: An Open Benchmark Dataset for Acoustic Traffic Monitoring Research
Abeßer, Jakob; Gourishetti, Saichand; Kátai, András; Clauß, Tobias; Sharma, Prachi; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2021 Improving Semi-Supervised Learning for Audio Classification with FixMatch
Grollmisch, Sascha; Cano, Estefanía
Zeitschriftenaufsatz
Journal Article
2020 Sound Event Detection with Depthwise Separable and Dilated Convolutions
Drossos, Konstantinos; Mimilakis, Stylianos I.; Gharib, Shayan; Li, Yanxiong; Virtanen, Tuomas
Paper
2020 Identifikation urbaner Geräuschquellen mittels maschineller Lernverfahren
Clauß, T.; Abeßer, Jakob
Zeitschriftenaufsatz
Journal Article
2020 Analyzing the potential of pre-trained embeddings for audio classification tasks
Grollmisch, Sascha; Kehling, Christian; Taenzer, Michael; Cano, E.
Konferenzbeitrag
Conference Paper
2020 A Review of Deep Learning Based Methods for Acoustic Scene Classification
Abeßer, Jakob
Zeitschriftenaufsatz
Journal Article
2019 Smart Solutions to Cope with Urban Noise Pollution
Abeßer, Jakob; Kepplinger, Sara
Zeitschriftenaufsatz
Journal Article
2018 Stadtlärm - a distributed system for noise level measurement and noise source identification in a smart city environment
Clauß, Tobias; Abeßer, Jakob; Lukashevich, Hanna; Gräfe, Robert; Häuser, Franz; Kühn, Christian; Sporer, Thomas
Konferenzbeitrag
Conference Paper
2017 Acoustic scene classification by combining autoencoder-based dimensionality reduction and convolutional neural networks
Abeßer, Jakob; Gräfe, Robert; Lukashevich, Hanna; Mimilakis, Stylianos-Ioannis
Konferenzbeitrag
Conference Paper
2015 Quantifying auditory perception: Underlying dimensions of pleasantness and unpleasantness
Liebetrau, Judith; Sporer, Thomas; Becker, Marius; Duong, Thanh Phong; Ebert, Andreas; Härtig, Martin; Heidrich, Phillip; Kirner, Jakob; Rehling, Oliver; Vöst, Dominik; Walter, Roberto; Zierenner, Michael; Clauß, Tobias
Konferenzbeitrag
Conference Paper
2012 Sound field reproduction analysis in a car cabin based on microphone array measurements
Nowak, Johannes; Strauß, Michael
Konferenzbeitrag
Conference Paper
2010 Active minimization of periodic sound inside a vibro-acoustic rectangular enclosure using finite element model
Mohamady, S.; Montazeri, A.; Ahmad, R.K.R.
Konferenzbeitrag
Conference Paper
Diese Liste ist ein Auszug aus der Publikationsplattform Fraunhofer-Publica

This list has been generated from the publication platform Fraunhofer-Publica