Industrial Sound Analysis

AI-based acoustic monitoring in production

The research activities in the field Industrial Sound Analysis are focused on the development of AI-based methods for automated acoustic monitoring of products and processes for use in in-line and end-of-line quality control, process monitoring and predictive maintenance.

News

Industrial Sound Analysis

LinkedIn post / 23.4.2024

We were there!

Control Messe, Stuttgart

The potential of acoustic quality assessment in production

How do you use acoustic signals to assess quality?

Your car is a good example to show the potential of quality assessment based on acoustic signals. You know the familiar driving sound caused by the engine, tyre wear and airflow. But what do you feel when this noise changes? A rattling or grinding noise might worry you and make you wonder if your car is still safe to drive. You probably react to this unknown and unexpected condition by driving to the workshop or at least stopping your car. It's no different in industry. We analyse the sound of your product or production process and let you know when something is not working as expected.

In industry, wherever there is movement, there are audible sounds that indicate the quality of products or processes. At the Fraunhofer IDMT in Ilmenau, Germany, we develop AI-based methods for sound analysis - especially for the analysis of industrial sounds - and thus create innovative approaches for automated acoustic monitoring (amo) of products and processes. amo can be used along the entire value chain for quality assessment and offers added value where, for example, optical monitoring methods reach their limits.

Our team of scientists from the fields of data science, data analysis, software development and project management is therefore researching AI-based solutions for audio signal analysis. The innovation at amo is that the measurement data is processed without any connection to an external cloud, so our solutions can be used locally within the company or directly on the machine.

Challenges in sensor-based machine monitoring

Machine and plant manufacturers are faced with the challenge of providing their customers with comprehensive sensor-based machine monitoring. However, existing methods cannot solve all the problems that arise, such as unexpected downtime, continuous monitoring of the machine or evaluation of existing machine data. In addition, there is a shortage of specialists who know their machines so well that they can immediately identify faults by sound and take action if necessary.

AI-based acoustic monitoring

Components of a pilot project

  • Analysis and interpretation of the soundscape in the production environment
  • Create a customized setup to record the sounds
  • Systematically record acoustic signals
  • Select and apply analysis methods
  • Consideration of privacy and security issues
  • Evaluate the feasibility of acoustic monitoring for specific applications

Overview

Research field "amo”

Research project

DIAMOSS-I

Development of an Intelligent, Automonitored Sound Sensor System for Harsh Industrial Conditions

 

Research project

OptiStrick

Development of an AI-supported inline quality assurance system for optimizing highly flexible knitting technologies

 

Research project

RapidKI

Intelligent inline control for green laser ablation processes

Research project

HybridDigital

Digitalization for efficient process selection and design of hybrid structures based on experimental and synthetic data

Research project

ML-S-LeAF

Development of machine learning algorithms based on virtual sound data for lightweight construction for quality assurance in additive manufacturing.

 

Research project

AKoS

Acoustic inspection of weld seams of safety-critical components as part of quality assurance

Research project

e-LAS+

Multimodal quality assurance for production of electricity storage in safety-critical systems

Research project

ExtrudEAR

Mapping auditory perception and human expert knowledge to extruder process control

Research project

SEC-Learn

Sensor Edge Cloud for federated learning

 

ZIM-Netzwerk »AkuPro«

Akustische Analysen in Produktionssystemen

Werden Sie Netzwerkpartner!

Application meets partner

Networks (German)

Leistungszentrum InSignA – Intelligent Signal Analysis and Assistance Systems

AkuPro – Acoustic Analyses in Production Systems (ZIM Network)

IMAMF Innovationsform – AI-based acoustic monitoring of manufacturing processes

 

 

Leistungszentrum

InSignA

Examples of successfully completed feasibility studies

Automatic detection of compressed air leaks

Reliably and automatically detect audible leaks thanks to airborne sound analysis and machine learning methods.

Compressed air is an indispensable resource for many German industrial and trade companies for the operation of machines and plants. At the same time, it also represents a high cost factor on the electricity bill. On average, companies waste 30 percent of the energy generated due to the unnoticed escape of expensive compressed air. To detect such leaks, they rely, among other things, on ultrasound testing methods and on the mere hearing of trained personnel.

Fraunhofer IDMT has now tested in an experiment whether this "hearing" can be automated and reproduced using microphones in combination with machine learning methods in order to develop a reliable system for leakage detection. First results show that this is generally possible.

Automatic detection and differentiation of electric engines

The engine demo shows the performance of acoustic monitoring in industrial applications. The detection of three different sounding electric engines was chosen as an application example.

In this experiment, three different operating states were simulated: running, broken, and running under heavy load. Each motor is in one of these operational states which sound differently. The researchers placed a MEMS (micro-electro-mechanical systems for signal recording) microphone in front of the engines, which recorded the operating sounds of the running engines. Recordings were made for each engine, which were analyzed using machine learning techniques with and without background noise.

Automatic detection of different materials based on pling sounds

The principle demonstrator "Air-Hockey table" offers visitors to trade fairs and conferences a sporty variety in addition to a business talk.

Using an Air-Hockey table modified for research purposes, the researchers are working on novel approaches for acoustic quality assurance in an industrial context. They use pucks, which are made of different materials and cause different "pling" sounds as soon as they hit the board of the game table. During the game, these acoustic signals occur so frequently and irregularly that they can be used for analysis by means of machine learning methods to make a reliable statement about the material the pucks are made of.

This non-contact method, in which Fraunhofer IDMT combines its many years of expertise in the fields of acoustic measurement technology, signal processing and machine learning, can be used, for example, to detect material defects or in the in-line monitoring of welding processes. If acoustically perceptible defects are already detected during the process, it can be aborted and restarted promptly. Fraunhofer IDMT's acoustic testing method is also non-destructive and thus serves to reduce expensive test scrap.

Automatic detection of material defects based on rolling sounds

The marble track demonstrator illustrates the performance of the acoustic monitoring method in industrial applications. The detection of defects in surfaces was chosen as an application example.

Three differently coated balls roll in random order through the ball track system. The moving noises of the balls are picked up by small microphones and analyzed by machine learning algorithms. In this way, the balls are monitored in real time and identified by their specific rolling sound. The analysis result - the order of arrival - is immediately shown graphically on a display. With this principle demonstrator, the institute shows new acoustic methods for quality assurance - contactless, non-destructive and safely integrated.

"Putting a stop to burglars"

AI demonstrator for a smart acoustic burglary surveillance system.

Automated acoustic crack detection

Reliably detect hairline cracks in turned parts with airborne sound analysis and AI

As part of the quality inspection, the turned parts shown usually undergo a 100 percent visual final inspection. They are checked for finest hairline cracks, which are hardly visible, but indicate a quality defect. In an experiment, Fraunhofer IDMT investigated to what extent hairline cracks can be detected automatically by airborne sound analysis with AI. As a testing step in the final inspection, the impact of a turned part on a plate was recorded and assigned to the classes "ok" and "not ok" accordingly. The scientists investigated whether the sound spectra of the impact allow a distinction between good and bad parts, which parameters influence these sound spectra (e.g. drop height or material of the plate) and which microphone positions are best suited for the recordings as well as for later analysis. First results showed that it is generally possible to distinguish turned parts into two classes based on the impact sound. 

amo Christmas greetings - because the applications of our AI-based processes are more than diverse.

Turn up your speakers: "We save Christmas"

We offer services in the field of research and development and point out numerous possible applications of acoustic monitoring (amo) to our customers in the context of an industrial project. In addition, we work in partly publicly funded research and development projects to develop application-specific solutions together with wide-ranging expert consortia.

ways of working with Fraunhofer IDMT; Industry project, public funding, Fraunhofer networks
Possibilities of joint project work

The applications for acoustic monitoring are many and varied. Wherever audible sounds occur in processes or on products, the use of automated acoustic testing methods can add value. We are continually exploring the use of our AI algorithms for acoustic quality control in welding and machining operations, as well as other applications where acoustic signals provide information about quality and grade.

Services

  • Feasibility studies for individual use cases
  • Development of industrial prototypes

Acoustic monitoring can be used in machining or welding, among other applications. Use cases are: Inline monitoring, process monitoring or predictive maintenance.

Our scientists exchange information within their research community and with stakeholders from industry and present current research results at scientific meetings and conferences. In addition, we regularly publish articles in professional journals that deal with application topics in our research field, including the German speaking magazines "Maschinenmarkt" or "Fertigungstechnik".

Jahr
Year
Titel/Autor:in
Title/Author
Publikationstyp
Publication Type
2023 Empirical Study on DED-Arc Welding Quality Inspection Using Airborne Sound Analysis
Chauhan, Jaydeep; Gourishetti, Saichand; Rohe, Maximilian; Sennewald, Martin; Hildebrand, Jörg; Bergmann, Jean Pierre
Konferenzbeitrag
Conference Paper
2023 Temporal Resolution of Acoustic Process Emissions for Monitoring Joint Gap Formation in Laser Beam Butt Welding
Kodera, Sayako; Schmidt, Leander; Römer, Florian; Schricker, Klaus; Gourishetti, Saichand; Böttger, David; Krüger, Tanja; Kátai, András; Straß, Benjamin; Wolter, Bernd; Bergmann, Jean Pierre
Zeitschriftenaufsatz
Journal Article
2023 Ohren für Schweißroboter
Breitbarth, Kati; Krüger, Tanja; Liebetrau, Judith
Zeitschriftenaufsatz
Journal Article
2023 Automated Quality Inspection in Additive Manufacturing for Lightweight Construction: A New Approach Based on Virtual Sonic Data and Machine Learning (ML-S-LeAF)
Yildiz, Ömer Faruk; Fritz, Alexander; Storch, Julian; Kátai, András; Ribecky, Sebastian; Hofmann, Peter; Talagini Ashoka, Anitha Bhat; Fassbender, Rene; Marckmann, Hannes; Grollmisch, Sascha; Jansen, Stefan; Adams, Christian; Kroh, Irina; Zaleski, Olgierd; Manohar, Aswin; Keuchel, Sören; Schröder, Thorben; Ren, Yaxiong; Boni, Christiano de; Balestra, Italo; Bös, Joachim; Ferretti, Raphael; Schötz, Johannes; Merschroth, Holger; Gross, Peter; Weigold, Matthias
Konferenzbeitrag
Conference Paper
2023 Acoustic data acquisition for quality monitoring during Powder Bed Fusion with Laser Beam (PBF-LB)
Ren, Yaxiong; Adams, Christian; Gross, Peter; Talagini Ashoka, Anitha Bhat; Kátai, András; Weigold, Matthias; Melz, Tobias
Konferenzbeitrag
Conference Paper
2023 Arc Welding Process Monitoring Using Neural Networks and Audio Signal Analysis
Gourishetti, Saichand; Chauhan, Jaydeep; Grollmisch, Sascha; Rohe, Maximilian; Sennewald, Martin; Hildebrand, Jörg; Bergmann, Jean Pierre
Konferenzbeitrag
Conference Paper
2023 Monitoring of Joint Gap Formation in Laser Beam Butt Welding Using Neural Network-Based Acoustic Emission Analysis
Gourishetti, Saichand; Schmidt, Leander; Römer, Florian; Schricker, Klaus; Kodera, Sayako; Böttger, David; Krüger, Tanja; Kátai, András; Bös, Joachim; Straß, Benjamin; Wolter, Bernd; Bergmann, Jean Pierre
Zeitschriftenaufsatz
Journal Article
2022 Potentials and Challenges of AI-based Audio Analysis in Industrial Sound Analysis
Gourishetti, Saichand; Grollmisch, Sascha; Abeßer, Jakob; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2022 Intelligentes akustisches Monitoring durch ausgewählte Mikrofonierungskonzepte
Fritsch, Tobias; Bös, Joachim; Grollmisch, Sascha; Gourishetti, Saichand; Hofmann, Peter; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2021 Improving Semi-Supervised Learning for Audio Classification with FixMatch
Grollmisch, Sascha; Cano, Estefanía
Zeitschriftenaufsatz
Journal Article
2021 Partial discharge monitoring using deep neural networks with acoustic emission
Gourishetti, Saichand; Johnson, David; Werner, Sara; Kátai, András; Holstein, Peter
Konferenzbeitrag
Conference Paper
2021 The Sounds of Partial Discharge
Gourishetti, Saichand; Werner, Sara; Kátai, András; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2021 Investigating the influence of microphone mismatch for acoustic traffic monitoring
Gourishetti, Saichand; Abeßer, Jakob; Grollmisch, Sascha; Kátai, András; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2021 Investigation of the directional characteristics of the emitted airborne sound by Friction Stir Welding for online process monitoring
Bös, J.; Katai, A.; Grätzel, M.; Other, S.; Stoll, B.; Rohe, M.; Hasieber, M.; Löhn, T.; Hildebrand, J.; Bergmann, J.P.; Breitbarth, K.
Poster
2021 IDMT-Traffic: An Open Benchmark Dataset for Acoustic Traffic Monitoring Research
Abeßer, Jakob; Gourishetti, Saichand; Kátai, András; Clauß, Tobias; Sharma, Prachi; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2020 Techniques improving the robustness of deep learning models for industrial sound analysis
Grollmisch, S.; Johnson, D.S.
Konferenzbeitrag
Conference Paper
2020 IAEO3 - Combining OpenL3 Embeddings and Interpolation Autoencoder for Anomalous Sound Detection
Grollmisch, Sascha; Johnson, David; Abeßer, Jakob; Lukashevich, Hanna
Vortrag
Presentation
2020 Visualizing Neural Network Decisions for Industrial Sound Analysis
Grollmisch, Sascha; Johnson, David; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2020 Plastic Material Classification using Neural Network based Audio Signal Analysis
Grollmisch, Sascha; Johnson, David; Krüger, Tobias; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2020 Compressed air leakage detection using acoustic emissions with neural networks
Johnson, D.; Kirner, J.; Grollmisch, Sascha; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2019 Anwendung von (Luft-)Schallanalyse als ein Verfahren der berührungslosen Qualitätssicherung für die vorausschauende Wartung
Kepplinger, Sara; Helbig, Mareike; Clauss, Tobias; Lukashevich, Hanna
Konferenzbeitrag
Conference Paper
2018 Störschallunterdrückung bei Luftschallanalysen in industriellen Fertigungsstrecken
Nowak, Johannes; Grollmisch, Sascha; Cano, Estefanía; Lukashevich, Hanna; Liebetrau, Judith
Konferenzbeitrag
Conference Paper
2018 Stadtlärm - a distributed system for noise level measurement and noise source identification in a smart city environment
Clauß, Tobias; Abeßer, Jakob; Lukashevich, Hanna; Gräfe, Robert; Häuser, Franz; Kühn, Christian; Sporer, Thomas
Konferenzbeitrag
Conference Paper
2018 Luftschallbasierte Rissdetektion von Metallteilen
Liebetrau, Judith; Grollmisch, Sascha; Nowak, Johannes
Konferenzbeitrag
Conference Paper
Diese Liste ist ein Auszug aus der Publikationsplattform Fraunhofer-Publica

This list has been generated from the publication platform Fraunhofer-Publica
 

Public Audio Datasets

Fraunhofer IDMT has compiled further audio data sets for various research areas such as instrument recognition, fingerings or game analysis.