Industrial Sound Analysis

AI-based acoustic monitoring in production

The research activities in the field Industrial Sound Analysis are focused on the development of AI-based methods for automated acoustic monitoring of products and processes for use in in-line and end-of-line quality control, process monitoring and predictive maintenance.

Aktuelles

Industrial Sound Analysis

International trade fair for quality assurance

36. Control in Stuttgart

We will be showing "Methods for AI-based acoustic quality control in production" from April 23 to 26, 2024

The potential of acoustic quality assessment in production

How do you use acoustic signals to assess quality?

Your car is a good example to show the potential of quality assessment based on acoustic signals. You know the familiar driving sound caused by the engine, tyre wear and airflow. But what do you feel when this noise changes? A rattling or grinding noise might worry you and make you wonder if your car is still safe to drive. You probably react to this unknown and unexpected condition by driving to the workshop or at least stopping your car. It's no different in industry. We analyse the sound of your product or production process and let you know when something is not working as expected.

In industry, wherever there is movement, there are audible sounds that indicate the quality of products or processes. At the Fraunhofer IDMT in Ilmenau, Germany, we develop AI-based methods for sound analysis - especially for the analysis of industrial sounds - and thus create innovative approaches for automated acoustic monitoring (amo) of products and processes. amo can be used along the entire value chain for quality assessment and offers added value where, for example, optical monitoring methods reach their limits.

Our team of scientists from the fields of data science, data analysis, software development and project management is therefore researching AI-based solutions for audio signal analysis. The innovation at amo is that the measurement data is processed without any connection to an external cloud, so our solutions can be used locally within the company or directly on the machine.

Challenges in sensor-based machine monitoring

Machine and plant manufacturers are faced with the challenge of providing their customers with comprehensive sensor-based machine monitoring. However, existing methods cannot solve all the problems that arise, such as unexpected downtime, continuous monitoring of the machine or evaluation of existing machine data. In addition, there is a shortage of specialists who know their machines so well that they can immediately identify faults by sound and take action if necessary.

AI-based acoustic monitoring

Components of a pilot project

  • Analysis and interpretation of the soundscape in the production environment
  • Create a customized setup to record the sounds
  • Systematically record acoustic signals
  • Select and apply analysis methods
  • Consideration of privacy and security issues
  • Evaluate the feasibility of acoustic monitoring for specific applications

Overview

Research field "amo”

Research project

DIAMOSS-I

Development of an Intelligent, Automonitored Sound Sensor System for Harsh Industrial Conditions

 

Research project

OptiStrick

Development of an AI-supported inline quality assurance system for optimizing highly flexible knitting technologies

 

Research project

RapidKI

Intelligent inline control for green laser ablation processes

Research project

HybridDigital

Digitalization for efficient process selection and design of hybrid structures based on experimental and synthetic data

Research project

ML-S-LeAF

Development of machine learning algorithms based on virtual sound data for lightweight construction for quality assurance in additive manufacturing.

 

Research project

AKoS

Acoustic inspection of weld seams of safety-critical components as part of quality assurance

Research project

e-LAS+

Multimodal quality assurance for production of electricity storage in safety-critical systems

Research project

ExtrudEAR

Mapping auditory perception and human expert knowledge to extruder process control

Research project

SEC-Learn

Sensor Edge Cloud for federated learning

 

ZIM-Netzwerk »AkuPro«

Akustische Analysen in Produktionssystemen

Werden Sie Netzwerkpartner!

Application meets partner

Networks (German)

Leistungszentrum InSignA – Intelligent Signal Analysis and Assistance Systems

AkuPro – Acoustic Analyses in Production Systems (ZIM Network)

IMAMF Innovationsform – AI-based acoustic monitoring of manufacturing processes

 

 

Leistungszentrum

InSignA

Examples of successfully completed feasibility studies

Automatic detection of compressed air leaks

Reliably and automatically detect audible leaks thanks to airborne sound analysis and machine learning methods.

Compressed air is an indispensable resource for many German industrial and trade companies for the operation of machines and plants. At the same time, it also represents a high cost factor on the electricity bill. On average, companies waste 30 percent of the energy generated due to the unnoticed escape of expensive compressed air. To detect such leaks, they rely, among other things, on ultrasound testing methods and on the mere hearing of trained personnel.

Fraunhofer IDMT has now tested in an experiment whether this "hearing" can be automated and reproduced using microphones in combination with machine learning methods in order to develop a reliable system for leakage detection. First results show that this is generally possible.

Automatic detection and differentiation of electric engines

The engine demo shows the performance of acoustic monitoring in industrial applications. The detection of three different sounding electric engines was chosen as an application example.

In this experiment, three different operating states were simulated: running, broken, and running under heavy load. Each motor is in one of these operational states which sound differently. The researchers placed a MEMS (micro-electro-mechanical systems for signal recording) microphone in front of the engines, which recorded the operating sounds of the running engines. Recordings were made for each engine, which were analyzed using machine learning techniques with and without background noise.

Automatic detection of different materials based on pling sounds

The principle demonstrator "Air-Hockey table" offers visitors to trade fairs and conferences a sporty variety in addition to a business talk.

Using an Air-Hockey table modified for research purposes, the researchers are working on novel approaches for acoustic quality assurance in an industrial context. They use pucks, which are made of different materials and cause different "pling" sounds as soon as they hit the board of the game table. During the game, these acoustic signals occur so frequently and irregularly that they can be used for analysis by means of machine learning methods to make a reliable statement about the material the pucks are made of.

This non-contact method, in which Fraunhofer IDMT combines its many years of expertise in the fields of acoustic measurement technology, signal processing and machine learning, can be used, for example, to detect material defects or in the in-line monitoring of welding processes. If acoustically perceptible defects are already detected during the process, it can be aborted and restarted promptly. Fraunhofer IDMT's acoustic testing method is also non-destructive and thus serves to reduce expensive test scrap.

Automatic detection of material defects based on rolling sounds

The marble track demonstrator illustrates the performance of the acoustic monitoring method in industrial applications. The detection of defects in surfaces was chosen as an application example.

Three differently coated balls roll in random order through the ball track system. The moving noises of the balls are picked up by small microphones and analyzed by machine learning algorithms. In this way, the balls are monitored in real time and identified by their specific rolling sound. The analysis result - the order of arrival - is immediately shown graphically on a display. With this principle demonstrator, the institute shows new acoustic methods for quality assurance - contactless, non-destructive and safely integrated.

"Putting a stop to burglars"

AI demonstrator for a smart acoustic burglary surveillance system.

Automated acoustic crack detection

Reliably detect hairline cracks in turned parts with airborne sound analysis and AI

As part of the quality inspection, the turned parts shown usually undergo a 100 percent visual final inspection. They are checked for finest hairline cracks, which are hardly visible, but indicate a quality defect. In an experiment, Fraunhofer IDMT investigated to what extent hairline cracks can be detected automatically by airborne sound analysis with AI. As a testing step in the final inspection, the impact of a turned part on a plate was recorded and assigned to the classes "ok" and "not ok" accordingly. The scientists investigated whether the sound spectra of the impact allow a distinction between good and bad parts, which parameters influence these sound spectra (e.g. drop height or material of the plate) and which microphone positions are best suited for the recordings as well as for later analysis. First results showed that it is generally possible to distinguish turned parts into two classes based on the impact sound. 

amo Christmas greetings - because the applications of our AI-based processes are more than diverse.

Turn up your speakers: "We save Christmas"

We offer services in the field of research and development and point out numerous possible applications of acoustic monitoring (amo) to our customers in the context of an industrial project. In addition, we work in partly publicly funded research and development projects to develop application-specific solutions together with wide-ranging expert consortia.

ways of working with Fraunhofer IDMT; Industry project, public funding, Fraunhofer networks
Possibilities of joint project work

The applications for acoustic monitoring are many and varied. Wherever audible sounds occur in processes or on products, the use of automated acoustic testing methods can add value. We are continually exploring the use of our AI algorithms for acoustic quality control in welding and machining operations, as well as other applications where acoustic signals provide information about quality and grade.

Services

  • Feasibility studies for individual use cases
  • Development of industrial prototypes

Acoustic monitoring can be used in machining or welding, among other applications. Use cases are: Inline monitoring, process monitoring or predictive maintenance.

Our scientists exchange information within their research community and with stakeholders from industry and present current research results at scientific meetings and conferences. In addition, we regularly publish articles in professional journals that deal with application topics in our research field, including the German speaking magazines "Maschinenmarkt" or "Fertigungstechnik".

Chauhan, J., Gourishetti, S., Rohe, M., Sennewald, M., Hildebrand, J., & Bergmann, J. P. (2023).
Empirical study on DED-Arc welding quality inspection using airborne sound analysis.
In K.-U. Sattler (Ed.), Engineering for a Changing World: Proceedings; 60th ISC, Ilmenau Scientific Colloquium, Technische Universität Ilmenau, September 04-08, 2023 (1.3.055). Ilmenau Scientific Colloquium. Technische Universität Ilmenau ; 60 (Ilmenau) : 2023.09.04-08. https://doi.org/10.22032/dbt.58732

 

Kodera S, Schmidt L, Römer F, Schricker K, Gourishetti S, Böttger D, Krüger T, Kátai A, Straß B, Wolter B, et al.
Temporal Resolution of Acoustic Process Emissions for Monitoring Joint Gap Formation in Laser Beam Butt Welding.
Applied Sciences. 2023; 13(18):10548. https://doi.org/10.3390/app131810548

 

Saichand Gourishetti, Sascha Grollmisch, Jaydeep Chauhan; Jean Pierre Bergmann, Jörg Hildebrand, Maximilian Rohe, Martin Sennewald (Technische Universität Ilmenau, FG Fertigungstechnik)
Arc Welding Process Monitoring Using Neural Networks and Audio Signal Analysis
Proceedings SMSI 2023 - Sensor and Measurement Science International

 

Y. Ren, C. Adams, P. Gross, A.B. Talagini Ashoka, A. Kátai, M. Weigold and T. Melz
Acoustic data acquisition for quality monitoring during Powder Bed Fusion with Laser Beam (PBF-LB)
Proceedings, Fortschritte der Akustik - DAGA 2023

 

Ö.F. Yildiz, A. Fritz, J. Storch, A. Kátai, S. Ribecky, P. Hofmann, A.B. Talagini Ashoka, R. Fassbender, H. Marckmann, S. Grollmisch, S. Jansen, C. Adams, I. Kroh, O. Zaleski, A. Manohar, S. Keuchel, H. Merschroth, T. Schröder, P. Gross, Y. Ren, C. De Boni, I. Balestra, R. Ferretti, J. Schötz, J. Bös and M. Weigold

Automated Quality Inspection in Additive Manufacturing for Lightweight Construction: A New Approach Based on Virtual Sonic Data and Machine Learning (ML-S-LeAF)
Proceedings, Fortschritte der Akustik - DAGA 2023

 

Fritsch, Tobias; Bös, Joachim; Grollmisch, Sascha; Gourishetti, Saichand; Hofmann, Peter; Liebetrau, Judith
Intelligentes akustisches Monitoring durch ausgewählte Mikrofonierungskonzepte
Tagungsband, Fortschritte der Akustik - DAGA 2022

 

Gourishetti, Saichand ; Grollmisch, Sascha; Abeßer, Jakob; Liebetrau, Judith:
Potentials and Challenges of AI-based Audio Analysis in Industrial Sound Analysis
Tagungsband, Fortschritte der Akustik - DAGA 2022

 

Grätzel, M.; Other, S.; Stoll, B.; Rohe, M.; Hasieber, M.; Löhn, T.; Hildebrand, J.; Bergmann, J. P. (Fachgebiet Fertigungstechnik, TU Ilmenau); Kátai, A.; Breitbarth, K.; Bös, J. (Fraunhofer IDMT):
Investigation of the directional characteristics of the emitted airborne sound by Friction Stir Welding for online process monitoring
2nd International Conference on Advanced Joining Processes (AJP 2021)

 

Johnson, David S.; Lorenz, Wolfgang; Tänzer, Michael; Mimilakis, Stylianos; Grollmisch, Sascha; Abeßer, Jakob; Lukashevich, Hanna:
DESED-FL and URBAN-FL: Federated LearningDatasets for Sound Event Detection
29th European Signal Processing Conference, EUSIPCO 2021

 

Abeßer, Jakob; Gourishetti, Saichand; Kátai, András; Clauß, Tobias; Sharma, Prachi; Liebetrau, Judith:
IDMT-Traffic: An Open Benchmark Dataset forAcoustic Traffic Monitoring Research
29th European Signal Processing Conference, EUSIPCO 2021

 

Gourishetti, Saichand; Johnson, David; Werner, Sara; Kátai, András; Holstein, Peter (SONOTEC GmbH):
Partial discharge monitoring using deep neural networks with acoustic emission
Inter-Noise 2021 Congress

 

Gourishetti, Saichand; Werner, Sara; Kátai, András; Liebetrau, Judith:
The Sounds of Partial Discharge
Tagungsband, Fortschritte der Akustik - DAGA 2021

 

Holstein, Peter (SONOTEC GmbH); Gourishetti, Saichand (Fraunhofer IDMT); Fuchs, Karsten (TU Ilmenau); Berger, Frank (TU Ilmenau);  Vyas, Daxeshkumar Harikeshbhai (TU Ilmenau/SONOTEC GmbH):
Acoustics of electrical discharges
27th International Congress on Sound and Vibration, 2021

 

Grollmisch, Sascha; Cano, Estefanía:
Improving Semi-Supervised Learning for Audio Classification with FixMatch
Special Issue Machine Learning Applied to Music/Audio Signal Processing, MDPI, 2021

 

Johnson, David; Grollmisch, Sascha:
Techniques Improving the Robustness of Deep Learning Models for Industrial Sound Analysis
28th European Signal Processing Conference EUSIPCO 2020 

 

Grollmisch, Sascha (TU Ilmenau); Johnson, David und Liebetrau, Judith (Fraunhofer IDMT): 
Visualizing Neural Network Decisions for Industrial Sound Analysis
Proceedings SMSI 2020 – Sensor and Measurement Science International (Konferenz hat nicht stattgefunden)

 

Grollmisch, Sascha (TU Ilmenau); Johnson, David; Krüger, Tobias und Liebetrau, Judith (Fraunhofer IDMT): 
Plastic Material Classification using Neural Network based Audio Signal Analysis
Proceedings SMSI 2020 – Sensor and Measurement Science International (Konferenz hat nicht stattgefunden)

 

Helbig, Mareike; Clauß, Tobias; Kepplinger, Sara; Lukashevich, Hanna:
Anwendung von (Luft-)Schallanalyse als ein Verfahren der berührungslosen Qualitätssicherung für die vorausschauende Wartung
Tagungsband, Instandhaltungsforum, 22.-23. Mai 2019, Dortmund

 

Liebetrau, Judith; Grollmisch, Sascha; Nowak, Johannes:
Luftschallbasierte Rissdetektion in kleinen Metallteilen
Proceedings, 44. Deutsche Jahrestagung für Akustik (DAGA) 19.- 22. März 2018, München

 

Nowak, Johannes; Grollmisch, Sascha; Cano Estefanía; Lukashevich, Hanna; Liebetrau, Judith:
Störschallunterdrückung bei Luftschallanalysen in industriellen Fertigungsstrecken
Proceedings, 44. Deutsche Jahrestagung für Akustik (DAGA) 19.- 22. März 2018, München

 

Clauß, Tobias; Abeßer, Jakob; Lukashevich, Hanna; Gräfe, Robert; Häuser, Franz; Kühn, Christian; Sporer, Thomas:
StadtLärm – A distributed system for noise level measurement and noise source identification in a smart city environment
Proceedings, 44. Deutsche Jahrestagung für Akustik (DAGA) 19.- 22. März 2018, München

 

Public Audio Datasets

Fraunhofer IDMT has compiled further audio data sets for various research areas such as instrument recognition, fingerings or game analysis.