Automatic Music Analysis

Audio and Visual Content Analysis

Audio signal processing and machine learning for music analysis

Audio signal processing and machine learning are revolutionizing music analysis. From audio matching to music annotation and similarity search, from automatic music transcription to music generation, new application possibilities are emerging for broadcast monitoring, music search and recommendation, music production, and music learning programs.

Our goal is to enable quick and customized access to musical content through our approaches and techniques for automatic music analysis. We are developing practical solutions that are applicable in various domains such as the music industry, entertainment, education, and music production. In addition to enhancing existing technologies, we aim to showcase new application possibilities for automatic music analysis and contribute to the evolution of algorithms and methods.

News and upcoming events

 

New Dataset

Sounds Queer

Andrew McLeod and Sabine Weber present a paper and a dataset at ACM FAccT conference on how text-to-music models represent queer identities.

 

New project

MusicSphere

In the EU project MusicSphere, we are working with our partners to develop digital twins of historical musical instruments for research, restoration, and immersive teaching.

 

Award / 18.12.2025

AI music generation for figure skating

Our student team “Dynamic Beats” wins the hackathon “Beat to Glide – The Ultimate Skating Soundtrack Challenge.”

Understanding Music

How do I quickly find a suitable music piece in a large music catalog? Can I automatically receive recommendations for the perfect beat that harmonizes well with a music production I'm currently working on? Which programs in my archive are the most successful? These are typical questions where our technologies for automatic music analysis can help.

Audio signal processing and machine learning have fundamentally changed music analysis. The multidisciplinary research field "Music Information Retrieval" encompasses algorithms and techniques for extracting musical information from audio data, transforming it into interpretable formats. The results are applied in areas such as broadcast monitoring, music search and recommendation, music production, content tracking, and music eductaion.

AI-based music analysis technologies

General challenges in automatic music analysis include processing large amounts of data, considering musical diversity and context, robustness to variations in recording quality, and the efficient deployment of real-time processing for various applications.

Audio-Matching

Audio matching via audio fingerprinting enables the identification of specific audio recordings in music collections and streams. Media content is compared and matched based on acoustic fingerprints. Audio matching is used for analyzing music usage in broadcast monitoring, content tracking applications, archive maintenance, as well as in music search engines and recommendation systems.
 

At Fraunhofer IDMT, we research how to further improve the accuracy and efficiency of audio matching techniques in order to enable more precise detection and identification of media content.

Annotation and similarity search for music

Annotation and similarity search for music facilitate the organization of music collections and simplify access to musical content. The use of metadata allows for versatile search and recommendation systems, automating the discovery of suitable music or musical elements. This is applicable, for example, in end-user streaming services or music production.


We are working on enhancing annotation and similarity search, particularly for large and diverse music collections, while also considering to user preferences and contextual information.

Automatic music transcription

Automatic music transcription involves converting music signals into symbolic music notation and extracting musical structures such as melodies, chords, and rhythms. These techniques are used in music learning programs, music game development, and music theoretical studies.


The specific challenges of automatic music transcription lie in precisely, reliably, and real-time capturing complex musical structures, even in polyphonic musical pieces or situations with background and ambient noise.

Automatic music generation

Automatic music generation involves the development of algorithms and AI systems capable of creating their own original musical pieces or parts thereof. It provides automated support in the music production process and during live performances, for instance, by generating melodies based on harmonies. This emerging field  introduces new creative approaches to music composition and production.


However, automatic music generation is still a relatively young research field and requires further progress to produce realistic and coherent musical results that meet the expectations of music creators and listeners. At Fraunhofer IDMT, we are researching ways to make the AI composition process transparent and controllable. Our aim is to support the creative collaboration between music creators and AI.

 

Research project

Perspective 2036

The Impact of Generative AI on the Music Industry

 

Research project

MDQS

Metadata Quality and Security: Holistic Data Validation and License Data Management in the Music Industry

 

Research project

MusicSphere

Development of digital twins of historical musical instruments for research, restoration, and immersive teaching

 

Research project

MusicDNA

Facilitating search in large music catalogs

 

Research project

Music Automaton (Musik-Automat)

Development of an AI-based composition app

 

Research project

ISAD 2

Develop explainable and comprehensible deep learning models to better understand sound source characteristics of music, environmental and ambient sounds

 

Research project

AI4Media

Center of excellence for AI in media – Our contributions: Audio forensics, audio provenance analysis, music analysis, privacy and recommendation systems

 

Research project

MusicBricks

Musical Building Blocks for Digital Makers and Content Creators: Transfer state-of-the-art ICT to Creative SMEs in order to develop novel business models.

 

Research project

SyncGlobal

Global music search applied to cross-modal synchronization with video content

 

Research project

GlobalMusic2one

Adaptive, hybrid search technologies for global music portfolios

Research project

MuSEc

Audio analysis and PET for the MusicDNA sustainable eco system

 

Research project

Emused

Interactive app for learning how to improvise on a musical instrument

 

Research project

MiCO

Platform for multimodal and context-based analysis, into which a wide variety of analysis components for different media types can be integrated

Products

 

SoundsLike

AI-based Tagging and Search for Large Music Catalogs

 

Audio Matching

Detect a given audio query within a stream or file – even under noisy conditions or with a very short query

 

Speech and Music Detector

Software tool for automatic detection of music and speech sequences to optimize broadcasting programs or provide accurate accounting for copyright agencies

 

Automatic Music Transcription

Intelligent music transcription – fast and precise

With our technical solutions and services, we provide companies and institutions with concrete support and real added value for their use cases. Contact us to discuss your application!

 

AI-powered metadata analysis for music

 

Automatic melody and chord recognition for music apps

Interested in further use cases?

Here you will find an overview of our use cases.

 

Reference project

Jamahook – AI Sound Matching

Search engine for loops and beats based on SoundsLike

 

Reference project

SWR Media Services

Audio matching software for automatic advertising monitoring of SWR radio programs

Interview on AI-based music analysis technologies with Hanna Lukashevich

Hanna talks about typical use cases, where music content plays an important role and how the AI-based music analysis solutions help professional media organizations and music technology companies optimize their businesses. Fraunhofer TECHNICAL SPECIAL - 2022/8/18