Older people in particular find it difficult or tiring to follow the spoken contributions in a film, radio or TV series. To ensure that dialogues in media can always be mixed in the most comprehensible way, Steinberg has equipped the latest version 11 of its production solution Nuendo with a new module based on algorithms developed by Fraunhofer IDMT. The new feature analyzes incoming audio signals via a speech intelligibility model with automatic speech recognition technology and calculates how much effort the listener must put in to understand the spoken words within the mix. »The tool Intelligibility Meter measures objective speech intelligibility in media production in real time, controlled by artificial intelligence. It is a results of several years of hearing research in Oldenburg«, explains Dr. Jan Rennies-Hochmuth, Head of Personalized Hearing Systems at Fraunhofer IDMT.

Demographic change and acoustic convergence

»In order to do justice to different hearing preferences and hearing losses associated with demographic change, it is important for producers and sound designers to be able to objectively approximate the actual hearing ability of end users. We are pleased to offer our customers the Intelligibility Meter, a major enhancement to the Nuendo product«, explains Timo Wildenhain, Head of ProAudio at Steinberg Media Technologies GmbH.

The method developed at Fraunhofer IDMT is based on neural networks and enables an automatic and target group specific measurement of speech intelligibility across applications. Thus, not only sound creators are supported in postproduction. It can also be used directly on set or in the field of sound reinforcement technology.

»The technology is also of interest to broadcasters, content providers and manufacturers from the home entertainment, consumer electronics and telecommunications sectors«, explains Dr. Jan Rennies-Hochmuth. »This is because we are addressing the key factors for poor speech intelligibility throughout the entire production and distribution chain and want to break down existing barriers for the greatest possible variety of target groups, applications and listening situations with the help of innovative software technologies«, the Fraunhofer expert continues.

Hearing, Speech and Audio Technology at Fraunhofer Institute for Digital Media Technology IDMT

The objective of the Division Hearing, Speech and Audio Technology HSA of the Fraunhofer Institute for Digital Media Technology IDMT is to transpose scientific findings related to hearing perception and man-machine interaction into technological applications. Its applied research priorities are the enhancement of sound quality and speech intelligibility, personalized audio reproduction, acoustic speech recognition and event detection with the help of artificial intelligence. A further focus is the use of mobile neurotechnologies, which facilitate the recording of brain activity and utilization of the resulting data outside the laboratory too. Application fields include consumer electronics, transport, the automotive sector, industrial production, security, telecommunications and healthcare. Through scientific partnerships, Fraunhofer IDMT-HSA has close links with the Carl von Ossietzky University of Oldenburg, Jade University and other institutions engaged in hearing research in Oldenburg as well as to the University of Applied Sciences Emden/Leer. Fraunhofer IDMT-HSA is a partner in the »Hearing4all« Cluster of Excellence.

The Division Hearing, Speech and Audio Technology HSA is funded in the program »Vorab« by the Lower Saxony Ministry of Science and Culture (MWK) and the Volkswagen Foundation for its further development.