Analysis and optimization of speech intelligibility

Fraunhofer IDMT’s Speech Activity Detection (SAD) reliably identifies spoken segments in audio signals. For example, the technology streamlines the workflow for audio professionals, as it makes it easy to locate speech during editing without having to listen to every segment. The technology was integrated into Steinberg’s post-production software Nuendo as the "Dialog Detection" feature. It is also used in other solutions developed by the Oldenburg Branch, such as its in-house speech and speaker recognition, algorithms for reducing background noise, and as a privacy filter. SAD can be used to filter out non-speech segments during audio processing. Conversely, it can ensure that speech is not recorded in the first place, thereby protecting users’ privacy—for example, in public spaces.

using LEAP technology (Listening Effort Prediction for Acoustic Parameters)

By combining machine learning and psychoacoustic modeling, researchers at Fraunhofer IDMT in Oldenburg have developed new technologies to objectively measure and visualize listening effort for individual audio segments in real time. This makes it possible, for example, to verify consistently good speech intelligibility in media productions and streams. The technology is already supporting audio professionals in post-production across various production solutions, such as the "LE-Meter" plug-in from NUGEN Audio, the "Speech Intelligibility Meter" in RTW’s hardware-based audio measurement platform, and the Nuendo production solution from Steinberg.

The algorithms are based on the latest findings of the Oldenburg hearing research. They have been evaluated multiple times in recent years and validated through formal listening tests with participants of various age groups.

using PSIO (Personal Speech Intelligibility Optimization) technologies

Various signal processing methods developed by Fraunhofer IDMT can be combined to improve speech intelligibility. The real-time prediction models described earlier detect speech activity and identify the objective listening effort required for an audio signal (e.g., a broadcast video). Speech separation technologies isolate speech content from background noise and split it into two sound objects that can be processed independently of one another. Background music and sound effects, for example, could be reduced according to personal listening preferences or needs to achieve better intelligibility of dialogue.

These technologies are ideally suited for use in media production, both for live broadcasts and in post-production. Media libraries, streaming services, and communication service providers can also offer their customers added value, ranging from alternative soundtracks with improved intelligibility to customizable solutions on devices. 

In addition, adaptive signal processing can enhance the speech signal when the technical transmission of speech is marred by background noise and reverberation. Especially when it matters most—for example, in emergency dispatch centers, public address systems, or conference and telephony solutions—not a single word should be lost.

Another technology developed by Fraunhofer IDMT allows users to customize speech and background sounds based on their own audio preferences to create a personalized listening experience. Using a particularly simple and elegant process, sound and dynamics can be personalized without requiring users to navigate complex submenus or adjust parameters.

The so-called "YourSound" technology is already being used for the individual configuration of media playback in cars and headphones. Once set via an intuitive user interface, the personalized parameters have a positive effect on the overall sound. This results in a better listening experience, regardless of playback volume or driving conditions. Thanks to Fraunhofer algorithms, the sound of music and movies is optimally tailored to individual sound preferences.