Hearing, speech, audio and neurotechnology for the automotive sector

How does a car learn to hear? How can the sound in the vehicle interior be adjusted to individual requirements? What possibilities are there for acoustic monitoring in production? How quickly do mobile EEG systems detect driver attention during monotonous and autonomous driving?

Fraunhofer IDMT in Oldenburg can answer all these questions.

Whether ambulance siren, wet road or nail in the tyre – external noises deliver important information that conventional sensors on the vehicle can only capture with difficulty or at a late stage. That is why the Oldenburg Branch for Hearing, Speech and Audio Technology HSA of the Fraunhofer Institute for Digital Media Technology IDMT is developing smart technologies for acoustic event detection as well as methods and concepts for optimal signal capture on the vehicle within the project “The Hearing Car”. In close collaboration with manufacturers and suppliers in the automotive sector, we are working on a sense of hearing for the car, which should ensure more safety, greater reliability and innovative applications in road traffic. In order to make “The Hearing Car” reality, our developers are contributing their own special project experience in the automotive sector as well as cross-group expertise. Apart from developing AI algorithms for acoustic event detection, researchers at Fraunhofer IDMT are also addressing other challenges. These include optimal signal capture through sensor positioning, signal preprocessing and enhancement, and suppression of background noise.

• Acoustic environment perception sensors for unique assistance systems
• New approaches in predictive maintenance and adaptronics
• Maximum data protection through acoustic anonymisation strategies
• Use of our “The Hearing Car” environment for client-specific development of hardware and software
• Acoustic end-of-line testing on short test tracks at the end of vehicle production 

To participate safely in road traffic, drivers must pay attention to the traffic situation. Modern vehicles are generally equipped with a system that monitors the driver’s attention and advises them, if necessary, to take a break. Fraunhofer IDMT in Oldenburg is studying how driver attention, also known as vigilance, changes during monotonous car journeys and whether there is a difference between manual and autonomous driving. The starting point for the “Mobile Neurotechnologies” group is to measure where attention originates: in brain activity. With the help of mobile EEG systems and other vital data, such as heart rate or blink rate, the researchers are examining whether a change in the level of vigilance can be detected more swiftly on the basis of brain activity than via other signs of fatigue such as yawning. In the future, it should be possible to use the knowledge acquired to test driver assistance systems. For easy use of these methods outside the laboratory, Fraunhofer IDMT is developing flexible electrode grids to facilitate long-term EEG recordings with excellent signal quality and high wearing comfort at the same time.

• Research studies and evaluation of driver assistance systems
• Possible combination with existing solutions for mobile EEG recording
• EEG recording in everyday situations with flexible electrode grid

For contactless health monitoring, Fraunhofer IDMT from Oldenburg relies on lateral positioning of the radar measuring device. This allows the human body to be divided into independent sections from the radar's point of view. This prevents different vital parameters from overlapping each other in the measurement. The division allows different parameters to be monitored simultaneously along the entire body - for example, breathing on the torso and pulse on the legs. Monitoring vital signs with the new measurement and analysis methods is not only conceivable in the medical field. The Fraunhofer IDMT team is already testing the use of radar in vehicles for people in a seated position in order to monitor the vital parameters of the driver, e.g., for professional drivers or, in perspective, in autonomous vehicles. In this case, positioning the radar in the vehicle ceiling, for example, is conceivable.

• Contactless recording of limb, respiratory and pulse movements based on novel evaluation algorithms
• Cross-industry application: automotive/autonomous driving, sleep monitoring, nursing, intensive care medicine
• Synchronous measurement of additional sensors such as EEG and audio

The Oldenburg Branch HSA has exciting developments for vehicle interiors at the ready that make it possible to personalise the sound of audio systems quickly and intuitively.

With conventional equalisers, it is not possible to adjust the sound quickly and while driving. In addition, they do not adequately take different preferences at different playback volumes into consideration. Inadequate usability and too much listening effort pose unnecessary safety risks.

One-time sound personalisation with the help of “YourSound”, which can be integrated into an infotainment system, forms the basis for all future audio playback in the vehicle. Once set, users experience an individually enhanced listening experience, regardless of playback volume and driving situation.

Here you can see how easy it is to adjust your own sound preferences.

• User-friendly adjustment of sound to personal listening preferences
• Integrable in the widest variety of systems: multimedia systems, smartphones, TV systems, infotainment platforms
• Preferred balance between high and low sounds is automatic at any volume.
• In this way, all frequencies are constantly audible and match the listener’s preferred sound. 

In vehicle production, manufacturers often rely on practical plug connections. However, time and money are wasted if a connection engages incorrectly and this goes unnoticed. Researchers at the Oldenburg Branch for Hearing, Speech and Audio Technology HSA of the Fraunhofer Institute for Digital Media Technology IDMT have developed an audio-based technology that analyses the clicking noise produced when a connection engages. Microphones record the noise, which algorithms then analyse. The system then gives positive feedback or sends a warning if a connection has not clicked properly into place.

Our smart solutions for automated and non-contact production control make downtime and maintenance plannable because the monitoring system detects damage before it occurs. Quality assurance is possible not only in vehicle production but also during test drives and in test facilities or ongoing monitoring during driving (The Hearing Car).

• Reliable detection of deviations from the noise
• Process documentation and quality assurance
• Digitisation of operating procedures