A fake voice message, a manipulated interview, or a seemingly authentic call from a company executive: audio deepfakes are becoming increasingly realistic and difficult to detect. Modern generative AI can now imitate voices, speaking styles, and emotional expression with remarkable accuracy. As a result, existing detection methods are increasingly reaching their limits.

This challenge is at the core of the new research and development project PADSE (person-centric audio- and speech-based Deepfake and Shallowfake Detection), which has been launched at the Fraunhofer Institute for Digital Media Technology IDMT. Together with Deutsche Welle and the Intelligent Information Systems research group at Bauhaus-Universität Weimar, the consortium is developing and evaluating novel methods for detecting synthetic and manipulated speech content.

Person-centric approach enhances generic detection

Current detection methods primarily focus on identifying general characteristics of synthetic or manipulated content. PADSE extends this approach by introducing person-centric reference profiles. These profiles describe characteristic features of persons to be protected, including voice, speaking style, emotional expression, and linguistic patterns.

New content can therefore be assessed not only for general deepfake indicators but also for its consistency with the respective person’s reference profile.

This approach aims to improve the reliable detection and classification of audio manipulations and synthetic speech, even in the face of increasingly advanced generative AI technologies. Related privacy and security challenges are addressed from the very beginning.

Bridging research and real-world applications

PADSE is an application-oriented research and development project. Its objective is not only to develop novel detection methods but also to evaluate them under real-world conditions.

To achieve this, the consortium combines complementary expertise in audio forensics, text analysis, and journalistic practice. The Intelligent Information Systems research group at Bauhaus-Universität Weimar develops methods for text and authorship analysis. Deutsche Welle contributes requirements based on journalistic workflows, coordinates the creation and annotation of training and test data, and validates the developed methods in real-world application scenarios. Fraunhofer IDMT coordinates the project and develops the core technologies for detecting speech synthesis and audio manipulation as well as for technical data protection and data security.

Goal: More reliable detection of synthetic speech

The technologies developed within PADSE are intended to support media organizations, companies, public authorities, and other institutions in more reliably detecting and assessing manipulated and synthetic speech content.

In the long term, PADSE aims to strengthen trust in digital communication and contribute to enhancing Germany’s technological sovereignty in dealing with AI-generated speech content.

Funding

PADSE is funded by the German Federal Ministry for Research, Technology and Space (BMFTR) under the funding initiative “Trust in Democracy and the State: Detecting and Countering Digital Disinformation.” The project will run until 2029.

Funding reference: 16KIS2468K