CAPA

People with aphasia, a neurologically caused language disorder, often face significant communication barriers in their daily lives. Because of difficulties with speaking, understanding speech, reading, and writing, they often find it hard to interact with others, which can increasingly lead to social isolation. Various communication aids are already available to those affected, such as symbol boards or electronic talkers with speech output. However, many of these solutions lack flexibility, offer little customization, and primarily support speech production, while speech comprehension is often neglected.

The CAPA research project (Communication support through AI to increase the participation in aphasia) aims to develop an intelligent, mobile communication application for people with aphasia. The project’s goal is to provide people with aphasia with a practical, everyday tool to express themselves clearly and actively participate in communication.

AI technologies for speech recognition and response generation are intended to support both speech comprehension and active speaking. Communication can take place through various modalities such as text, images, or symbols. A key feature of the CAPA app will be the automatic, individual adaptation of content. Context of use, language skills, and usage patterns will be continuously taken into account so that the application can evolve to fit the user perfectly as it is used more frequently.

Tasks of the project partners

In this project, Fraunhofer IDMT in Oldenburg is further developing technologies for speech recognition and AI-based speech analysis for people with aphasia of varying severity. To this end, they are using methods of automatic speech recognition (ASR) and natural language processing (NLP). These methods enable the app to record spoken words, convert them into text, and analyze them in context.

Limedix GmbH coordinates the CAPA project and is responsible for app development and integration as well as data collection. The Fraunhofer IPA focuses on researching algorithmic methods for generating response options using Large Language Models (LLM). The Johanniter Hospital in Stendal supports the project with clinical expertise and conducts data collection as part of the feasibility study.