MusicSphere – Digital twin for historical musical instruments

Historical church organs are among the most technically and acoustically sophisticated musical instruments and are also important testimonies to European cultural history. However, many of these instruments are endangered by aging processes, climatic influences, or structural changes. The European research project MusicSphere is therefore developing innovative digital methods to precisely record their structural, mechanical, and tonal properties and make them scientifically usable in the long term in the form of a digital twin. MusicSphere thus makes a decisive contribution to the digital preservation, research, and communication of European music culture.

A special focus is placed on traditional wind instruments such as pipe organs and their ancient precursors, such as the Greek hydraulis. With the help of multimodal technologies, their construction, functionality, and unique sound characteristics are to be reproduced in detail and made accessible for research, restoration, and the public.

 

Structural, mechanical, and sound analysis

Historical musical instruments combine complex material structures, precision mechanical processes, and characteristic sound generation processes. Organs are always in close interaction with their surrounding architecture, as they are usually built specifically for the respective room. Correct recording and mapping of the respective room acoustics is therefore of central importance.

MusicSphere creates precise 3D models and acoustic simulations for this purpose, which serve as the basis for the subsequent digital twin. The multimodal approach includes:

• High-resolution 3D geometry capture
• Material acoustic measurement methods that show how materials influence sound
• Flow and vibration analyses for investigating and modeling different types of pipes
• AI-supported modeling and sound synthesis

These integrated methods enable a deep understanding of the construction, mechanics, and sound production of historical musical instruments and form the basis for digital modeling as a digital twin.

 

Digital reconstruction and scientific applications

Based on the collected data, MusicSphere is developing a modular technology kit that can be used to physically and acoustically model historical musical instruments and simulate their dynamic behavior. This includes:

• The mapping of geometric and material-related properties
• The modeling of mechanical processes
• Simulation of flow- and vibration-based sound-relevant processes
  

The result is a digital twin that realistically reproduces the structural, functional, and tonal properties. This opens up a wide range of scientific and restoration applications, including:

• Scientific analysis
• Reconstruction of lost or no longer playable instruments
• Simulation-based evaluation of restoration measuresc
• Comparative research into historical construction methods
   

Interactive applications and long-term digital preservation

In addition, immersive VR and AR applications are being developed that allow the digital twin to be explored virtually, played, and experienced in different spatial contexts. In the future, users will be able to:

• Experience instruments in VR and AR environments from new perspectives
• Visualize mechanical processes
• Virtually reconstruct historical performance situations
• Play organs and other musical instruments virtually

These formats are intended to open up new opportunities for education, research, and cultural participation, and to make rare or delicate instruments accessible to a wider audience.

Digital models open up additional potential for the long-term preservation of historical instruments, for example through:

• Secure digital archiving
• Making them available internationally for research and teaching
• Protecting them from loss due to aging, modifications, or disasters
• Reconstructing sound characteristics that can no longer be played
• Acoustic simulations of material, geometric, and room acoustic influences
• Ancorporating historical sources for the most authentic sound reconstruction possible

Fraunhofer IDMT's area of responsibility

Fraunhofer IDMT develops physical and acoustic models that realistically describe the sound generation and radiation behavior of historical musical instruments—especially organs—taking into account changing material properties. The simulations are validated by measurements on real instruments and thus provide the data basis for the digital twins. At the same time, the institute is researching AI-based methods for automatic transcription and digital sound synthesis. These technologies support the lifelike reconstruction of historical sound characteristics and form the basis for virtual instruments and immersive applications in virtual and augmented reality.