Web-based recognition of polyphonic tone sequences
Fraunhofer IDMT and Skoove developed an algorithm for web-based recognition of polyphonic tone sequences
There is a special piano teacher, always available and always listening closely to what the students are playing. His name is Skoove, and he has been around since November 2015. Skoove is a browser-based learning platform that offers interactive lessons for students who want to learn to play the piano, electric piano, or keyboard. What makes Skoove so special is that students receive immediate feedback on their performance as they play. This is made possible by the melody recognition algorithm developed by Fraunhofer IDMT.
So how does the algorithm work? The student plays a piece of music to the laptop’s microphone. The algorithm, developed by Fraunhofer IDMT in close collaboration with Skoove, executes a polyphonic validation of the notes. A function named »multipitch validation« checks whether the student has actually played the predefined reference notes correctly and in the correct order.
Melody recognition is done in real-time without disturbing the student’s performance. To do so, the algorithm must be computationally efficient, reliable, and low latency. Another important aspect is robustness in the recognition of piano notes in the presence of interfering sounds from the environment (produced by air condition systems, for example), different volume levels, or out-of-tune pianos. The robustness of the algorithm has been confirmed in extensive testing procedures including various kinds of interfering sounds.
»It doesn’t matter if you use a cheap keyboard or a premium piano – with Skoove, beginners and advanced players get the chance to learn to play their instrument quickly and have fun with it«, says Florian Plenge, managing director of Skoove, a Berlin based company.
Fraunhofer IDMT is well-established in the field of automatic melody recognition. The tools developed by Fraunhofer IDMT allow analysis and evaluation of basic musical features, such as rhythm, melody/harmony, structure, tuning, or genre. They are used for example, in combination with music search engines, educational software, or music games.