A/V Analyzing Toolbox

Fraunhofer Institute for Digital Media Technology

Television Broadcast
© istock.com/Anthony Brown

Automated Audio and Video Analysis

A/V Analyzing Toolbox is a modular content analysis framework which allows comprehensive automated processes to be applied to audio-visual content for the optimization of broadcasting, distribution and archiving workflows. It inspects A/V data with respect to technical and perceptual parameters and extracts metadata, as well as semantic information. A/V Analyzing Toolbox provides dedicated analyzing components for several uses cases and application scenarios.

QC Components

The A/V Error Detection Libraries provide automatic detection of audio-visual errors and quality issues for A/V production, content management, and archiving. Since the detections run on signal level, the technology can be applied at any position within a process chain, e.g. ingest, play-out, transcoding, etc.

Speech and Music Detection

The automatic Speech and Music Detection segments audio content with respect to containing speech, music, speech and background music, and silence, which can be used for automatic music royalty reporting, cue sheet generation, content search, and as a preparatory step to improve robust content identification.

Video Segment Matching

Video Segment Matching is a software which recognizes identical video segments in different videos and determines their exact position and length. It allows broadcasting corporations, video archive operators, and content portal operators to improve content tracking, facilitate video data management, and identify copyright infringement.

Audio Forensics

Audio Forensics Tools offer effective solutions for detecting editing and manipulation in audio material, for technical quality assessment, and for subsequent recognition of processing steps and recording devices in audio production processes.

Actor Recognition

Actor Recognition is a face detection and recognition software which allows identification of individuals (celebreties, moderators, or athletes, etc.) in videos or photos. Using Actor Recognition large media and broadcasting archives can be searched through efficiently to quickly retrieve photo or video segments containing certain individuals.

Temporal Video Segmentation

The Temporal Video Segmentation detects and compiles shots, representative key frames, and scenes in movies or movie clips for browsing, exploration, management, and content presentation.

Video Motion Analysis

The Video Motion Analysis detects motion in videos, e.g. pan, tilt, zoom, or camera unsteadiness. Based on motion detection, further video analysis approaches are applied, such as visual rhythm pattern analysis, e.g. inner rhythm classification: fluid, staccato, or no motion.

Semantic Video Analysis

The semantic video analysis enables automated tagging of video segments through the classification of visual concepts or classes ranging from time of day, landscape (e.g. beach or forest) and mood. The system can be trained to analyze different concepts or class types based upon the unique needs of the customer. The semantic video analysis is also capable of generating recommendations for different concepts based upon the customer’s use. The functionality of the software is helpful for browsing or searching in large video archives.





  • Universally applicable signal analysis framework for A/V content
  • Analysis methods encapsulated in dedicated modules
  • Modular architecture supporting customization to individual requirements
  • Components available as C++ libraries for Windows, Linux and OS X for integration into existing systems
  • Evaluation versions available upon request

The A/V Analyzing Toolbox is being continuously enhanced. If you have specific requirements, or would like more information please contact us.

Social Bookmarks