A/V Analyzing Toolbox
Fraunhofer Institute for Digital Media Technology
- © istock.com/Anthony Brown
Automated Audio and Video Analysis
A/V Analyzing Toolbox is a modular content analysis framework which allows comprehensive automated processes to be applied to audio-visual content for the optimization of broadcasting, distribution and archiving workflows. It inspects A/V data with respect to technical and perceptual parameters and extracts metadata, as well as semantic information. A/V Analyzing Toolbox provides dedicated analyzing components for several uses cases and application scenarios.
The A/V Error Detection Libraries provide automatic detection of audio-visual errors and quality issues for A/V production, content management, and archiving. Since the detections run on signal level, the technology can be applied at any position within a process chain, e.g. ingest, play-out, transcoding, etc.
Speech and Music Detection
The automatic Speech and Music Detection segments audio content with respect to containing speech, music, speech and background music, and silence, which can be used for automatic music royalty reporting, cue sheet generation, content search, and as a preparatory step to improve robust content identification.
Video Segment Matching
Video Segment Matching is a software which recognizes identical video segments in different videos and determines their exact position and length. It allows broadcasting corporations, video archive operators, and content portal operators to improve content tracking, facilitate video data management, and identify copyright infringement.
Audio Forensics Tools offer effective solutions for detecting editing and manipulation in audio material, for technical quality assessment, and for subsequent recognition of processing steps and recording devices in audio production processes.
Actor Recognition is a face detection and recognition software which allows identification of individuals (celebreties, moderators, or athletes, etc.) in videos or photos. Using Actor Recognition large media and broadcasting archives can be searched through efficiently to quickly retrieve photo or video segments containing certain individuals.
Temporal Video Segmentation
The Temporal Video Segmentation automatically detects and compiles shots, representative key frames, and scenes in movies or movie clips for browsing, exploration, management, and content presentation.
Video Motion Analysis
The Video Motion Analysis detects global motions in videos, e.g. camera motions. Further video analysis approaches are applied, such as visual rhythm pattern analysis, and motion tempo detection.
The semantic video analysis enables automated tagging of video segments through the classification of visual concepts or classes ranging from time of day, landscape (e.g. beach or forest) and mood. The system can be trained to analyze different concepts or class types based upon the unique needs of the customer. The semantic video analysis is also capable of generating recommendations for different concepts based upon the customer’s use. The functionality of the software is helpful for browsing or searching in large video archives.