A/V Analyzing Toolbox
Fraunhofer Institute for Digital Media Technology
- © istock.com/Anthony Brown
Automated Audio and Video Analysis
A/V Analyzing Toolbox is a modular content analysis framework which allows comprehensive automated processes to be applied to audio-visual content for the optimization of broadcasting, distribution and archiving workflows. It inspects A/V data with respect to technical and perceptual parameters and extracts metadata, as well as semantic information. A/V Analyzing Toolbox provides dedicated analyzing components for several uses cases and application scenarios.
The A/V Error Detection Libraries provide automatic detection of audio-visual errors and quality issues for A/V production, content management, and archiving. Since the detections run on signal level, the technology can be applied at any position within a process chain, e.g. ingest, play-out, transcoding, etc.
Audio Forensics Toolbox
Recording, editing and coding steps usually leave characteristic traces within audio material. The Audio Forensics Toolbox developed by Fraunhofer IDMT allows users to analyze such traces and detect the reuse of material. Broadcasters, media producers, and media archives may use the Toolbox to conduct quality assessments, facilitate more efficient annotation of metadata, and detect unintended or intended editing of audio material.
Music and Speech Detection
The automatic Music and Speech Detection segments audio content with respect to containing speech, music, speech and background music, and silence, which can be used for automatic music royalty reporting, cue sheet generation, content search, and as a preparatory step to improve robust content identification.
Video Segment Matching
Video Segment Matching is a software which recognizes identical video segments in different videos and determines their exact position and length. It allows broadcasting corporations, video archive operators, and content portal operators to improve content tracking, facilitate video data management, and identify copyright infringement.
Actor Recognition is a face detection and recognition software which allows identification of individuals (celebreties, moderators, or athletes, etc.) in videos or photos. Using Actor Recognition large media and broadcasting archives can be searched through efficiently to quickly retrieve photo or video segments containing certain individuals.
Temporal Video Segmentation
The Temporal Video Segmentation automatically detects and compiles shots, representative key frames, and scenes in movies or movie clips for browsing, exploration, management, and content presentation.
Video Motion Analysis
The Video Motion Analysis detects global motions in videos, e.g. camera motions. Further video analysis approaches are applied, such as visual rhythm pattern analysis, and motion tempo detection.
The semantic video analysis enables automated tagging of video segments through the classification of visual concepts or classes ranging from time of day, landscape (e.g. beach or forest) and mood. The system can be trained to analyze different concepts or class types based upon the unique needs of the customer. The semantic video analysis is also capable of generating recommendations for different concepts based upon the customer’s use. The functionality of the software is helpful for browsing or searching in large video archives.