A/V Analyzing Toolbox

Automated Audio and Video Analysis

A/V Analyzing Toolbox is a modular content analysis framework which allows comprehensive automated processes to be applied to audio-visual content for the optimization of broadcasting, distribution and archiving workflows. It inspects A/V data with respect to technical and perceptual parameters and extracts metadata, as well as semantic information. A/V Analyzing Toolbox provides dedicated analyzing components for several uses cases and application scenarios.

QC Components

The A/V Error Detection Libraries provide automatic detection of audio-visual errors and quality issues for A/V production, content management, and archiving. Since the detections run on signal level, the technology can be applied at any position within a process chain, e.g. ingest, play-out, transcoding, etc.

Audio Forensics Toolbox

Recording, editing and coding steps usually leave characteristic traces within audio material. The Audio Forensics Toolbox developed by Fraunhofer IDMT allows users to analyze such traces and detect the reuse of material. Broadcasters, media producers, and media archives may use the Toolbox to conduct quality assessments, facilitate more efficient annotation of metadata, and detect unintended or intended editing of audio material.

Music and Speech Detection

The automatic Music and Speech Detection segments audio content with respect to containing speech, music, speech and background music, and silence, which can be used for automatic music royalty reporting, cue sheet generation, content search, and as a preparatory step to improve robust content identification.

Video Segment Matching

Video Segment Matching is a software which recognizes identical video segments in different videos and determines their exact position and length. It allows broadcasting corporations, video archive operators, and content portal operators to improve content tracking, facilitate video data management, and identify copyright infringement.

Audio Segment Matching

Audio segment matching is a technology that detects identical audio segments/matches among audio items or streams. It can be used for duplicate detection and removal, metadata and rights tracking within and across archives, and to detect content reuse e.g. for media analysis. Audio phylogeny, on the other hand, is about analyzing the processing history of content, which can be useful for duplicate detection because it helps to identify original versus derived versions e.g. created by transcoding or reencoding within a set of duplicates (in order to decide which version should be kept and which can be safely removed.

Actor Recognition

Actor Recognition is a face detection and recognition software which allows identification of individuals (celebreties, moderators, or athletes, etc.) in videos or photos. Using Actor Recognition large media and broadcasting archives can be searched through efficiently to quickly retrieve photo or video segments containing certain individuals.

Temporal Video Segmentation

The Temporal Video Segmentation automatically detects and compiles shots, representative key frames, and scenes in movies or movie clips for browsing, exploration, management, and content presentation.

Video Motion Analysis

The Video Motion Analysis detects global motions in videos, e.g. camera motions. Further video analysis approaches are applied, such as visual rhythm pattern analysis, and motion tempo detection.

Semantic Video Analysis

The semantic video analysis enables automated tagging of video segments through the classification of visual concepts or classes ranging from time of day, landscape (e.g. beach or forest) and mood. The system can be trained to analyze different concepts or class types based upon the unique needs of the customer. The semantic video analysis is also capable of generating recommendations for different concepts based upon the customer’s use. The functionality of the software is helpful for browsing or searching in large video archives.

Business Unit Media Management and Delivery

More information on products and services of the business unit Media Management & Delivery