Multi Modal Dynamic System For Generic and Personalized Video Summarization Using Machine Learning

Abstract

Video Summarization is a video compression and video compaction technique to newlinecreate a shorter yet informative version of the original video. It is very important to account for video compaction techniques to handle the storage and space complexities of raw video. newlineThis thesis presents multiple solutions to fulfill these gaps concerning automatic Cricket video summarization. The first contribution of this thesis provides a unified framework for newlinesummarization based on different criteria and also compares different literature work newlinerelated to video summarization. newlineThe framework deals with formulating a model for video summarization based on different newlinecriteria. This framework works as a single solution to all types of video summarization newlineaspects. newlineThe second contribution of this thesis includes a multi-modal, dynamic and generic newlinevideo summarization approach to summarize Cricket sport videos using domain newlineknowledge acquired from multi-modal information and audio-visual cues. This thesis includes two novel curated datasets in the form of an image dataset (named DPCS dataset) and an audio-based dataset (named EXINP Cricket audio dataset) to serve as a standard and benchmark for video segmentation, key segment detection, and video summarization in Cricket sports videos respectively. Another contribution presents a stacked Convolutional Neural Network (CNN) approach for personalized and dynamic video summarization by embedding action-based and rule-based domain knowledge of Cricket sport. newlineA Multi-CNN approach for dynamic and personalized Video Summarization is newlinepresented. The proposed approach is grounded on the Cricket Sport domain knowledge newlineto learn complex and domain features. This thesis also proposes two novel summary evaluation metrics based on user reactions, namely, User Rating Score and Composite Summary Score (CS-Score) to evaluate personalized video summary. newlineThis thesis establishes a standard and benchmark platform for research in the newlinegeneric, personalized and dynamic video summarization domain. newline