Semantic Enriched Lecture Video Retrieval using Machine Learning Techniques

Abstract

Retrieving relevant information from a large collection of videos is a tedious and newlinetime-consuming process for a user. The main objective of this research is to enhance the efficiency of lecture video retrieval by incorporating semantics and data mining techniques. Videos are processed to identify keyframes and from each keyframe the text and texture features are extracted. Texts are recognized using Tesseract Optical Character Recognition (OCR) which is considered as one of the most accurate open-source OCR engine and texture features are extracted from keyframes using Gabor Ordinal Measure (GOM). Semantic words are also identified for each extracted word using WordNet. A feature database is created which contains the semantic words, text and texture features. Videos are grouped based on the similarities of their features using k-means clustering, to speed up the retrieval process. From the clustered features, relevant videos are retrieved using a combination of correlation and Naïve Bayes classification techniques. Domain ontologies are created for each category of videos in the database by a domain expert. Each keyword from the feature database is mapped with the keywords in the domain ontology. These keywords are used for annotating the videos. Semantic annotation of videos enhances the lecture video retrieval. Deep learning strategies are used to further improve the process of classification. Deep Belief Network (DBN) classification is used for retrieving the relevant videos which improve the accuracy of retrieval results. Whenever a query is given by the user, the classifier identifies the optimal cluster centroid related to the query. The performance of the proposed techniques are evaluated by considering text query and video query. Three evaluation metrics, namely, recall, F-measure, and precision are considered for analyzing the performance of retrieval results. The experimental results have proved that after incorporating the semantic and data miningtechniques to the video retrieval process the informati