Behaviour Extraction Using Social Media Analytics

Abstract

newlineThe widespread adoption of opinion mining and sentiment analysis in higher newlinecognitive processes encourages the need for real-time processing of social media data to newlinecapture insights about sentiment analysis, user ranking, and current trends of the newlinedomain. In data analytics, only 30% of the time is consumed in modeling and evaluation newlinestages, and the rest 70% of the time is consumed in data engineering tasks. In recent newlineyears lots of researches were conducted and various machine learning algorithms were newlinedeveloped around the processing of data to achieve higher accuracy but it lacks in newlinehandling data and its transformations so-called data engineering tasks while reducing the newlineprocessing time is still challenging. Big data technologies came to handle these newlinechallenges but it have its own set of complexities along with having hardware newlinedeadweight on the system. Thus, the need for simple and lightweight frameworks arises newlineto process these posts effectively and eliminate the aforementioned challenges. The newlinethesis has contributed to design various such frameworks to process social media posts newlineeffectively in order to extract the insights. Additionally, these frameworks were newlineevaluated on various use cases like sentiment analysis, current trend identification, and newlineuser ranking in a specific domain or a topic. The frameworks mentioned in this thesis newlinehave utilized Hadoop Eco-systems as well as modern technologies and databases to newlinemake it suitable for small-scale to large-scale applications. The effectiveness and newlinebenchmarking of the proposed frameworks is also measured by comparing the results newlinewith existing frameworks. newlineThis thesis has contributed to design a Redis and Elasticsearch based parallelly newlinescalable, effective, responsive, and fault-tolerant framework to perform end-to-end data newlineanalytics tasks in a real-time and batch processing manner. Experimental analysis on newlineTwitter posts supported the claims and signifies the benefits of parallelism of data newlineprocessing units. This work has highlighted the importance of processing m