Multimodal Responsive System for Human Affective State Analysis based on Physiological Signal and Human Computer Interaction

Abstract

Electroencephalography (EEG) is used to record brain signals, and it has received major attention in the Affective Computing domain since it is a simple solution for human emotion recognition. EEG has influenced the field of Affective Computing research, indicating that it can represent human emotions more precisely as compared to facial expressions, text, body gestures, or speech signals. The human brain signals generated during emotional activities can be recorded and analyzed to understand the state of mind. Emotion classification with machine learning includes the selection and extraction of various emotion-related features taken from EEG signals. Out of these, features considered from the frequency domain are significant and exist in most studies related to human emotion. Feature extraction from EEG signals includes transformation from one domain to another followed by classification through various machine learning models. This work focuses on the detailed study of EEG-based human emotion recognition using parameters such as emotion models, EEG devices or datasets, features, methods of feature extraction, and classifiers. The major contribution of this research work is to address the challenge of proper feature selection and accurately classifying emotions based on real-time EEG data. The proposed system compares time, frequency and wavelet domain features. Fast Fourier Transform is a feature extraction method in frequency domain that is used to extract the band power of each channel, based on which the emotion classification is done. Random Forest classifier gave a maximum accuracy of 97% for classifying four emotions named happy, sad, calm, and stressed for test-train dataset split in the ratio of 80:20 and 90:10.K-Nearest Neighbor performed quiet well with 94% accuracy followed by Decision Tree that had 93% accuracy. Wavelet domain features had 89% accuracy with Random Forest consistently performing best among classifiers as compared to Support Vector Machine, K Nearest Neighbor, Gradient Boosting and Decision Tree, though still not matching the frequency domain results. With the achievement of higher accuracy, this work is one of the few efforts to accurately understand a person s affective state which can solve many problems related to social and personal aspects of life.