e-ISSN 2231-8526
ISSN 0128-7680
Sarthak Kapaliya, Debabrata Swain, Ritu Sharma, Kanishka Varyani and Jyoti Thakar
Pertanika Journal of Science & Technology, Volume 32, Issue 5, August 2024
DOI: https://doi.org/10.47836/pjst.32.5.02
Keywords: Deep learning, emotion, facial emotion, human-computer interaction, image classification, neural networks
Published on: 26 August 2024
Emotions play a significant role in both verbal and nonverbal communication. Facial emotion recognition has applications in various sectors where we can get real-time feedback about student activeness by detecting their expression. In this paper, we aim to provide an improved deep-learning technique to detect emotions by using publicly available datasets to perform this detection. To get more data for the well-being of the Machine Learning Model, we have used data augmentation using the TensorFlow framework. Visual Geometry Group-16 (VGG16) is a convolutional neural network of 16 layers deep. There has been an alteration to the default VGG16 structure to get better classification results. Various optimization algorithms and loss functions increase the model’s accuracy. We have used many evaluation parameters from the technical side, like precision, accuracy, recall, Area Under the Receiver Operating Characteristic Curve (AUC), and F1 Score. The proposed model has an accuracy of 89% while having a precision of 81 percent for classification. We have achieved an F1 Score of 0.42 and an area under the ROC curve (AUC) of 0.734. Overall, it would be beneficial for analyzing and categorizing positive and negative emotions, which would aid in detecting signs of stress, anxiety, and burnout, as well as taking preventive actions to enhance well-being.
Alamsyah, T. M. S. N., Abidin, T. F., Ferdhiana, R., Dirhamsyah, M., & Chaidir, M. (2022, December 8-9). Analysis of face data augmentation in various poses for face recognition model. [Paper presentation]. Seventh International Conference on Informatics and Computing (ICIC), Bali, Indonesia. https://doi.org/10.1109/ICIC56845.2022.10006997
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
Darwin, C. (1872). The expression of the emotions in man and animals. John Murray. https://doi.org/10.1037/10001-000
Devi, B., & Preetha, M. M. S. J. (2023). A descriptive survey on face emotion recognition techniques. International Journal of Image and Graphics, 23(1), Article 2350008. https://doi.org/10.1142/S0219467823500080
Ekman, P. (2009). Darwin’s contributions to our understanding of emotional expressions. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3449–3451. https://doi.org/10.1098/rstb.2009.0189
Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129. https://doi.org/10.1037/h0030377
Garcia-Garcia, J. M., Penichet, V. M. R., Lozano, M. D., & Fernando, A. (2022). Using emotion recognition technologies to teach children with autism spectrum disorder how to identify and express emotions. Universal Access in the Information Society, 21(4), 809–825. https://doi.org/10.1007/s10209-021-00818-y
Jaymon, N., Nagdeote, S., Yadav, A., & Rodrigues, R. (2021, February 19-20). Real-time emotion detection using deep learning. [Paper presentation]. International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), Bhilai, India. https://doi.org/10.1109/ICAECT49130.2021.9392584
Mandal, M. K. (2004). Emotion. Affiliated East-West Press.
Ortony, A. (2022). Are all “basic emotions” emotions? A problem for the (Basic) emotions construct. Perspectives on Psychological Science, 17(1), 41–61. https://doi.org/10.1177/1745691620985415
Pabst, A., Bollen, Z., Masson, N., Billaux, P., Timary, P. D., & Maurage, P. (2023). An eye-tracking study of biased attentional processing of emotional faces in severe alcohol use disorder. Journal of Affective Disorders, 323, 778–787. https://doi.org/10.1016/j.jad.2022.12.027
Salve, P. (2022, October 3). India’s suicide rate has increased. But is it because of better reporting or rising distress? Croll.in. https://scroll.in/article/1034045/indias-suicide-rate-has-increased-but-is-it-because-of-better-reporting-or-rising-distress
Saini, A., Khaparde, A. R., Kumari, S., Shamsher, S., Joteeswaran, J., & Kadry, S. (2023). An investigation of machine learning techniques in speech emotion recognition. Indonesian Journal of Electrical Engineering and Computer Science, 29(2), Article 875. https://doi.org/10.11591/ijeecs.v29.i2.pp875-882
Shank, D. B. (2014). Technology and emotions. In J. E. Stets & J. H. Turner (Eds.), Handbook of the Sociology of Emotions: Volume II (pp. 511–528). Springer. https://doi.org/10.1007/978-94-017-9130-4_24
Siam, A. I., Soliman, N. F., Algarni, A. D., El-Samie, F. E. A., & Sedik, A. (2022). Deploying machine learning techniques for human emotion detection. Computational Intelligence and Neuroscience, 2022, Article 8032673. https://doi.org/10.1155/2022/8032673
Song, S., Zhao, S., Gao, Z., Lu, M., Zhang, M., Gao, S., & Zheng, Y. (2022). Influence of affective verbal context on emotional facial expression perception of social anxiety. International Journal of Psychophysiology, 181, 141–149. https://doi.org/10.1016/j.ijpsycho.2022.09.002
Song, Z. (2021). Facial expression emotion recognition model integrating philosophy and machine learning theory. Frontiers in Psychology, 12, Article 759485. https://doi.org/10.3389/fpsyg.2021.759485
Subhashini, R., & Niveditha, P. R. (2015). Analyzing and detecting employee’s emotions for amelioration of organizations. Procedia Computer Science, 48, 530–536. https://doi.org/10.1016/j.procs.2015.04.131
ISSN 0128-7680
e-ISSN 2231-8526