PERTANIKA JOURNAL OF SCIENCE AND TECHNOLOGY

Movie reviews provide a medium of communication for the movie fans community. Movie reviews not only help viewers and potential viewers to obtain a general opinion about a movie but also allow the fans to construct an opinion of the movie. In this work, an analysis of over 60,000 movie reviews has been implemented to find meaningful text representation via text embedding. We improved the text embedding by proposing an attention-based Bidirectional Long-Short Term Memory (Bi-LSTM) network by using over 60,000 movie review text data as the training set and over 20,000 movie review text data as the testing set. Based on the data features, we performed a probabilistic analysis of the information related to words and phrases, combined the analysis results with text embedding, spatialized the text embedding, and compared the performance of the proposed attention-based spatialized word embedding Bi-LSTM model with several traditional machine learning models. The attention-based spatialized word embedding Bi-LSTM model proposed in this paper achieves an F1 score of 0.91 on the movie review sentiment classification dataset, with a prediction accuracy of 91%, outperforming the results of the current state-of-the-art research. The model can effectively identify the sentimental tendencies of movie reviews and use the analyzed sentimental tendencies to guide consumers in their consumption and obtain feedback on movie content.

• Abdullah, N. A. S., & Rusli, N. I. A. (2021). Multilingual sentiment analysis: A systematic literature review. Pertanika Journal of Science and Technology, 29(1), 445-470. https://doi.org/10.47836/pjst.29.1.25

• AlKhwiter, W., & Al-Twairesh, N. (2021). Part-of-speech tagging for Arabic tweets using CRF and Bi-LSTM. Computer Speech and Language, 65, Article 101138. https://doi.org/10.1016/j.csl.2020.101138

• Asghar, M. Z., Habib, A., Habib, A., Khan, A., Ali, R., & Khattak, A. (2021). Exploring deep neural networks for rumor detection. Journal of Ambient Intelligence and Humanized Computing, 12(4), 4315-4333. https://doi.org/10.1007/s12652-019-01527-4

• Behera, R. K., Jena, M., Rath, S. K., & Misra, S. (2021). Co-LSTM: Convolutional LSTM model for sentiment analysis in social big data. Information Processing and Management, 58(1), Article 102435. https://doi.org/10.1016/j.ipm.2020.102435

• Briskilal, J., & Subalalitha, C. N. (2022). An ensemble model for classifying idioms and literal texts using BERT and RoBERTa. Information Processing and Management, 59(1), Article 102756. https://doi.org/10.1016/j.ipm.2021.102756

• Chen, W., Zhang, Y., Yeo, C. K., Lau, C. T., & Lee, B. S. (2018). Unsupervised rumor detection based on users’ behaviors using neural networks. Pattern Recognition Letters, 105, 226-233. https://doi.org/10.1016/j.patrec.2017.10.014

• Chen, Y. (2015). Convolutional Neural Network for Sentence Classification (Unpublished Master’s thesis). University of Waterloo, Canada. https://uwspace.uwaterloo.ca/handle/10012/9592

• Cheng, Y., Yao, L., Xiang, G., Zhang, G., Tang, T., & Zhong, L. (2020). Text sentiment orientation analysis based on multi-channel CNN and bidirectional GRU with attention mechanism. IEEE Access, 8, 134964-134975. https://doi.org/10.1109/ACCESS.2020.3005823

• Chiny, M., Chihab, M., Chihab, Y., & Bencharef, O. (2021). LSTM, VADER and TF-IDF based hybrid sentiment analysis model. International Journal of Advanced Computer Science and Applications, 12(7), 265-275. https://doi.org/10.14569/IJACSA.2021.0120730

• Dai, Y., Guo, W., Chen, X., & Zhang, Z. (2018). Relation classification via LSTMs based on sequence and tree structure. IEEE Access, 6, 64927-64937. https://doi.org/10.1109/ACCESS.2018.2877934

• Demotte, P., Senevirathne, L., Karunanayake, B., Munasinghe, U., & Ranathunga, S. (2020). Sentiment analysis of Sinhala news comments using sentence-state LSTM networks. In MERCon 2020 - 6th International Multidisciplinary Moratuwa Engineering Research Conference (pp. 283-288). IEEE Publishing. https://doi.org/10.1109/MERCon50084.2020.9185327

• Fernandes, B., & Mannepalli, K. (2021a). An analysis of emotional speech recognition for tamil language using deep learning gate recurrent unit. Pertanika Journal of Science and Technology, 29(3), 1937-1961. https://doi.org/10.47836/pjst.29.3.37

• Fernandes, B., & Mannepalli, K. (2021b). Speech emotion recognition using deep learning LSTM for tamil language. Pertanika Journal of Science and Technology, 29(3), 1915-1936. https://doi.org/10.47836/pjst.29.3.33

• Giatsoglou, M., Vozalis, M. G., Diamantaras, K., Vakali, A., Sarigiannidis, G., & Chatzisavvas, K. C. (2017). Sentiment analysis leveraging emotions and word embeddings. Expert Systems with Applications, 69, 214-224. https://doi.org/10.1016/j.eswa.2016.10.043

• Gu, W., Tandon, A., Ahn, Y. Y., & Radicchi, F. (2021). Principled approach to the selection of the embedding dimension of networks. Nature Communications, 12(1), 1-10. https://doi.org/10.1038/s41467-021-23795-5

• Gupta, C., Chawla, G., Rawlley, K., Bisht, K., & Sharma, M. (2021). Senti_ALSTM: Sentiment analysis of movie reviews using attention-based-LSTM. In Proceedings of 3rd International Conference on Computing Informatics and Networks: ICCIN 2020 (pp. 211-219). Springer. https://doi.org/10.1007/978-981-15-9712-1_18

• Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1007/978-1-4757-5388-2_2

• Islam, M. U., Hossain, M. M., & Kashem, M. A. (2021). COVFake: A word embedding coupled with LSTM approach for COVID related fake news detection. International Journal of Computer Applications, 174(10), 1-5. https://doi.org/10.5120/ijca2021920977

• Jain, P. K., Saravanan, V., & Pamula, R. (2021). A hybrid CNN-LSTM: A deep learning approach for consumer sentiment analysis using qualitative user-generated contents. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(5), Article 84. https://doi.org/10.1145/3457206

• Jianqiang, Z., Xiaolin, G., & Xuejun, Z. (2018). Deep convolution neural networks for twitter sentiment analysis. IEEE Access, 6, 23253-23260. https://doi.org/10.1109/ACCESS.2017.2776930

• Kumar, A., Verma, S., & Sharan, A. (2021). ATE-SPD: Simultaneous extraction of aspect-term and aspect sentiment polarity using Bi-LSTM-CRF neural network. Journal of Experimental and Theoretical Artificial Intelligence, 33(3), 487-508. https://doi.org/10.1080/0952813X.2020.1764632

• Kumar, K., Harish, B. S., & Darshan, H. K. (2019). Sentiment analysis on IMDb movie reviews using hybrid feature extraction method. International Journal of Interactive Multimedia and Artificial Intelligence, 5(5), Article 109. https://doi.org/10.9781/ijimai.2018.12.005

• Lai, S., Xu, L., Liu, K., & Zhao, J. (2015). Recurrent convolutional neural networks for text classification. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 29, No. 1). AAAI Press. https://doi.org/10.1609/aaai.v29i1.9513

• Lee, J., Seo, S., & Choi, Y. S. (2019). Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing. Symmetry, 11(6), Article 785. https://doi.org/10.3390/sym11060785

• Leng, X. L., Miao, X. A., & Liu, T. (2021). Using recurrent neural network structure with enhanced multi-head self-attention for sentiment analysis. Multimedia Tools and Applications, 80(8), 12581-12600. https://doi.org/10.1007/s11042-020-10336-3

• Li, W., Liu, P., Zhang, Q., & Liu, W. (2019). An improved approach for text sentiment classification based on a deep neural network via a sentiment attention mechanism. Future Internet, 11(4), Article 96. https://doi.org/10.3390/FI11040096

• Li, W., Qi, F., Tang, M., & Yu, Z. (2020). Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification. Neurocomputing, 387, 63-77. https://doi.org/10.1016/j.neucom.2020.01.006

• Lim, C. T., Bong, C. H., Wong, W. S., & Lee, N. K. (2021). A comprehensive review of automated essay scoring (AES) research and development. Pertanika Journal of Science and Technology, 29(3), 1875-1899. https://doi.org/10.47836/pjst.29.3.27

• Liu, B. (2012). Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1), 1-167. https://doi.org/10.1142/9789813100459_0007

• Liu, Y., Jin, X., & Shen, H. (2019). Towards early identification of online rumors based on long short-term memory networks. Information Processing and Management, 56(4), 1457-1467. https://doi.org/10.1016/j.ipm.2018.11.003

• Mondal, T., Pramanik, P., Bhattacharya, I., Boral, N., & Ghosh, S. (2018). Analysis and early detection of rumors in a post disaster scenario. Information Systems Frontiers, 20(5), 961-979. https://doi.org/10.1007/s10796-018-9837-8

• Muhammad, P. F., Kusumaningrum, R., & Wibowo, A. (2021). Sentiment analysis using Word2vec and long short-term memory (LSTM) for Indonesian hotel reviews. Procedia Computer Science, 179, 728-735. https://doi.org/10.1016/j.procs.2021.01.061

• Munshi, A. A., AlSabban, W. H., Farag, A. T., Rakha, O. E., Al Sallab, A., & Alotaibi, M. (2022). Automated Islamic jurisprudential legal opinions generation using artificial intelligence. Pertanika Journal of Science and Technology, 30(2), 1135-1156. https://doi.org/10.47836/pjst.30.2.16

• Nasukawa, T., & Yi, J. (2003). Sentiment analysis: Capturing favorability using natural language processing. In Proceedings of the 2nd International Conference on Knowledge Capture (pp. 70-77). John Wiley & Sons. https://doi.org/10.1111/j.1469-185X.1956.tb01550.x

• Nayak, S. K., Rout, P. K., Jagadev, A. K., & Swarnkar, T. (2018). Elitism-based multi-objective differential evolution with extreme learning machine for feature selection: A novel searching technique. Connection Science, 30(4), 362-387. https://doi.org/10.1080/09540091.2018.1487384

• Onan, A. (2018a). An ensemble scheme based on language function analysis and feature engineering for text genre classification. Journal of Information Science, 44(1), 28-47. https://doi.org/10.1177/0165551516677911

• Onan, A. (2018b). Biomedical text categorization based on ensemble pruning and optimized topic modelling. Computational and Mathematical Methods in Medicine, 2018, Article 2497471. https://doi.org/10.1155/2018/2497471

• Onan, A. (2019a). Consensus clustering-based undersampling approach to imbalanced learning. Scientific Programming, 2019, Article 5901087. https://doi.org/10.1155/2019/5901087

• Onan, A. (2019b). Topic-enriched word embeddings for sarcasm identification. In Software Engineering Methods in Intelligent Algorithms: Proceedings of 8th Computer Science Online Conference (pp. 293-304). Springer. https://doi.org/10.1007/978-3-030-19807-7_29

• Onan, A. (2019c). Two-stage topic extraction model for bibliometric data analysis based on word embeddings and clustering. IEEE Access, 7, 145614-145633. https://doi.org/10.1109/ACCESS.2019.2945911

• Onan, A. (2020). Mining opinions from instructor evaluation reviews: A deep learning approach. Computer Applications in Engineering Education, 28(1), 117-138. https://doi.org/10.1002/cae.22179

• Onan, A. (2021a). Sentiment analysis on massive open online course evaluations: A text mining and deep learning approach. Computer Applications in Engineering Education, 29(3), 572-589. https://doi.org/10.1002/cae.22253

• Onan, A. (2021b). Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks. Concurrency and Computation: Practice and Experience, 33(23), Article e5909. https://doi.org/10.1002/cpe.5909

• Onan, A. (2022). Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification. Journal of King Saud University - Computer and Information Sciences, 34(5), 2098-2117. https://doi.org/10.1016/j.jksuci.2022.02.025

• Onan, A., Bulut, H., & Korukoǧlu, S. (2017). An improved ant algorithm with LDA-based representation for text document clustering. Journal of Information Science, 43(2), 275-292. https://doi.org/10.1177/0165551516638784

• Onan, A., & Korukoǧlu, S. (2017). A feature selection model based on genetic rank aggregation for text sentiment classification. Journal of Information Science, 43(1), 25-38. https://doi.org/10.1177/0165551515613226

• Onan, A., Korukoğlu, S., & Bulut, H. (2017). A hybrid ensemble pruning approach based on consensus clustering and multi-objective evolutionary algorithm for sentiment classification. Information Processing and Management, 53(4), 814-833. https://doi.org/10.1016/j.ipm.2017.02.008

• Onan, A., & Tocoglu, M. A. (2021). A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification. IEEE Access, 9, 7701-7722. https://doi.org/10.1109/ACCESS.2021.3049734

• Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. In EMNLP ‘02: Proceedings of the ACL-02 conference on Empirical methods in natural language processing (pp. 79-86). ACM Publishing. https://doi.org/10.3115/1118693.1118704

• Ranathunga, S., & Liyanage, I. U. (2021). Sentiment analysis of Sinhala news comments. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(4), Article 59. https://doi.org/10.1145/3445035

• Rani, S., & Kumar, P. (2019). Deep learning based sentiment analysis using convolution neural network. Arabian Journal for Science and Engineering, 44(4), 3305-3314. https://doi.org/10.1007/s13369-018-3500-z

• Rasool, A., Jiang, Q., Qu, Q., & Ji, C. (2021). WRS: A novel word-embedding method for real-time sentiment with integrated LSTM-CNN model. In 2021 IEEE International Conference on Real-Time Computing and Robotics (RCAR) (pp. 590-595). IEEE Publishing. https://doi.org/10.1109/RCAR52367.2021.9517671

• Sarzynska-Wawer, J., Wawer, A., Pawlak, A., Szymanowska, J., Stefaniak, I., Jarkiewicz, M., & Okruszek, L. (2021). Detecting formal thought disorder by deep contextualized word representations. Psychiatry Research, 304, Article 114135. https://doi.org/10.1016/j.psychres.2021.114135

• Shen, Y., & Huang, X. J. (2016). Attention-based convolutional neural network for semantic relation extraction. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp. 2526-2536). The COLING 2016 Organizing Committee.

• Shrivastava, K., Kumar, S., & Jain, D. K. (2019). An effective approach for emotion detection in multimedia text data using sequence based convolutional neural network. Multimedia Tools and Applications, 78(20), 29607-29639. https://doi.org/10.1007/s11042-019-07813-9

• Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012). Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (pp. 1201-1211). Association for Computational Linguistics.

• Solanki, V. K., Cuong, N. H. H., & Lu, Z. J. (2019). Opinion mining: using machine learning techniques. In Extracting Knowledge from Opinion Mining (pp. 66-82). IGI Global. https://doi.org/10.4018/978-1-5225-6117-0.ch004

• Tan, T. P., Lim, C. K., & Rahman, W. R. E. A. (2022). Sliding window and parallel LSTM with attention and CNN for sentence alignment on low-resource languages. Pertanika Journal of Science and Technology, 30(1), 97-121. https://doi.org/10.47836/pjst.30.1.06

• Wang, Q., Zhu, G., Zhang, S., Li, K., Chen, X., & Xu, H. (2020). Extending emotional lexicon for improving the classification accuracy of Chinese film reviews. Connection Science, 33(2), 153-172. https://doi.org/10.1080/09540091.2020.1782839

• Wei, J., & Zou, K. (2019). EDA: Easy data augmentation techniques for boosting performance on text classification tasks. ArXiv Preprint. https://doi.org/10.18653/v1/d19-1670

• Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 1480-1489). Association for Computational Linguistics.

• Zhen, F., Yi, G., Zhenhao, Z., & Meiqi, H. (2018). Sentiment analysis of movie reviews based on dictionary and weak tagging information. Journal of Computer Applications, 38(11), 3084-3088.