EMOTION RECOGNITION IN DIGITAL ART USING DEEP LEARNING
DOI:
https://doi.org/10.29121/shodhkosh.v6.i3s.2025.6779Keywords:
Emotion Recognition, Digital Art, Deep Learning, Vision Transformers, Affective ComputingAbstract [English]
The concept of emotion recognition in visual media has been an area of increasing discussion due to the emergence of digital art as a powerful creative art form in the digital realm. Digital art in contrast to photographs can have stylized and exaggerated, or non-realistic visual qualities, which adds further complexity to affective interpretation. This paper examines a framework of deep-learning-based emotion detection in digital art using a combination of the principles of the psychological theory of emotions and computer vision advancements. We discuss the affective perception of human viewers, basing on the simple emotions described by Ekman, the wheel of emotions by Plutchik, and the circumplex model offered by Russell to form a powerful labeling system that can be applied to artistic images. This is a selective collection of digital artworks which is compiled based on multiple online collections, and then annotated systematically according to structured guidelines assembling form of reduced subjectivity. The given model is a hybrid of CNN-based local feature extraction and transformer-based global attention mechanisms that can extract both fine-grained stylistic cues and larger compositional patterns which are characteristic of digital art. The experimental findings prove that the hybrid architecture compares to standalone CNN and ViT baselines in classifying emotional categories and especially in artworks that have an abstract or non-photorealistic style.
References
Ahadit, A. B., and Jatoth, R. K. (2022). A Novel Multi-Feature Fusion Deep Neural Network using HOG and VGG-Face for Facial Expression Classification. Machine Vision and Applications, 33(2), 55. https://doi.org/10.1007/s00138-022-01277-1 DOI: https://doi.org/10.1007/s00138-022-01304-y
Chaudhari, A., Bhatt, C., Krishna, A., and Travieso-González, C. M. (2023). Facial Emotion Recognition with Inter-Modality-Attention-Transformer-Based Self-Supervised Learning. Electronics, 12(2), 288. https://doi.org/10.3390/electronics12020288 DOI: https://doi.org/10.3390/electronics12020288
Cîrneanu, A. L., Popescu, D., and Iordache, D. (2023). New Trends in Emotion Recognition using Image Analysis by Neural Networks: A Systematic Review. Sensors, 23(13), 7092. https://doi.org/10.3390/s23137092 DOI: https://doi.org/10.3390/s23167092
De Lope, J., and Grana, M. (2022). A Hybrid Time-Distributed Deep Neural Architecture for Speech Emotion Recognition. International Journal of Neural Systems, 32(2), 2250024. https://doi.org/10.1142/S0129065722500240 DOI: https://doi.org/10.1142/S0129065722500241
Hayale, W., Negi, P. S., and Mahoor, M. H. (2023). Deep Siamese Neural Networks for Facial Expression Recognition in the wild. IEEE Transactions on Affective Computing, 14(2), 1148–1158. https://doi.org/10.1109/TAFFC.2021.3102952 DOI: https://doi.org/10.1109/TAFFC.2021.3077248
Khan, A., Sohail, A., Zahoora, U., and Qureshi, A. S. (2020). A Survey of the Recent Architectures of Deep Convolutional Neural Networks. Artificial Intelligence Review, 53(8), 5455–5516. https://doi.org/10.1007/s10462-020-09825-6 DOI: https://doi.org/10.1007/s10462-020-09825-6
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Shen, Z., and Shah, M. (2022). Transformers in Vision: A Survey. ACM Computing Surveys, 54(10), 200. https://doi.org/10.1145/3505244 DOI: https://doi.org/10.1145/3505244
Kosti, R., Alvarez, J. M., Recasens, A., and Lapedriza, A. (2020). Context Based Emotion Recognition Using EMOTIC Dataset. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(11), 2755–2766. https://doi.org/10.1109/TPAMI.2019.2910520
Liu, Y., Li, Y., Yi, X., Hu, Z., Zhang, H., and Liu, Y. (2022). Lightweight ViT Model for Micro-Expression Recognition Enhanced by Transfer Learning. Frontiers in Neurorobotics, 16, 922761. https://doi.org/10.3389/fnbot.2022.922761 DOI: https://doi.org/10.3389/fnbot.2022.922761
Mylonas, P., Karkaletsis, L., and Maragoudakis, M. (2023). Convolutional Neural Networks: A Survey. Computers, 12(6), 151. https://doi.org/10.3390/computers12060151 DOI: https://doi.org/10.3390/computers12080151
Márquez, G., Singh, K., Illés, Z., He, E., Chen, Q., and Zhong, Q. (2023). SL-Swin: A Transformer-Based Deep Learning Approach for Macro- And Micro-Expression Spotting on small-size Expression Datasets. Electronics, 12(12), 2656. https://doi.org/10.3390/electronics12122656 DOI: https://doi.org/10.3390/electronics12122656
Naveen, P. (2023). Occlusion-Aware Facial Expression Recognition: A Deep Learning Approach. Multimedia Tools and Applications, 83(23), 32895–32921. https://doi.org/10.1007/s11042-023-14616-9 DOI: https://doi.org/10.1007/s11042-023-17013-1
Shabbir, N., and Rout, R. K. (2023). FgbCNN: A Unified Bilinear Architecture for Learning a Fine-Grained Feature Representation in Facial Expression Recognition. Image and Vision Computing, 137, 104770. https://doi.org/10.1016/j.imavis.2023.104770 DOI: https://doi.org/10.1016/j.imavis.2023.104770
Souza, L. S., Sogi, N., Gatto, B. B., Kobayashi, T., and Fukui, K. (2023). Grassmannian Learning Mutual Subspace Method for Image Set Recognition. Neurocomputing, 517, 20–33. https://doi.org/10.1016/j.neucom.2022.11.021 DOI: https://doi.org/10.1016/j.neucom.2022.10.040
Yang, Y., Hu, L., Zu, C., Zhou, Q., Wu, X., Zhou, J., and Wang, Y. (2023). Facial Expression Recognition with Contrastive Learning and Uncertainty-Guided Relabeling. International Journal of Neural Systems, 33(3), 2350032. https://doi.org/10.1142/S0129065723500322 DOI: https://doi.org/10.1142/S0129065723500326
Yao, H., Yang, X., Chen, D., Wang, Z., and Tian, Y. (2023). Facial Expression Recognition based on Fine-Tuned Channel–Spatial Attention Transformer. Sensors, 23(12), 6799. https://doi.org/10.3390/s23126799 DOI: https://doi.org/10.3390/s23156799
Zaman, K., Zhaoyun, S., Shah, S. M., Shoaib, M., Lili, P., and Hussain, A. (2022). Driver Emotions Recognition Based on Improved Faster R-CNN and Neural Architectural Search Network. Symmetry, 14(4), 687. https://doi.org/10.3390/sym14040687 DOI: https://doi.org/10.3390/sym14040687
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Ramneek Kelsang Bawa, Nidhi Sharma, Manisha Chandna, Lakshya Swarup, Ms. Sunitha BK, Subramanian Karthick
This work is licensed under a Creative Commons Attribution 4.0 International License.
With the licence CC-BY, authors retain the copyright, allowing anyone to download, reuse, re-print, modify, distribute, and/or copy their contribution. The work must be properly attributed to its author.
It is not necessary to ask for further permission from the author or journal board.
This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.
