AI-BASED SEMANTIC TAGGING OF MODERN ART COLLECTIONS
DOI:
https://doi.org/10.29121/shodhkosh.v6.i3s.2025.6777Keywords:
Semantic Tagging, Modern Art, Multimodal Learning, Ontology Alignment, Knowledge Graph, Digital Curation, Explainable AIAbstract [English]
The proposed research will employ deep learning through multimodality and ontology-based reasoning to generate metadata of a modern art collection using an AI-based semantic tagging system. The system uses Vision Transformers (ViT) and BERT/CLIP encoders to pull out visual and textual embeddings and combine them using a contrastive learning method, and align them with cultural ontologies, like the CIDOC-CRM and the Art and Architecture Thesaurus (AAT). The resulting tags, an ontology-related, include aesthetic, emotional and conceptual aspects of works of art beyond traditional metadata domains. The quality of experimental results on 60,000-image dataset show that impressive gains are made over the baseline models, with Precision = 0.89, mAP = 0.88, and a Semantic Consistency Index of 0.91 showing that it is the most accurate model in terms of context and curatorial. Knowledge graph integration also allows cross-collection reasoning, smart retrieval and visually depicting curating. Cultural sensitivity and interpretive integrity is guaranteed by the ethical protection measures, such as human-in-the-loop validation and bias mitigation. The suggested framework opens the road to explainable and inclusive AI in the context of digital heritage, recreating the nature of modern art cataloguing, analysis, and experience in museum and research setting.
References
Bonaduce, I., Duce, C., Lluveras-Tenorio, A., Lee, J., Ormsby, B., Burnstock, A., and Van den Berg, K. J. (2019). Conservation issues of modern oil paintings: A Molecular Model on Paint Curing. Accounts of Chemical Research, 52, 3397-3406. https://doi.org/10.1021/acs.accounts.9b00296 DOI: https://doi.org/10.1021/acs.accounts.9b00296
Bunz, M. (2023). The role of culture in the intelligence of AI. In S. Thiel and J. Bernhardt (Eds.), AI in Museums: Reflections, perspectives and applications (23-29). Transcript. https://doi.org/10.1515/9783839467107-003 DOI: https://doi.org/10.1515/9783839467107-003
Engel, C., Mangiafico, P., Issavi, J., and Lukas, D. (2019). Computer Vision and Image Recognition in Archaeology. In Proceedings of the Conference on Artificial Intelligence for Data Discovery and Reuse (AIDR '19) (Article 5, 1-4). Association for Computing Machinery. https://doi.org/10.1145/3359115.3359117 DOI: https://doi.org/10.1145/3359115.3359117
Fuchsgruber, L. (2023). Dead end or way out? Generating Critical Information about Painting Collections with AI. In S. Thiel andJ. Bernhardt (Eds.), AI in Museums: Reflections, perspectives and applications (65-72). Transcript. https://doi.org/10.1515/9783839467107-007 DOI: https://doi.org/10.1515/9783839467107-007
Gettens, R. J., and Stout, G. L. (2012). Painting materials: A Short Encyclopedia. Courier Corporation.
High-Level Expert Group on Artificial Intelligence. (2019). A Definition of AI: Main Capabilities and Scientific Disciplines. European Commission.
Ho, J., Jain, A., and Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. arXiv.
Hong, J.-W., and Curran, N. M. (2019). Artificial Intelligence, Artists, and art. ACM Transactions on Multimedia Computing, Communications, and Applications, 15, 1-16. https://doi.org/10.1145/3326337 DOI: https://doi.org/10.1145/3326337
How, M.-L., and Hung, W. L. D. (2019). Educing AI-thinking in Science, Technology, Engineering, Arts, and Mathematics (STEAM) education. Education Sciences, 9, 184. https://doi.org/10.3390/educsci9030184 DOI: https://doi.org/10.3390/educsci9030184
Khanam, R., Hussain, M., Hill, R., and Allen, P. A. (2024). Comprehensive Review of Convolutional Neural Networks for Defect Detection in Industrial Applications. IEEE Access, 12, 94250-94295. https://doi.org/10.1109/ACCESS.2024.3425166 DOI: https://doi.org/10.1109/ACCESS.2024.3425166
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521, 436-444. https://doi.org/10.1038/nature14539 DOI: https://doi.org/10.1038/nature14539
Marques, R. D. R. (2023). Alligatoring: An Investigation into Paint Failure and Loss of Image Integrity in 19th Century Oil Paintings (Master's thesis). Universidade NOVA de Lisboa.
Mecklenburg, M. F. (2020). Methods and Materials and the Durability of Canvas Paintings: A Preface to the Topical Collection Failure Mechanisms in Picasso's paintings. SN Applied Sciences, 2, 2182. https://doi.org/10.1007/s42452-020-03832-6 DOI: https://doi.org/10.1007/s42452-020-03832-6
Mikalonyté, E. S., and Kneer, M. (2022). Can Artificial Intelligence Make Art? Folk Intuitions as to Whether AI-driven Robots can be Viewed as Artists and Produce Art. ACM Transactions on Human-Robot Interaction, 11, Article 43. https://doi.org/10.1145/3530875 DOI: https://doi.org/10.1145/3530875
Millet, K., Buehler, F., Du, G., and Kokkoris, M. D. (2023). Defending Humankind: Anthropocentric Bias in the Appreciation of AI Art. Computers in Human Behavior, 143, 107707. https://doi.org/10.1016/j.chb.2023.107707 DOI: https://doi.org/10.1016/j.chb.2023.107707
Nardelli, F., Martini, F., Lee, J., Lluveras-Tenorio, A., La Nasa, J., Duce, C., Ormsby, B., Geppi, M., and Bonaduce, I. (2021). The Stability of Paintings and the Molecular Structure of the Oil Paint Polymeric Network. Scientific Reports, 11, 14202. https://doi.org/10.1038/s41598-021-93268-8 DOI: https://doi.org/10.1038/s41598-021-93268-8
Tiribelli, S., Pansoni, S., Frontoni, E., and Giovanola, B. (2024). Ethics of Artificial Intelligence for Cultural Heritage: Opportunities and Challenges. IEEE Transactions on Technology and Society, 5, 293-305. https://doi.org/10.1109/TTS.2024.3432407 DOI: https://doi.org/10.1109/TTS.2024.3432407
Virto, N. R., and López, M. F. B. (2019). Robots, Artificial Intelligence, and Service Automation to the Core: Remastering Experiences at Museums. In S. Ivanov and C. Webster (Eds.), Robots, Artificial Intelligence, and Service Automation in Travel, Tourism and hospitality. Emerald Publishing.
Wen, J., and Ma, B. (2024). Enhancing Museum Experience Through Deep Learning and Multimedia Technology. Heliyon, 10, e32706. https://doi.org/10.1016/j.heliyon.2024.e32706 DOI: https://doi.org/10.1016/j.heliyon.2024.e32706
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Bhanu Juneja, Avni Garg, Praveen Pwaskar, Dr. Nitin Ajabrao Dhawas, Alok Kumar, Dr. Anand Kumar Gupta, Archana Haribhau Bhapkar
This work is licensed under a Creative Commons Attribution 4.0 International License.
With the licence CC-BY, authors retain the copyright, allowing anyone to download, reuse, re-print, modify, distribute, and/or copy their contribution. The work must be properly attributed to its author.
It is not necessary to ask for further permission from the author or journal board.
This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.
