AI-GENERATED VISUALIZATIONS OF MUSICAL PATTERNS

Pravat Kumar Routray; Manish Nagpal; Megha Gupta; Amanveer Singh

doi:10.29121/shodhkosh.v6.i2s.2025.6697

Authors

Dr. Pravat Kumar Routray Assistant Professor, Department of Computer Science and Engineering, Institute of Technical Education and Research, Siksha 'O' Anusandhan (Deemed to be University) Bhubaneswar, Odisha, India
Manish Nagpal Chitkara Centre for Research and Development, Chitkara University, Himachal Pradesh, Solan, 174103, India
Dr. Megha Gupta Assistant Professor, Department of Computer Science & Engineering, Noida Institute of Engineering and Technology, Greater Noida, Uttar Pradesh, India
Amanveer Singh Centre of Research Impact and Outcome, Chitkara University, Rajpura- 140417, Punjab, India

DOI:

https://doi.org/10.29121/shodhkosh.v6.i2s.2025.6697

Keywords:

AI-Generated Art, Music Visualization, Multimodal Learning, Computational Creativity, Aesthetic Cognition

Abstract [English]

The paper will discuss the new area of AI-composed visualizations of musical patterns, which is a convergence of music theory, computational creativity, and visual arts. It explores how such audio formats as pitch, rhythm, harmony, and timbre can be produced using artificial intelligence via visual media in a dynamic and aesthetically coherent format. The research places this change in the context of interdisciplinary concepts, taking into account the aspects of perceptual, cognitive, and ethical. The system recapitulates the objective and the affective aspects of music by using neural networks that have been trained on multimodal data and relates them to the visual parameter of color, geometry and motion. The proposed system architecture combines the steps of the audio feature extraction, representation learning, and visual synthesis. As a result of experimental production and qualitative analysis, a number of recurring motifs and emergent visual structures are found, which represent associations between tonal density and color complexity, regularity in rhythm, and geometric symmetry, and harmonic consonance and spatial fluidity. The findings illustrate the interpretive depth of the AI-mediated sound-image translations to illustrate how artificial intelligence can generate work of art that can trigger aesthetic responses in the human condition. Nevertheless, the paper also notes that there are a number of limitations: the visualization in real-time is technically limited; the assessment of aesthetics is subjective by nature; and the biases of data in training corpora are potentially a problem concerning the reproducibility and the fairness.

References

Briot, J.-P., and Pachet, F. (2020). Music Generation by Deep Learning—Challenges and Directions. Neural Computing and Applications, 32, 981–993. https://doi.org/10.1007/s00521-018-3813-6 DOI: https://doi.org/10.1007/s00521-018-3813-6

Deng, Q., Yang, Q., Yuan, R., Huang, Y., Wang, Y., Liu, X., Tian, Z., Pan, J., Zhang, G., Lin, H., et al. (2024). ComposerX: Multi-Agent Symbolic Music Composition with LLMs. arXiv.

Guo, Z., Dimos, M., and Dorien, H. (2021). Hierarchical Recurrent Neural Networks for Conditional Melody Generation With Long-Term Structure. arXiv.

Hernandez-Olivan, C., and Beltran, J. R. (2021). Music Composition with Deep Learning: A Review. Springer. DOI: https://doi.org/10.1007/978-3-031-18444-4_2

Herremans, D., Chuan, C.-H., and Chew, E. (2017). A Functional Taxonomy of Music Generation Systems. ACM Computing Surveys, 50(5), Article 69. https://doi.org/10.1145/3108242 DOI: https://doi.org/10.1145/3108242

Ji, S., Luo, J., and Yang, X. (2020). A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions. arXiv.

Ji, S., Yang, X., and Luo, J. (2023). A Survey on Deep Learning for Symbolic Music Generation: Representations, Algorithms, Evaluations, and Challenges. ACM Computing Surveys, 56(1), 1–39. https://doi.org/10.1145/3561800 DOI: https://doi.org/10.1145/3597493

Liang, X., Du, X., Lin, J., Zou, P., Wan, Y., and Zhu, B. (2024). ByteComposer: A Human-Like Melody Composition Method Based on Language Model Agent. arXiv.

Lu, P., Xu, X., Kang, C., Yu, B., Xing, C., Tan, X., and Bian, J. (2023). MuseCoco: Generating Symbolic Music from Text. arXiv.

Ma, Y., Øland, A., Ragni, A., Sette, B. M. D., Saitis, C., Donahue, C., Lin, C., Plachouras, C., Benetos, E., Shatri, E., et al. (2024). Foundation Models for Music: A survey. arXiv.

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv.

Von Rütte, D., Biggio, L., Kilcher, Y., and Hofmann, T. (2022). FIGARO: Controllable Music Generation Using leqarned and Expert Features. arXiv.

Wen, Y.-W., and Ting, C.-K. (2023). Recent Advances of Computational Intelligence Techniques for Composing Music. IEEE Transactions on Emerging Topics in Computational Intelligence, 7(3), 578–597. https://doi.org/10.1109/TETCI.2022.3220901 DOI: https://doi.org/10.1109/TETCI.2022.3221126

Wu, J., Hu, C., Wang, Y., Hu, X., and Zhu, J. (2020). A Hierarchical Recurrent Neural Network for Symbolic Melody Generation. IEEE Transactions on Cybernetics, 50(6), 2749–2757. https://doi.org/10.1109/TCYB.2019.2893242 DOI: https://doi.org/10.1109/TCYB.2019.2953194

Yu, Y., Zhang, Z., Duan, W., Srivastava, A., Shah, R., and Ren, Y. (2023). Conditional Hybrid GAN for Melody Generation from Lyrics. Neural Computing and Applications, 35, 3191–3202. https://doi.org/10.1007/s00521-022-07542-1 DOI: https://doi.org/10.1007/s00521-022-07863-5

Yuan, R., Lin, H., Wang, Y., Tian, Z., Wu, S., Shen, T., Zhang, G., Wu, Y., Liu, C., Zhou, Z., et al. (2024). ChatMusician: Understanding and Generating Music Intrinsically with Large Language modqels. arXiv. https://arxiv.org/abs/2402.16153

Zhang, Z., Yu, Y., and Takasu, A. (2023). Controllable Lyrics-to-Melody Generation. Neural Computing and Applications, 35, 19805–19819. https://doi.org/10.1007/s00521-023-08566-7 DOI: https://doi.org/10.1007/s00521-023-08728-1

Zhu, Y., Baca, J., Rekabdar, B., and Rawassizadeh, R. (2023). A Survey of AI Music Generation Tools and Models. arXiv.