|
ShodhKosh: Journal of Visual and Performing ArtsISSN (Online): 2582-7472
Real-Time AI Feedback for Dance Students Suraj Bhan 1 1 Assistant
Professor, School of Engineering, and Technology, Noida International, University,
203201, India 2 Greater
Noida, Uttar Pradesh 201306, India 3 Assistant Professor, Department of Management Studies, JAIN
(Deemed-to-be University), Bengaluru, Karnataka, India 4 Centre of Research Impact and Outcome, Chitkara University, Rajpura-
140417, Punjab, India 5 Chitkara Centre for Research and Development, Chitkara University,
Himachal Pradesh, Solan, 174103, India 6 HOD Professor, Department of Design, Vivekananda Global University,
Jaipur, India 7 Department of Engineering, Science and Humanities Vishwakarma
Institute of Technology, Pune, Maharashtra, 411037, India
1. INTRODUCTION Dance as an art and a discipline requires accuracy, beat, synchronization and movement. The classical teaching of dances is based on the use of human observation, during which the teachers assess the work of the students and exercise the feedback in written or non-worded form. Nonetheless, this feedback process although efficient has a limit inherent to human perception, fatigue and availability. Recently, recent developments in Artificial Intelligence (AI) and computer vision have created new opportunities to enhance the conventional teaching process with intelligent, data-driven ones. The technologies are capable of studying the patterns of motion, detecting the performance deviations and providing immediate corrective recommendations- a new age of real-time AI feedback in students of dance is born Hou (2024). The combination of AI and dance does not simply concern itself with automation; this is a question of improved human creativity and learning with the help of technology. By tracking the body posture, joint positioning and movement paths in real-time, real-time feedback systems can assist dancers to develop their technique further. In contrast to the traditional learning in which the post-performance assessment method is relied on, the AI-related systems enable the provision of feedback in real-time, which means that the student can make the needed adjustments to the movements during the training. This feature will greatly speed up the learning curve and facilitate the process of self-directed enhancement Hong (2024). In addition, it makes quality access to dance education democratic to learners who might lack access to professional tutors or formal training conditions. The center of such systems is the combination of motion tracking and pose estimation algorithms innovations that are fast developing with the development of machine learning and computer vision. Other systems such as OpenPose, MediaPipe and DeepLabCut have facilitated the accurate following of skeletal joints of humans using regular video footage, without the need to purchase costly motion-capture suits. Using such tools, AI systems are able to recognize movements in a dance and compare the performance with a pre-trained model or an expert reference data in real-time Wu et al. (2023). This objective concept of movement is where objective performance evaluation is based in a domain that is otherwise subjective. Moreover, the advent of deep learning has brought to change the manner in which motion data is processed. CNNs and RNNs are capable of processing spatial and temporal attributes of sequence of movements, allowing the system to not only identify the static postures but also the dynamic attributes of flow, time, and music synchronization Copet et al. (2023). These systems in combination with the signal analysis of sounds can be used to measure rhythmical correctness so that the motions are in time with the tempo and musical indications. Technical correction is not the only possible way of improving the real-time AI feedback. It may support individual learning processes by personalizing the training modules depending on the strengths, weaknesses, and progress speed of a dancer. And as an instructor, AI systems can act as an analytical assistant AI can provide performance summaries, error heatmaps and progress analytics to allow an instructor to tailor instruction based on these analytics. To students, it provides a non-judgmental and regular feedback mechanism that promotes trial and error and constant refinement Chen et al. (2021). Although these have such benefits, implementation of such systems comes with a number of challenges. The considerations are ensuring low-latency processing, accuracy of processing across various dance styles and ensuring the issues of data privacy and ethical use of AI in education. 2. Literature Review 2.1. Existing AI systems for motion tracking and performance analysis In recent decades, a number of researchers have designed diverse AI-based motion tracking and analysis systems of performance, which have started to be used in dance and other related movement fields. Other primitive computer vision systems like Pfinder were the basis to this as they were capable of recognising human forms and simple gestures in the video feed making it possible to track body parts in real-time without requiring the user to wear a special suit. Later on, the barrier to using regular cameras to perform markerless human pose estimation has been drastically reduced by open-source frameworks such as OpenPose, MediaPipe, AlphaPose and DensePose. These pose-estimation algorithms project video frames onto skeletal joint locations (keypoints), and subsequent analysis of posture, movement paths and body alignment can then be done. Based on these, there have been suggestions of some specialized systems of dance practice An and Oliver (2021). An example of such studies featured a real-time dance analysis system which based pose estimation on demonstration of posture differences in dance routines in order to provide objective feedback to learners and demonstrating improvements in novice and intermediate dancers. More sophisticated pipelines also integrate primitives estimation with time-series analysis and machine learning: one example is a system that breaks down the performance of a dancer into primitive motion units (through a model such as MoveNet + a vision-transformer), matches the segments of those unit to reference performance through methods such as Dynamic Time Warping (DTW) and then provides the dancer with correction feedback Julia and Calderón (2021). These systems demonstrate the possibility of providing move level feedback (granular feedback) instead of merely posture snapshots, and thus they are promising solutions to real-time coaching tools. 2.2. Applications of Computer Vision and Machine Learning in Dance The overlap of computer vision, deep learning, and dance has recently become a matter of academic interest, specifically in performing movement recognition, choreography analysis, performance evaluation, and even creative augmentation of dance. A more recent systematic review has proven that machine learning, such as pose detection and action recognition, is currently used to facilitate choreographic composition, create new visualisations of movement, and aid in the assessment of a performance Cao et al. (2022). Deep neural network models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) / Long Short-Term Memory (LSTM) networks and Graph Convolutional Networks (GCNs) have been used in practice to process both temporal and spatial aspects of dance movements. Indicatively, a most recent study has suggested a model that incorporates both multi-kinematic features fusion with spatio-temporal graph convolution and attention to identify and predict time-varying dance poses. These models do not only allow identifying the static postures but also forecasting further sequence of poses in the future- which can be applied to anticipate and correct future poses, and thus can be applied to training which is safer and more efficient Ben et al. (2024). 2.3. Limitations of Current Feedback Systems Nevertheless, regardless of the promising development, the existing AI-based feedback machines used in dance have some serious weaknesses. One of these problems is inherent to the nature of human pose estimation and motion capture: whereas 2D markerless pose-estimation systems are convenient and cost-effective, they have problems with occlusions, out-of-plane motions, and complicated 3D motions, which are present in expressive dance. Additionally, a large number of systems are trained and tested on a small number of dance styles or controlled studio conditions. Consequently, their strength in different genres of dancing, changing lights, costumes, stage conditions, or group-dancing conditions are still small Emma et al. (2020). On a more technical level, the computational demands of high-fidelity analysis can be low when deep learning models are used, particularly when they incorporate temporal and spatial modeling, and this can likewise impact on real-time performance on devices with limited resources. Table 1 presents a brief review of existing research that has been dedicated to AI-based systems of dance analysis. Pedagogically, most feedback systems give low-level, technical corrections (e.g. joint alignment or timing), and are not capable of assessing the expressive, aesthetic, or stylistic aspects of dance, which are implicitly subjective and have a strong relationship to cultural context or artistic intent. Table 1
3. Methodology 3.1. System design and architecture The suggested dance student real-time AI feedback system has a modular architecture that aims at efficiency, scalability, and low latency. The system is designed into four major layers namely: data acquisition, processing, analytics and delivery of feedback. Motion is represented in the data acquisition layer by means of motion data obtained by either RGB video data or inertial data sensors. The processing layer deals with data preprocessing, i.e. background subtraction, frame normalization, and keypoint extraction. Fundamentally, the analytics layer uses a pose estimation component (based on deep learning, such as OpenPose or MediaPipe Holistic) to estimate skeletal joint coordinates Mokmin and Jamiat (2021). These coordinates are then inputted to a performance evaluation module that is based on temporal analysis and pattern recognition to evaluate alignment, rhythm and movement flow based on expert reference datasets. This analysis is represented as actionable information by the feedback delivery layer as visual overlays, color-coded body maps, or audible information. Figure 1 describes modular parts with the help of which real-time analysis of AI-based dance performance is possible. The design allows cloud based processing of complex calculations as well as edge deployment to be used offline in studios or classrooms. Figure 1 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Table 2 System Performance Metrics |
||
|
Parameter |
Mean Value |
Standard Deviation |
|
Pose Estimation Accuracy (%) |
92.4 |
±3.1 |
|
Feedback Latency (ms) |
78 |
±9 |
|
Motion Classification
Accuracy (%) |
89.7 |
±4.5 |
|
Rhythm Synchronization Score (%) |
91.3 |
±2.7 |
Table 2 shows quantitative analysis of the performance of the proposed system of real time AI feedback. The findings indicate a high level of precision and responsiveness which confirms the ability of the system to work effectively as live dance sessions happen. Figure 3 illustrates variations and changes in major motion performance measures.
Figure 3

Figure 3 Trends in Motion Performance Metrics with
Variability
The accuracy of pose estimation at 92.4% (±3.1) means that the model accurately predicts and follows skeletal keypoints with a variety of dance motions, and thus the model is consistently reliable to capture motion correctly even with moderately dark and background changes. On the same note, the classification accuracy of 89.7% (±4.5) of the motion classification indicates the ability to recognize intricate dance patterns, and, therefore, the CNN-LSTM framework is effective in model motion dynamics (both spatial and temporal).
Figure 4

Figure 4 Comparison of Mean and Variability in Motion
Evaluation Parameters
The feedback lag was averaged at 78 ms (±9) which is far below the perceptible threshold of delay and so, the dancers received immediate and continuous corrective feedback. Comparison of mean values and variability of the major motion evaluation parameters are compared in Figure 4. The high level of temporal alignment of the dancer motions with the music indicated by the rhythm synchronization score of 91.3% (±2.7) is important in the process of keeping the flow of the performance and ensuring that the timing is accurate.
7. Conclusion
The paper introduced a new model of real-time AI feedback in learning dance based on a computer vision, pose estimation, and machine learning to provide performance feedback and individual learning. The system also found that the systematic design and evaluation allowed it to provide correct and low-latency feedback to enable students to refine their movements by themselves. It is a breakthrough method of acquiring motor skills, rhythmic synchronization and body positioning by filling the gap between conventional training and the training implemented with technological aid. One of the strength of this framework is the ability to be flexible and scalable. It may be applied with the help of affordable video devices or combined with the highly developed motion sensors in a professional work. The deep learning architecture of CNN-LSTM hybrids based on real-time analytics pipeline allows the interpretations of spatial and temporal patterns of motion in a nuanced manner. This guarantees that there is a balance between quantitative accuracy and artistic fluidity two very important elements of dance performance. Nevertheless, the study also provides an insight into the persistent issues, such as embracing the strength in various environmental conditions, data privacy, and being fair in the context of various forms of dance and body types. The possible future enhancements include 3D pose estimation and edge-AI optimization and emotion-aware feedback systems that are capable of capturing the expressive features of a performance.
CONFLICT OF INTERESTS
None.
ACKNOWLEDGMENTS
None.
REFERENCES
An, T., and Oliver, M. (2021). What in the World is Educational Technology? Rethinking the Field from the Perspective of the Philosophy of Technology. Learning, Media and Technology, 46(1), 6–19. https://doi.org/10.1080/17439884.2020.1810066
Ben, D., Seunghyun, B., Donal, H., Yongjin, L., and Judy, F. (2024). Students’ Perspectives of Social and Emotional Learning in a High School Physical Education Program. Journal of Teaching in Physical Education, 43(4), 549–556. https://doi.org/10.1123/jtpe.2023-0090
Cao, F., Lei, M., Lin, S., and Xiang, M. (2022). Application of Artificial Intelligence-Based Big Data Technology in Physical Education Reform. Mobile Information Systems, 2022, Article 4017151. https://doi.org/10.1155/2022/4017151
Chen, K., Tan, Z., Lei, J., Zhang, S.-H., Guo, Y.-C., Zhang, W., and Hu, S.-M. (2021). ChoreoMaster: Choreography-Oriented Music-Driven Dance Synthesis. ACM Transactions on Graphics, 40(4), Article 1–13. https://doi.org/10.1145/3450626.3459932
Copet, J., Kreuk, F., Gat, I., Remez, T., Kant, D., Synnaeve, G., Adi, Y., and Défossez, A. (2023). Simple and Controllable Music Generation (arXiv:2306.05284). arXiv.
Emma, R., Lewis, S., Miah, A., Lupton, D., and Piwek, L. (2020). Digital Health Generation? Young People’s Use of “Healthy Lifestyle” Technologies. University of Bath.
Gil-Espinosa, F. J., Nielsen-Rodríguez, A., Romance, R., and Burgueño, R. (2022). Smartphone Applications for Physical Activity Promotion from Physical Education. Education and Information Technologies, 27(8), 11759–11779. https://doi.org/10.1007/s10639-022-11108-2
Guo, E., and Cui, X. (2024). Simulation of Optical Sensor Network Based on Edge Computing in Athlete Physical Fitness Monitoring System. Optical and Quantum Electronics, 56, Article 637. https://doi.org/10.1007/s11082-024-07608-9
Hong, C. (2024). Application of Virtual Digital People in the Inheritance and Development of Intangible Cultural Heritage. People’s Forum, 6, 103–105.
Hou, C. (2024). Artificial Intelligence Technology Drives Intelligent Transformation of Music Education. Applied Mathematics and Nonlinear Sciences, 9(1), 21–23. https://doi.org/10.2478/amns-2024-1947
Julia, S., and Calderón, A. (2021). Technology-Enhanced Learning in Physical Education? A Critical Review of the Literature. Journal of Teaching in Physical Education, 41(4), 689–709. https://doi.org/10.1123/jtpe.2021-0136
Kebao, Z., Kehu, Z., and Liu, W. (2024). The Evaluation of Sports Performance in Tennis Based on Flexible Piezoresistive Pressure Sensing Technology. IEEE Sensors Journal, 24(22), 28111–28118. https://doi.org/10.1109/JSEN.2024.3425591
Mokmin, N. A. M. (2020). The Effectiveness of a Personalized Virtual Fitness Trainer in Teaching Physical Education by Applying the Artificial Intelligent Algorithm. International Journal of Human Movement and Sports Sciences, 8(5), 258–264. https://doi.org/10.13189/saj.2020.080514
Mokmin, N. A. M., and Jamiat, N. (2021). The Effectiveness of a Virtual Fitness Trainer App in Motivating and Engaging Students for Fitness Activity by Applying Motor Learning Theory. Education and Information Technologies, 26(2), 1847–1864. https://doi.org/10.1007/s10639-020-10337-7
Wu, J., Gan, W., Chen, Z., Wan, S., and Lin, H. (2023). AI-Generated Content (AIGC): A Survey (arXiv:2304.06632). arXiv.
Yang, Z., Wang, Q., Yu, H., Xu, Q., Li, Y., Cao, M., Dhakal, R., Li, Y., and Yao, Z. (2024). Self-Powered Biomimetic Pressure Sensor Based on Mn–Ag Electrochemical Reaction for Monitoring Rehabilitation Training of Athletes. Advanced Science, 11(15), Article 2401515. https://doi.org/10.1002/advs.202401515
Yu, C. W. (2022). A Case Study on Physical Education Class Using Physical Activity Solution App. Journal of Research in Curriculum and Instruction, 26(3), 458–471.
|
|
This work is licensed under a: Creative Commons Attribution 4.0 International License
© ShodhKosh 2024. All Rights Reserved.