LIGHTWEIGHT EDGE-AI FRAMEWORK FOR REAL-TIME SUSPICIOUS ACTIVITY AND EMOTION-AWARE SMART SURVEILLANCE

Nidhi P. Nandankar; Shraddha R. Khonde

doi:10.29121/shodhkosh.v7.i10s.2026.8118

Authors

Nidhi P. Nandankar Student, Department of Computer Engineering, Modern Education Society's Wadia College of Engineering, Pune 411001, India
Shraddha R. Khonde Associate Professor, Department of Computer Engineering, Modern Education Society's Wadia College of Engineering, Pune 411001, India

DOI:

https://doi.org/10.29121/shodhkosh.v7.i10s.2026.8118

Keywords:

Edge-AI, Smart Surveillance, YOLOv8, Deep SORT, Facial Emotion Recognition, Suspicious Activity Detection, BILSTM, IOT-Based Security Systems

Abstract [English]

With the growing need for an intelligent public safety system, there is an emerging trend towards developing advanced automated smart surveillance systems that can detect suspicious behavior and evaluate emotions in humans instantly. Traditional surveillance systems largely depend on human operators for observation, which results in delayed reaction times and lower accuracy due to crowded environments. This paper presents a lightweight Edge-AI based smart surveillance solution that detects suspicious activities and evaluates human facial emotions through video analytics.

In particular, the proposed framework uses YOLOv8 for object and human detection and DeepSORT for multi-object tracking using continuous video frames. MTCNN is used for detecting and extracting faces from the video frame, whereas the CNN model is used for facial emotion detection. In addition, the BiLSTM model is used to identify abnormal activities and behavior patterns from the video feed using facial data. The proposed framework makes use of edge devices and IoT technologies to minimize computational delays and reduce dependence on cloud infrastructure.

Experimental analysis suggests that the proposed hybrid solution ensures efficient real-time processing and reliable detection accuracy for the target surveillance task.

References

A. Kuzdeuov, M. Zakaryanov, A. Tleuliyev, and H. A. Varol, “OpenThermalPose2: Extending the Open-Source Annotated Thermal Human Pose Dataset With More Data, Subjects, and Poses,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 7, no. 4, pp. 902–913, Oct. 2025, doi: 10.1109/TBIOM.2025.3575499.

A. M. Mekala et al., “Fused YOLO and Traditional Features for Emotion Recognition From Facial Images of Tamil and Russian Speaking Children: A Cross-Cultural Study,” IEEE Access, vol. 13, pp. 86828–86840, 2025, doi: 10.1109/ACCESS.2025.3569771.

A. Rehman, T. Saba, M. Z. Khan, R. Damaševičius, and S. A. Bahaj, “Internet-of-Things-Based Suspicious Activity Recognition Using Multimodalities of Computer Vision for Smart City Security,” Security and Communication Networks, vol. 2022, Art. no. 8383461, 2022.

B. Ma et al., “Distracted Driving Behavior and Driver’s Emotion Detection Based on Improved YOLOv8 With Attention Mechanism,” IEEE Access, vol. 12, pp. 37983–37994, 2024, doi: 10.1109/ACCESS.2024.3374726.

Eissa, M. E. A. (2025). The Influence of AI-Generated Content on Trust and Credibility Within Specialized Online Communities: A Brief Review on Proposed Conceptual Framework. ShodhAI: Journal of Artificial Intelligence, 2(2), 1–14. https://doi.org/10.29121/shodhai.v2.i2.2025.40

I. U. Khan, S. Afzal, and J. W. Lee, “Human Activity Recognition via Hybrid Deep Learning Based Model,” Sensors, vol. 22, no. 1, p. 323, 2022.

K. U. Duja, I. A. Khan, and M. Alsuhaibani, “Video Surveillance Anomaly Detection: A Review on Deep Learning Benchmarks,” IEEE Access, vol. 12, pp. 164811–164842, 2024, doi: 10.1109/ACCESS.2024.3491868.

M. M. A. Parambil et al., “Navigating the YOLO Landscape: A Comparative Study of Object Detection Models for Emotion Recognition,” IEEE Access, vol. 12, pp. 109427–109442, 2024, doi: 10.1109/ACCESS.2024.3439346.

M. M. Zaidi et al., “Suspicious Human Activity Recognition From Surveillance Videos Using Deep Learning,” IEEE Access, vol. 12, pp. 105497–105510, 2024, doi: 10.1109/ACCESS.2024.3436653.

M. Q. Gandapur and E. Verdú, “ConvGRU-CNN: Spatiotemporal Deep Learning for Real-World Anomaly Detection in Video Surveillance System,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 8, no. 4, p. 88, 2023.

N. Dwivedi, D. Singh, and D. Kushwaha, “A Novel Approach for Suspicious Activity Detection With Deep Learning,” Multimedia Tools and Applications, vol. 82, pp. 1–24, 2023, doi: 10.1007/s11042-023-14445-7.

O. Saidani et al., “An Efficient Human Activity Recognition Using Hybrid Features and Transformer Model,” IEEE Access, vol. 11, pp. 101373–101386, 2023, doi: 10.1109/ACCESS.2023.3314492.

P. Kuppusamy and V. C. Bharathi, “Human Abnormal Behaviour Detection Using CNNs in Crowded and Uncrowded Surveillance – A Survey,” Measurement: Sensors, vol. 24, p. 100510, 2022.

Q. U. A. Arshad et al., “Anomalous Situations Recognition in Surveillance Images Using Deep Learning,” Computers, Materials & Continua, vol. 76, no. 1, pp. 1103–1125, 2023.

R. R. Subramanian and V. Vasudevan, “A Deep Genetic Algorithm for Human Activity Recognition Leveraging Fog Computing Frameworks,” Journal of Visual Communication and Image Representation, vol. 77, p. 103132, 2021, doi: 10.1016/j.jvcir.2021.103132.

R. Rajeswari, “Anomalous Human Activity Recognition From Video Sequences Using BRISK Features and Convolutional Neural Networks,” Galaxy International Interdisciplinary Research Journal, vol. 10, no. 2, pp. 269–280, 2022.

R. Vrskova, R. Hudec, P. Kamencay, and P. Sykora, “A New Approach for Abnormal Human Activities Recognition Based on ConvLSTM Architecture,” Sensors, vol. 22, no. 8, p. 2946, 2022.

S. Khanam, M. Sharif, X. Cheng, and S. Kadry, “Suspicious Action Recognition in Surveillance Based on Handcrafted and Deep Learning Methods: A Survey of the State of the Art,” Computers and Electrical Engineering, vol. 120, p. 109811, 2024.

T. Saba et al., “Suspicious Activity Recognition Using Proposed Deep L4-Branched-ActionNet With Entropy Coded Ant Colony System Optimization,” IEEE Access, vol. 9, pp. 89181–89197, 2021, doi: 10.1109/ACCESS.2021.3091081.

W. Ullah et al., “CNN Features With Bi-Directional LSTM for Real-Time Anomaly Detection in Surveillance Networks,” Multimedia Tools and Applications, vol. 80, no. 11, pp. 16979–16995, 2021.

Y. Qin et al., “ORAF-YOLO: A Lightweight Factory Unsafe Behavior Recognition Method With Occlusion-Robust Adaptive Fusion,” IEEE Access, 2026, doi: 10.1109/ACCESS.2026.3687749.