AI-Based Multi-Person Emotion Recognition and Engagement Evaluation System

Abstract

The interaction of a group in the real world such as classroom, meeting or in the general assembly is very difficult to comprehend since they are dynamic and multi-person environment. Conventional systems have limited application in a crowd situation as they examine a single face. A multi-face mood detection and engagement assessment system in real-time is suggested in this study to capture a number of people at once, and evaluate their general behavioral condition. Our method employs advanced face detection techniques such as RetinaFace for detecting multiple faces in the video. For the emotion detection, we use DeepFace that uses a machine learning approach to understand facial expressions. It associates these emotions with types of engagement such as attentive, engaged, confused or disengaged, to gain an understanding of the group dynamics. It analyses video feeds to aggregate individual responses and engagement scores, for group emotions. Expert testing demonstrates that the system can effectively identify various faces under varying conditions and give pertinent data regarding group conduct. We use the methods of data visualization like pie charts, bar graphs, scatter plots etc., to visualize the extent of distribution of these emotions and their support. The work suggests a scalable practical solution to the automated engagement monitoring issue that may find its future applications in smart classes, distance learning, audience analytics and human-computer interaction. They also harbor an array of ethical issues of privacy and accountable use of data, which have to be resolved to permit safe usage.