PMID- 36015693
OWN - NLM
STAT- MEDLINE
DCOM- 20220829
LR  - 20220829
IS  - 1424-8220 (Electronic)
IS  - 1424-8220 (Linking)
VI  - 22
IP  - 16
DP  - 2022 Aug 9
TI  - Bimodal Learning Engagement Recognition from Videos in the Classroom.
LID - 10.3390/s22165932 [doi]
LID - 5932
AB  - Engagement plays an essential role in the learning process. Recognition of 
      learning engagement in the classroom helps us understand the student's learning 
      state and optimize the teaching and study processes. Traditional recognition 
      methods such as self-report and teacher observation are time-consuming and 
      obtrusive to satisfy the needs of large-scale classrooms. With the development of 
      big data analysis and artificial intelligence, applying intelligent methods such 
      as deep learning to recognize learning engagement has become the research hotspot 
      in education. In this paper, based on non-invasive classroom videos, first, a 
      multi-cues classroom learning engagement database was constructed. Then, we 
      introduced the power IoU loss function to You Only Look Once version 5 (YOLOv5) 
      to detect the students and obtained a precision of 95.4%. Finally, we designed a 
      bimodal learning engagement recognition method based on ResNet50 and CoAtNet. Our 
      proposed bimodal learning engagement method obtained an accuracy of 93.94% using 
      the KNN classifier. The experimental results confirmed that the proposed method 
      outperforms most state-of-the-art techniques.
FAU - Hu, Meijia
AU  - Hu M
AD  - Hubei Research Center for Educational Informationization, Faculty of Artificial 
      Intelligence in Education, Central China Normal University, Wuhan 430074, China.
AD  - Huanggang High School of Hubei Province, Huanggang 438000, China.
FAU - Wei, Yantao
AU  - Wei Y
AD  - Hubei Research Center for Educational Informationization, Faculty of Artificial 
      Intelligence in Education, Central China Normal University, Wuhan 430074, China.
FAU - Li, Mengsiying
AU  - Li M
AD  - School of Management, Wuhan College, Wuhan 430212, China.
FAU - Yao, Huang
AU  - Yao H
AUID- ORCID: 0000-0001-5055-4106
AD  - Hubei Research Center for Educational Informationization, Faculty of Artificial 
      Intelligence in Education, Central China Normal University, Wuhan 430074, China.
FAU - Deng, Wei
AU  - Deng W
AD  - Hubei Research Center for Educational Informationization, Faculty of Artificial 
      Intelligence in Education, Central China Normal University, Wuhan 430074, China.
FAU - Tong, Mingwen
AU  - Tong M
AD  - Hubei Research Center for Educational Informationization, Faculty of Artificial 
      Intelligence in Education, Central China Normal University, Wuhan 430074, China.
FAU - Liu, Qingtang
AU  - Liu Q
AD  - Hubei Research Center for Educational Informationization, Faculty of Artificial 
      Intelligence in Education, Central China Normal University, Wuhan 430074, China.
LA  - eng
GR  - CCNUTEIII-2021-19/National Collaborative Innovation Experimental Base 
      Construction Project for Teacher Develop-ment of Central China Normal University/
PT  - Journal Article
DEP - 20220809
PL  - Switzerland
TA  - Sensors (Basel)
JT  - Sensors (Basel, Switzerland)
JID - 101204366
SB  - IM
MH  - *Artificial Intelligence
MH  - Humans
MH  - *Problem-Based Learning/methods
MH  - Students
PMC - PMC9415674
OTO - NOTNLM
OT  - bimodal
OT  - classroom videos
OT  - deep learning
OT  - learning engagement
COIS- The authors declare no conflict of interest.
EDAT- 2022/08/27 06:00
MHDA- 2022/08/30 06:00
PMCR- 2022/08/09
CRDT- 2022/08/26 01:44
PHST- 2022/06/02 00:00 [received]
PHST- 2022/07/27 00:00 [revised]
PHST- 2022/07/27 00:00 [accepted]
PHST- 2022/08/26 01:44 [entrez]
PHST- 2022/08/27 06:00 [pubmed]
PHST- 2022/08/30 06:00 [medline]
PHST- 2022/08/09 00:00 [pmc-release]
AID - s22165932 [pii]
AID - sensors-22-05932 [pii]
AID - 10.3390/s22165932 [doi]
PST - epublish
SO  - Sensors (Basel). 2022 Aug 9;22(16):5932. doi: 10.3390/s22165932.