2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

慕课授课中的学生听课行为自动分析系统

戴亚平 杨方方 赵翰奕 贾之阳 广田熏

戴亚平, 杨方方, 赵翰奕, 贾之阳, 广田熏. 慕课授课中的学生听课行为自动分析系统. 自动化学报, 2020, 46(4): 681-694. doi: 10.16383/j.aas.c170416
引用本文: 戴亚平, 杨方方, 赵翰奕, 贾之阳, 广田熏. 慕课授课中的学生听课行为自动分析系统. 自动化学报, 2020, 46(4): 681-694. doi: 10.16383/j.aas.c170416
DAI Ya-Ping, YANG Fang-Fang, ZHAO Han-Yi, JIA Zhi-Yang, HIROTA Kaoru. Auto Analysis System of Students Behavior in MOOC Teaching. ACTA AUTOMATICA SINICA, 2020, 46(4): 681-694. doi: 10.16383/j.aas.c170416
Citation: DAI Ya-Ping, YANG Fang-Fang, ZHAO Han-Yi, JIA Zhi-Yang, HIROTA Kaoru. Auto Analysis System of Students Behavior in MOOC Teaching. ACTA AUTOMATICA SINICA, 2020, 46(4): 681-694. doi: 10.16383/j.aas.c170416

慕课授课中的学生听课行为自动分析系统

doi: 10.16383/j.aas.c170416
详细信息
    作者简介:

    戴亚平  北京理工大学自动化学院教授.主要研究方向为图像特征提取与识别, 多传感器数据融合与决策诊断技术, 人工智能与专家系统. E-mail: daiyaping@bit.edu.cn

    杨方方  北京理工大学自动化学院硕士研究生. 2016年获得东北大学学士学位.主要研究方向为计算机视觉与机器学习. E-mail: neuqyangfangfang@163.com

    赵翰奕  华北计算机系统工程研究所(信息产业部电子第六研究所)研究工程师. 2017年获得北京理工大学自动化学院硕士学位.主要研究方向为图像处理和深度学习. E-mail: zhao_angus@163.com

    广田熏  北京理工大学自动化学院"外专千人"特聘教授, 1976年和1979年分别获得东京工业大学电子工程专业硕士学位、博士学位.主要研究方向为图像模式识别, 智能机器人, 模糊控制, 人工智能及工业应用.E-mail: hirota@bit.edu.cn

    通讯作者:

    贾之阳  北京理工大学自动化学院助理教授. 2010年和2013年分别获得西北工业大学电气工程系学士学位和北京工业大学控制科学与工程系硕士学位. 2017年获得美国康涅狄格大学电气工程系博士学位.主要研究方向为智能制造, 生产系统建模, 性能分析, 能源高效生产管理.本文通信作者.E-mail: zhiyang.jia@bit.edu.cn

Auto Analysis System of Students Behavior in MOOC Teaching

More Information
    Author Bio:

    DAI Ya-Ping   Professor at the School of Automation, Beijing Institute of Technology, China. Her research interest covers image feature extraction and recognition, multi-sensor data fusion and decision diagnosis technology, artificial intelligence and expert system

    YANG Fang-Fang   Master student at School of Automation, Beijing Institute of Technology, China. She received her bachelor degree from School of Control Engineering, Northeastern University, China. Her research interest covers computer vision and machine learning

    ZHAO Han-Yi   Research engineer at The 6th Research Institute of China Electronics Corporation. He received his master degree from School of Automation, Beijing Institute of Technology, China in 2017. His research interest covers image processing and deep learning

    HIROTA Kaoru   Special professor of The Recruitment Program of Global Experts at the School of Automation, Beijing Institute of Technology, China. He received his master degree and Ph.D. degree from Electric Engineering, Tokyo Institute of Technology, Japan in 1976 and 1979. His research interest covers image pattern recognition, intelligent robotics, fuzzy control, artificial intelligence, and industrial applications of these topics

    Corresponding author: JIA Zhi-Yang   Assistant professor at the School of Automation, Beijing Institute of Technology. He received his bachelor degree and master degree from Department of Electrical Engineering, Northwestern Polytechnical University, and Department of Control Science and Engineering, Beijing University of Technology, China in 2010 and 2013. He received his Ph.D. degree from Department of Electrical and Computer Engineering, University of Connecticut, USA in 2017. His research interest covers smart manufacturing, modeling, analysis and control of production systems. Corresponding author of this paper
  • 摘要: 为了解决在线课程(Massive open online course, MOOC)授课过程中, 缺乏对于学生学习情况的跟踪与教学效果评估问题, 本文依据视频信息对学生行为进行建模, 提出了一种评判学生听课专心程度的行为自动分析算法.该算法能够有效跟踪学生的学习状态, 提取学生的行为特征参数, 并对这些参数进行D-S融合判决, 以获得学生的听课专注度.经过多次实验的结果表明, 本文采用的方法能够有效评判学生在授课期间的专心程度, 在数据融合上, 与贝叶斯推理方法相比, 采用D-S融合方法能有效提高实验结果的准确性和可靠性.
    Recommended by Associate Editor ZHU Jun
    1)  本文责任编委 朱军
  • 图  1  慕课授课过程中学生专注度自动检测与判定系统

    Fig.  1  Automatic detection system on student focus during MOOC teaching

    图  2  电脑摄像头提取的目标人体图像

    Fig.  2  Human body image obtained by computer camera

    图  3  电脑获取的人体图像的三种形态

    Fig.  3  Three forms of human body image acquired by computer

    图  4  $X$轴质心坐标变化曲线的两部分

    Fig.  4  Two parts of $X$ axis centroid position curve

    图  5  学生行为分析与推理决策流程图

    Fig.  5  Flow chart of student behavior analysis and decision-making

    图  6  端坐状态时关键帧的图像

    Fig.  6  Image of key frame in sit up condition

    图  7  端坐状态特征变化曲线

    Fig.  7  Characteristic curves of "sit up" state

    图  8  左顾右盼时关键帧的图像

    Fig.  8  Image of key frame in look around condition

    图  9  左顾右盼状态特征变化曲线

    Fig.  9  Characteristic curves of "look around" state

    图  10  埋头时关键帧的图像

    Fig.  10  Image of key frame in head drop

    图  11  埋头状态特征变化曲线

    Fig.  11  Characteristic curves of "head drop" state

    图  12  复杂状态实验1的关键帧序列

    Fig.  12  Key frame sequence under complex state experiment 1

    图  13  复杂状态1的特征变化曲线

    Fig.  13  Characteristic curves of complex state 1

    图  14  复杂状态实验2的关键帧序列

    Fig.  14  Key frame sequence under complex state Experiment 2

    图  15  复杂状态2的特征变化曲线

    Fig.  15  Characteristic curves of complex state 2 direction

    表  1  端坐时的基本概率赋值

    Table  1  The probability distributions when sit up

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{1j1})$ 0 0.4444 0.4444 0.1112
    $m(h_{1j2})$ 0 0.4528 0.4528 0.0944
    $m(h_{1j3})$ 0 0.4528 0.4528 0.0944
    $m(h_{2j1})$ 0.4444 0 0.4444 0.1112
    $m(h_{2j2})$ 0.4528 0 0.4528 0.0944
    $m(h_{2j3})$ 0.4528 0 0.4528 0.0944
    $m(h_{3j1})$ 0.4444 0 0.4444 0.1112
    $m(h_{3j2})$ 0.4528 0 0.4528 0.0944
    $m(h_{3j3})$ 0.4528 0 0.4528 0.0944
    $m(h_{4j1})$ 0 0 0.8888 0.1112
    $m(h_{4j2})$ 0 0 0.9056 0.0944
    $m(h_{4j3})$ 0 0 0.9056 0.0944
    下载: 导出CSV

    表  2  空间域融合后的结果

    Table  2  The results of fusion in the spatial domain

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{j1})$ 0.0191 0.0036 0.9773 0
    $m(h_{j2})$ 0.0142 0.0023 0.9834 0
    $m(h_{j3})$ 0.0142 0.0023 0.9834 0
    下载: 导出CSV

    表  3  多维特征信息融合结果对比

    Table  3  Comparison of multi-dimensional feature information fusion results

    实验方法 实验结果
    $m(h_{1})$ $m(h_{2})$ $m(h_{3})$ $m(U)$
    $X+Y(baseline)$ 0.0042 0.0042 0.9916 0
    $X+Y+S$ 0.0042 0 0.9958 0
    $X+Y+S+\theta$ 0 0 0.9999 0
    下载: 导出CSV

    表  4  左顾右盼状态的基本概率赋值

    Table  4  The probability distributions when look around

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{1j1})$ 0 0.4442 0.4442 0.1116
    $m(h_{1j2})$ 0.6271 0.1394 0.1394 0.0942
    $m(h_{1j3})$ 0.9058 0 0 0.0942
    $m(h_{2j1})$ 0.4442 0 0.4442 0.1116
    $m(h_{2j2})$ 0.4529 0 0.4529 0.0942
    $m(h_{2j3})$ 0.0784 0.7490 0.0784 0.0942
    $m(h_{3j1})$ 0.4442 0 0.4442 0.1116
    $m(h_{3j2})$ 0.4529 0 0.4529 0.0942
    $m(h_{3j3})$ 0.4529 0 0.4529 0.0942
    $m(h_{4j1})$ 0.0261 0.0261 0.8361 0.1116
    $m(h_{4j2})$ 0.3135 0.3135 0.2787 0.0942
    $m(h_{4j3})$ 0.4529 0.4529 0 0.0942
    下载: 导出CSV

    表  5  空间域融合后的结果

    Table  5  The results of fusion in the spatial domain

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{j1})$ 0.0249 0.0047 0.9704 0
    $m(h_{j2})$ 0.7743 0.0047 0.2210 0
    $m(h_{j3})$ 0.9228 0.0681 0.0091 0
    下载: 导出CSV

    表  6  埋头时的基本概率赋值

    Table  6  The probability distributions when head drop

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{1j1})$ 0 0.4500 0.4500 0.1000
    $m(h_{1j2})$ 0 0.4500 0.4500 0.1000
    $m(h_{1j3})$ 0 0.4500 0.4500 0.1000
    $m(h_{2j1})$ 0.2308 0.4385 0.2308 0.1000
    $m(h_{2j2})$ 0 0.9000 0 0.1000
    $m(h_{2j3})$ 0 0.9000 0 0.1000
    $m(h_{3j1})$ 0.4269 0.0462 0.4269 0.1000
    $m(h_{3j2})$ 0.2308 0.4385 0.2308 0.1000
    $m(h_{3j3})$ 0 0.9000 0 0.1000
    $m(h_{4j1})$ 0.2308 0.2308 0.4385 0.1000
    $m(h_{4j2})$ 0.0923 0.0923 0.7154 0.1000
    $m(h_{4j3})$ 0 0 0.9000 0.1000
    下载: 导出CSV

    表  7  空间域融合后的结果

    Table  7  The results of fusion in the spatial domain

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{j1})$ 0.0658 0.2048 0.7294 0
    $m(h_{j2})$ 0.0020 0.8131 0.1848 0
    $m(h_{j3})$ 0 0.9221 0.0778 0
    下载: 导出CSV

    表  8  证据体的基本概率赋值(复杂状态实验1)

    Table  8  Basic probability distribution of evidence bodies (Complex condition 1)

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{1j1})$ 0.1719 0.3617 0.3617 0.1048
    $m(h_{1j2})$ 0.0859 0.4082 0.4082 0.0976
    $m(h_{1j3})$ 0 0 0.4512 0.0976
    $m(h_{2j1})$ 0.4333 0.0286 0.4333 0.1048
    $m(h_{2j2})$ 0.4405 0.0215 0.4405 0.0976
    $m(h_{2j3})$ 0.4512 0 0.4512 0.0976
    $m(h_{3j1})$ 0.4369 0.0215 0.4369 0.1048
    $m(h_{3j2})$ 0.4515 0 0.4512 0.0976
    $m(h_{3j3})$ 0.4512 0.0430 0.4512 0.0976
    $m(h_{4j1})$ 0.0645 0.0645 0.7663 0.1048
    $m(h_{4j2})$ 0.0573 0.0573 0.7878 0.0976
    $m(h_{4j3})$ 0 0 0.9024 0.0976
    下载: 导出CSV

    表  9  空间域融合后的结果(复杂状态实验1)

    Table  9  Results of fusion in the spatial domain (Complex condition 1)

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{j1})$ 0.1003 0.0087 0.8910 0
    $m(h_{j2})$ 0.0567 0.0056 0.9377 0
    $m(h_{j3})$ 0.0151 0.0026 0.9823 0
    下载: 导出CSV

    表  10  证据体的基本概率赋值(复杂状态实验2)

    Table  10  Basic probability distribution of evidence bodies (Complex condition 2)

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{1j1})$ 0.3027 0.2951 0.2951 0.1027
    $m(h_{1j2})$ 0 0.4487 0.4487 0.1027
    $m(h_{1j3})$ 0 0.4527 0.4527 0.0946
    $m(h_{2j1})$ 0.4487 0 0.4487 0.1027
    $m(h_{2j2})$ 0.1415 0.6144 0.1455 0.1027
    $m(h_{2j3})$ 0 0.9054 0 0.0946
    $m(h_{3j1})$ 0.4487 0 0.4487 0.1027
    $m(h_{3j2})$ 0.1900 0.5174 0.1900 0.1027
    $m(h_{3j3})$ 0 0.9054 0 0.0946
    $m(h_{4j1})$ 0.3557 0.3557 0.1859 0.1027
    $m(h_{4j2})$ 0.2425 0.2425 0.4123 0.1027
    $m(h_{4j3})$ 0.4123 0.4123 0.0808 0.0946
    下载: 导出CSV

    表  11  空间域融合后的结果(复杂状态实验2)

    Table  11  Results of fusion in the spatial domain (Complex condition 2)

    $h_1$ $h_2$ $h_3$ $U$
    $m(h_{j1})$ 0.6109 0.0162 0.3728 0
    $m(h_{j2})$ 0.0164 0.7998 0.1837 0
    $m(h_{j3})$ 0.0001 0.9973 0.0026 0
    下载: 导出CSV

    表  12  D-S推理方法和贝叶斯推理方法的对比

    Table  12  Comparison of D-S inference and the Bayesian inference methods

    对比实验 理论方法 $h_1$ $h_2$ $h_3$ $U$
    对比实验1 D-S理论 0.0001 0 0.9999 0
    贝叶斯方法 0.2894 0.0415 0.6692 -
    对比实验2 D-S理论 0.0001 0.9863 0.0136 0
    贝叶斯方法 0.2705 0.5622 0.1673 -
    下载: 导出CSV
  • [1] Wong K, Patzelt M, Poulette B, Hathaway R. Scenario-based learning in a MOOC specialization capstone on software product management. In: Proceedings of the IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C). Buenos Aires, Argentina: IEEE, 2017. 317-318 https://www.researchgate.net/publication/318123881_Scenario-Based_Learning_in_a_MOOC_Specialization_Capstone_on_Software_Product_Management
    [2] Staubitz T, Willems C, Hagedorn C, Meinel C. The gamification of a MOOC platform. In: Proceedings of the 2017 IEEE Global Engineering Education Conference (EDUCON). Athens, Greece: IEEE, 2017. 383-892 http://www.researchgate.net/publication/316790928_The_Gamification_of_a_MOOC_Platform
    [3] Lei C U, Yeung Y C A, Kwok T T O, Lau R, Ang A. Leveraging videos and forums for small-class learning experience in a MOOC environment. In: Proceedings of the 2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE). Bangkok, Thailand: IEEE, 2016. 409-411
    [4] Luo H, Millet A, Alley R, Zuo M. Dealing with ethical issues in MOOC design and delivery: a case study. In: Proceedings of the 2016 Blended Learning: Aligning Theory with Practices. ICBL 2016. Lecture Notes in Computer Science, vol. 9757. Cham, Switzerland: Springer, 2016. 128-138 doi: 10.1007/978-3-319-41165-1_12
    [5] Kearney R C, Premaraj S, Smith B M, Olson G W, Williamson A E, Romanos G. Massive open online courses in dental education: two viewpoints: viewpoint 1: massive open online courses offer transformative technology for dental education and viewpoint 2: massive open online courses are not ready for primetime. Journal of Dental Education, 2016, 80(2): 121-127 http://cn.bing.com/academic/profile?id=1df6bb18edbf1e3bbb7a357ca961c6b1&encoded=0&v=paper_preview&mkt=zh-cn
    [6] Brinton C G, Buccapatnam S, Wong F M F, Chiang M, Poor H V. Social learning networks: efficiency optimization for MOOC forums. In: Proceedings of the IEEE INFOCOM 2016 — The 35th Annual IEEE International Conference on Computer Communications. San Francisco, USA: IEEE, 2016. 1-9 http://ieeexplore.ieee.org/document/7524579/
    [7] Jaouedi N, Boujnah N, Htiwich O, Bouhlel M S. Human action recognition to human behavior analysis. In: Proceedings of the 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT). Hammamet, Tunisia: IEEE, 2016. 263-266 http://ieeexplore.ieee.org/abstract/document/7939877/
    [8] Xiao B, Georgiou P, Baucom B, Narayanan S S. Head motion modeling for human behavior analysis in dyadic interaction. IEEE Transactions on Multimedia, 2015, 17(7): 1107-1119 doi: 10.1109/TMM.2015.2432671
    [9] Tsai H C, Chuang C H, Tseng S P, Wang J F. The optical flow-based analysis of human behavior-specific system. In: Proceedings of the 1st International Conference on Orange Technologies (ICOT). Tainan, China: IEEE, 2013. 214-218 http://ieeexplore.ieee.org/document/6521195/
    [10] Batchuluun G, Kim J H, Hong H G, Kang J K, Park K R. Fuzzy system based human behavior recognition by combining behavior prediction and recognition. Expert Systems with Applications, 2017, 81: 108-133 doi: 10.1016/j.eswa.2017.03.052
    [11] Alonso J B, Cabrera J, Travieso C M, López-de-Ipiña K. First approach to continuous tracking of emotional temperature. In: Proceedings of the 4th International Work Conference on Bioinspired Intelligence (IWOBI). San Sebastian, Spain: IEEE, 2015. 177-184 http://ieeexplore.ieee.org/document/7160163/
    [12] Hwang K A, Yang C H. Attentiveness assessment in learning based on fuzzy logic analysis. Expert Systems with Applications, 2009, 36(3): 6261-6265 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC025018788
    [13] Durucan E, Ebrahimi T. Moving object detection between multiple and color images. In: Proceedings of the 2013 IEEE Conference on Advanced Video and Signal Based Surveillance. Miami, FL, USA: IEEE, 2003. 243-251 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=1217928
    [14] Hu M J, Liu Z, Zhang J Y, Zhang G J. Robust object tracking via multi-cue fusion. Signal Processing, 2017, 139: 86-95 http://cn.bing.com/academic/profile?id=90bbb8ab657b7b82722022921f518324&encoded=0&v=paper_preview&mkt=zh-cn
    [15] Zhang P, Zhuo T, Huang W, Chen K L, Kankanhalli M. Online object tracking based on CNN with spatial-temporal saliency guided sampling. Neurocomputing, 2017, 257: 115-127 doi: 10.1016/j.neucom.2016.10.073
    [16] Bilge Y C, Kaya F, Cinbis N İ, Celikcan U, Sever H. Anomaly detection using improved background subtraction. In: Proceedings of the 25th Signal Processing and Communications Applications Conference (SIU). Antalya, Turkey: IEEE, 2017. 1-4 http://ieeexplore.ieee.org/document/7960592/
    [17] Elyounsi A, Tlijani H, Bouhlel M S. Shape detection by mathematical morphology techniques for radar target classification. In: Proceedings of the 17th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA). Sousse, Tunisia: IEEE, 2016. 352-356 https://ieeexplore.ieee.org/document/7952078/
    [18] Shi X S, Huang Y J, Liu Y G. Text on Oracle rubbing segmentation method based on connected domain. In: Proceedings of the 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference. Xi'an, China: IEEE, 2016. 414-418 http://ieeexplore.ieee.org/abstract/document/7867245/
    [19] Shi H B, Yang S Y, Cao Z L, Pan W, Li W H. An improved evidence combination method of D-S theory. In: Proceedings of the 2016 International Symposium on Computer, Consumer and Control. Xi'an, China: IEEE, 2016. 319-322 http://ieeexplore.ieee.org/document/7545200/
    [20] Chu Q, Ouyang W L, Li H S, Wang X G, Liu B, Yu N H. Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. In: Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE. 2017. 4846-4855
  • 加载中
图(15) / 表(12)
计量
  • 文章访问数:  3234
  • HTML全文浏览量:  698
  • PDF下载量:  357
  • 被引次数: 0
出版历程
  • 收稿日期:  2017-07-26
  • 录用日期:  2018-07-15
  • 刊出日期:  2020-04-24

目录

    /

    返回文章
    返回