| 138 | 0 | 46 |
| 下载次数 | 被引频次 | 阅读次数 |
随着人工智能技术的快速发展，情感识别已成为智能人机交互领域的重要研究方向。目前，大多数情感识别研究主要依赖面部表情和语音信息，但这些信息易受环境因素影响。相比之下，人体动作不仅能够有效传递情感信息，还能在一定程度上避免环境干扰。因此基于九轴惯性传感器采集了六种肢体动作的惯性数据，并结合手动标注与定量评分法生成情感标签。针对该情感识别任务，提出了一种融合多阶段特征的轻量级神经网络，以实现基于人体动作的情感识别。该方法利用MobileNet v2 网络提取深层特征，并结合局部二值模式（local binary pattern， LBP）特征增强浅层纹理信息，以减少特征丢失问题。提取的特征经过特征级融合后，被馈送至线性支持向量机（support vector machine， SVM）进行分类，以提升情感识别的准确性。实验在验证集和测试集上的准确率分别达到了99.3%和98.84%，该结果表明在计算复杂度较低的情况下，该模型仍能保持较高的识别精度和效率。模型在JAFFE和CK+数据集上同样达到较高水平，与传统方法相比，该模型在准确性和计算效率方面均表现出显著优势，展现了其在实际情感识别应用中的潜力。
Abstract：With the rapid advancement of artificial intelligence technology， emotion recognition has become a key research area in intelligent human-computer interaction. Currently， most emotion recognition studies primarily rely on facial expressions and speech， but these modalities are susceptible to environmental factors. In contrast， human body movements can effectively convey emotional information while mitigating environmental interference to a certain extent. Therefore， inertial data from six types of limb movements were collected using a nine-axis inertial sensor， and emotion labels were generated through a combination of manual annotation and a quantitative scoring method. To address this emotion recognition task， a lightweight neural network with multi-stage feature fusion is proposed to achieve emotion recognition based on human body movements. The proposed method employs the MobileNet v2 network to extract deep features while incorporating Local Binary Pattern (LBP) features to enhance shallow texture information and reduce feature loss. The extracted features， after feature-level fusion， are fed into a linear Support Vector Machine (SVM) for classification， enhancing the accuracy of emotion recognition. The experiment achieved accuracies of 99.3% on the validation set and 98.84% on the test set， demonstrating that it maintains high accuracy and efficiency even with low computational complexity. The model also exhibits high performance on the JAFFE and CK+ datasets. Compared with traditional methods， it achieves significant improvements in both accuracy and computational efficiency， highlighting its potential for practical applications in emotion recognition.
[1] LI M， SU Y S， HUANG H Y， et al. Language-specific representation of emotion-concept knowledge causally supports emotion inference[J]. iScience， 2024， 27(12): 111401 [2025-3-24]https://www.cell.com/iscience/fulltext/S2589-0042(24)02626-9
[2] 雷婕， 丁亚平. 面部表情: 一些争论[J]. 心理科学进展， 2013， 21(10): 1749-1754. LEI J， DING Y P. Facial expressions: some arguments[J]. Advances in Psychological Science， 2013， 21(10): 1749-1754. (in Chinese) [2025-3-24]https://journal.psych.ac.cn/xlkxjz/CN/10.3724/SP.J.1042.2013.01749
[3] 于文华， 鲁忠义. 手势认知功能研究的新视角: “空间化” 手势假设[J]. 心理科学进展， 2020， 28(3): 426-433.
[4] 张丹丹， 赵婷， 柳昀哲， 等. 恐惧情绪面孔和身体姿势加工的比较: 事件相关电位研究[J]. 心理学报， 2015， 47(8): 963-970.
[5] 赵春晓， 江光荣， 林秀彬. 心理咨询中的非言语行为[J]. 心理科学进展， 2016，24(8): 1257-1265.
[6] DEBNATH T， REZA M M， RAHMAN A， et al. Four-layer ConvNet to facial emotion recognition with minimal epochs and the significance of data diversity[J]. Scientific Reports， 2022， 12(1): 6991. [2025-3-24]https://www.nature.com/articles/s41598-022-11173-0
[7] MEHENDALE N. Facial emotion recognition using convolutional neural networks (FERC)[J]. SN Applied Sciences， 2020， 2(3): 446. [2025-3-24]https://link.springer.com/article/10.1007/s42452-020-2234-1
[8] LI B， LIMA D. Facial expression recognition via ResNet-50[J]. International Journal of Cognitive Computing in Engineering， 2021， 2: 57-64. [2025-3-24]https://www.sciencedirect.com/science/article/pii/S2666307421000073
[9] ANVARJON T， Mustaqeem， KWON S. Deep-net: a lightweight CNN-based speech emotion recognition system using deep frequency features[J]. Sensors， 2020， 20(18): 5212. [2025-3-24]https://www.mdpi.com/1424-8220/20/18/5212
[10] PENG Z X， LU Y， PAN S F， et al. Efficient speech emotion recognition using multi-scale CNN and attention[C]//Proceedings of the ICASSP 2021 - 2021 IEEE International Conference on Acoustics， Speech and Signal Processing (ICASSP)， June 6-11， 2021. Toronto， ON， Canada. New York: IEEE， 2021: 3020-3024. [2025-3-24]https://ieeexplore.ieee.org/document/9414286
[11] LUNA-JIMéNEZ C， GRIOL D， CALLEJAS Z， et al. Multimodal emotion recognition on RAVDESS dataset using transfer learning[J]. Sensors， 2021， 21(22): 7665. [2025-3-24]https://www.mdpi.com/1424-8220/21/22/7665
[12] XU S H， FANG J， HU X P， et al. Emotion recognition from gait analyses: current research and future directions[J]. IEEE Transactions on Computational Social Systems， 2022， 11(1): 363-377. [2025-3-24]https://ieeexplore.ieee.org/abstract/document/9969993
[13] GROSS M M， CRANE E A， FREDRICKSON B L. Effort-Shape and kinematic assessment of bodily expression of emotion during gait[J]. Human Movement Science， 2012， 31(1): 202-221. [2025-3-24]https://www.sciencedirect.com/science/article/abs/pii/S0167945711000741?via%3Dihub
[14] KLEINSMITH A， BIANCHI-BERTHOUZE N. Affective body expression perception and recognition: a survey[J]. IEEE Transactions on Affective Computing， 2013， 4(1): 15-33. [2025-3-24]https://ieeexplore.ieee.org/document/6212434/
[15] CASTELLANO G， VILLALBA S D， CAMURRI A. Recognising human emotions from body movement and gesture dynamics[C]// Affective Computing and Intelligent Interaction. Berlin， Heidelberg: Springer Berlin Heidelberg， 2007: 71-82. [2025-3-24]https://link.springer.com/chapter/10.1007/978-3-540-74889-2＿7
[16] SAHA S， DATTA S， KONAR A， et al. A study on emotion recognition from body gestures using Kinect sensor[C]//Proceedings of the 2014 International Conference on Communication and Signal Processing， April 3-5， 2014， Melmaruvathur， India. New York: IEEE， 2014: 56-60. [2025-3-24]https://ieeexplore.ieee.org/abstract/document/6949798/
[17] AVIEZER H， TROPE Y， TODOROV A. Body cues， not facial expressions， discriminate between intense positive and negative emotions[J]. Science， 2012， 338(6111): 1225-1229. [2025-3-24]https://www.science.org/doi/abs/10.1126/science.1224313
[18] Yang Z， Kay A， Li Y， et al. Pose-based body language recognition for emotion and psychiatric symptom interpretation[C]//Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR)， January 10-15， 2021， Milan， Italy. New York: IEEE， 2021. [2025-3-24]https://ieeexplore.ieee.org/abstract/document/9412591/
[19] LI S， CUI L Q， ZHU C Y， et al. Emotion recognition using Kinect motion capture data of human gaits[J]. PeerJ， 2016， 4: e2364. [2025-3-24]https://peerj.com/articles/2364/
[20] LUO Y， YE J B， ADAMS R B Jr， et al. ARBEE: towards automated recognition of bodily expression of emotion in the wild[J]. International Journal of Computer Vision， 2020， 128(1): 1-25. [2025-3-24]https://link.springer.com/article/10.1007/s11263-019-01215-y
[21] SHENG W J， LI X D. Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network[J]. Pattern Recognition， 2021， 114: 107868. [2025-3-24]https://www.sciencedirect.com/science/article/abs/pii/S0031320321000558
[22] ZHANG Z， SONG Y F， CUI L Q， et al. Emotion recognition based on customized smart bracelet with built-in accelerometer[J]. PeerJ， 2016， 4: e2258. [2025-3-24]https://peerj.com/articles/2258/
[23] HASHMI M A， RIAZ Q， ZEESHAN M， et al. Motion reveal emotions: identifying emotions from human walk using chest mounted smartphone[J]. IEEE Sensors Journal， 2020， 20(22): 13511-13522. [2025-3-24]https://ieeexplore.ieee.org/abstract/document/9123437
[24] SHU L， YU Y， CHEN W Z， et al. Wearable emotion recognition using heart rate data from a smart bracelet[J]. Sensors， 2020， 20(3): 718. [2025-3-24]https://www.mdpi.com/1424-8220/20/3/718
[25] GRAVINA R， LI Q M. Emotion-relevant activity recognition based on smart cushion using multi-sensor fusion[J]. Information Fusion， 2019， 48: 1-10. [2025-3-24]https://www.sciencedirect.com/science/article/abs/pii/S1566253518301064
[26] NOROOZI F， CORNEANU C A， KAMI?SKA D， et al. Survey on emotional body gesture recognition[J]. IEEE Transactions on Affective Computing， 2021， 12(2): 505-523. [2025-3-24]https://ieeexplore.ieee.org/abstract/document/8493586/
[27] MERCADANTE E， WITKOWER Z， TRACY J L. The psychological structure， social consequences， function， and expression of pride experiences[J]. Current Opinion in Behavioral Sciences， 2021， 39: 130-135. [2025-3-24]https://www.sciencedirect.com/science/article/abs/pii/S2352154621000589
[28] FEESE S， ARNRICH B， TR? STER G， et al. Quantifying behavioral mimicry by automatic detection of nonverbal cues from body motion[C]//Proceedings of the 2012 International Conference on Privacy， Security， Risk and Trust and 2012 International Confernece on Social Computing， September 3-5， 2012， Amsterdam， Netherlands. New York: IEEE， 2012: 520-525. [2025-3-24]https://ieeexplore.ieee.org/abstract/document/6406302/
[29] KANG J， KIM J， LEE S， et al. Transition activity recognition using fuzzy logic and overlapped sliding window-based convolutional neural networks[J]. The Journal of Supercomputing， 2020， 76(10): 8003-8020. [2025-3-24]https://link.springer.com/article/10.1007/s11227-018-2470-y
[30] 黄仕松， 李丹杨， 陈星， 等. 基于多标签随机游走的选择性集成方法用于表情识别[J]. Software Engineering and Applications， 2022， 11: 1344.
[31] 程换新， 王雪， 程力， 等. 基于 CNN 和 LSTM 的人脸表情识别模型设计[J]. 电子测量技术， 2021， 44(17): 160-164.
[32] 唐浩， 黄伟鹏， 李哲媛， 等. 基于改进的卷积神经网络的负面表情识别方法[J]. 华中科技大学学报(自然科学版)， 2015， 43(S1): 457-460.
[33] GAUTAM C， SEEJA K R. Facial emotion recognition using Handcrafted features and CNN[J]. Procedia Computer Science， 2023， 218: 1295-1303. [2025-3-24]https://www.sciencedirect.com/science/article/pii/S1877050923001084
[34] MEENA G， MOHBEY K K， KUMAR S， et al. Image-based sentiment analysis using InceptionV3 transfer learning approach[J]. SN Computer Science， 2023， 4(3): 242. [2025-3-24]https://link.springer.com/article/10.1007/s42979-023-01695-3
基本信息:
中图分类号:TP212.9;TP18
引用信息:
[1]许涵坤,郭明,周坤,等.基于多阶段特征融合的人体动作情感识别方法[J].南通大学学报(自然科学版)().
基金信息:
国家自然科学基金面上基金项目（62173175、61877033）;国家自然科学基金青年科学基金项目（62406132、61903170）; 山东省自然科学基金项目（ZR2024MF032）
2025-04-24
2025-04-24
2025-04-24