簡易檢索 / 詳目顯示

回結果列表
研究生： 宋承瀚
Sung, Chen-Han
論文名稱： 一個用於微表情識別的改良型注意力機制網路
An Improved Attention Network for Micro-expression Recognition
指導教授： 戴顯權
Tai, Shen-Chuan
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 72
中文關鍵詞： 微表情識別 、深度學習 、注意力機制 、ASPP block 、Focal loss
外文關鍵詞： micro-expression recognition, deep learning, attention mechanism, ASPP block, Focal loss
相關次數： 點閱：216  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 微表情是人在隱藏情緒時會產生的自發性肌肉運動。這個臉部肌肉運動會透露出人類所隱藏的情緒。通常這種情緒是無法作假的，所以若能辨識出人類的微表情所表達的情緒，便能將其應用在許多方面，比如說: 商業協商、謊言辨識、犯罪偵測等等…。
    本論文提出一個改良型注意力機制的深度學習網路。該網路是基於卷積神經網路，並加上能夠看到多尺度特徵的改良型Atrous Spatial Pyramid Pooling (ASPP) block和改良的注意力機制模型來辨識微表情的情緒是什麼。本論文所使用的資料集為Spontaneous actions and micro-movements (SAMM) dataset和Chinese academy of sciences micro-expression Ⅱ (CASME Ⅱ) dataset。兩個資料集都有資料不平均的問題存在。因此模型採用的是Focal loss作為辨識結果的損失函數，可以對樣本不均衡的資料集進行一個比較正確的分類，能夠讓資料在訓練的時候做到更好的優化。

    Micro-expression is a spontaneous facial muscle movement that occurs when people are trying to hide their emotions. The muscle movement will reveal the hidden emotions. These emotions cannot be faked, so if people can recognize the emotions expressed in a human micro-expression, they can apply it in many ways, such as business negotiations, lie detection, crime detection.
    This Thesis proposes an improved attention mechanism deep learning network for micro-expression recognition. The network is based on a convolution neural network, an Atrous Spatial Pyramid Pooling (ASPP) block which can extract multi-scale features and an improved attention mechanism module to recognize what the emotion of micro-expression is. The experiment is trained and tested on the Spontaneous actions and micro-movements (SAMM) dataset and the Chinese academy of sciences micro-expression Ⅱ (CASME Ⅱ) dataset. Both datasets exist as data unbalanced problems. Thus, the model uses Focal loss as the loss function of the identification results, which can be used to classify the unbalanced dataset and optimize the model when training.

    摘 要 i Abstract ii Acknowledgments iii Contents iv List of Tables vii List of Figures viii Chapter 1 Introduction 1 1.1 Micro-expression 1 1.2 Motivation 1 1.3 Overview 3 Chapter 2 Related Works 4 2.1 Overview 4 2.2 Methods of handcraft features 5 2.3 Methods of learning features 6 2.3.1 2D Convolution Neural Network 6 2.3.2 3D Convolution Neural Network 9 2.3.3 Recurrent Neural Network 10 2.4 Attention Module 12 2.4.1 Squeeze-and-Excitation (SE) block 13 2.4.2 Convolution Block Attention Module (CBAM) 14 2.4.3 Coordinate Attention (CA) block 15 2.5 Recurrent connection 18 2.6 Atrous Spatial Pyramid Pooling block 19 Chapter 3 The Proposed Method 22 3.1 Data Preprocessing 23 3.1.1 Cropped face 23 3.1.2 Optical flow map extraction 25 3.1.3 Image Augmentation 32 3.2 Proposed Network Architecture 33 3.2.1 Basic Network 34 3.2.2 Modified Atrous Spatial Pyramid Pooling Block 36 3.2.3 Proposed Attention Module 37 3.2.4 Classification Block 40 3.3 Loss Function 42 3.3.1 Focal Loss 42 Chapter 4 Experimental Results 45 4.1 Dataset 45 4.1.1 Spontaneous actions and micro-movements (SAMM) dataset 47 4.1.2 Chinese academy of sciences micro-expression Ⅱ (CASME Ⅱ) dataset 49 4.2 Parameter and Experimental Setting 51 4.3 Performance Evaluation 52 4.4 Ablation Experiment Result 53 4.4.1 Ablation study of pre-processing 54 4.4.2 Ablation study of proposed network architecture 56 4.5 Experimental Results and Comparison 64 Chapter 5 Conclusion and Future Work 67 5.1 Conclusion 67 5.2 Future Work 68 References 69

    [1] Porter, S. and L. Ten Brinke, Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological science, 2008. 19(5): p. 508-514.
    [2] Ekman, P., Lie catching and microexpressions. The philosophy of deception, 2009. 1(2): p. 5.
    [3] Wu, Q., X. Shen, and X. Fu. The machine knows what you are hiding: an automatic micro-expression recognition system. in international conference on affective computing and intelligent Interaction. 2011. Springer.
    [4] Polikovsky, S., Y. Kameda, and Y. Ohta, Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. 2009.
    [5] Lu, H., K. Kpalma, and J. Ronsin, Motion descriptors for micro-expression recognition. Signal Processing: Image Communication, 2018. 67: p. 108-117.
    [6] Pfister, T., et al. Recognising spontaneous facial micro-expressions. in 2011 international conference on computer vision. 2011. IEEE.
    [7] Wang, Y., et al. Dynamic facial expression recognition using local patch and lbp-top. in 2015 8th International conference on human system interaction (HSI). 2015. IEEE.
    [8] Fukushima, K. and S. Miyake, Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition, in Competition and cooperation in neural nets. 1982, Springer. p. 267-285.
    [9] Sun, B., et al., Dynamic micro-expression recognition using knowledge distillation. IEEE Transactions on Affective Computing, 2020.
    [10] Verma, M., et al., LEARNet: Dynamic imaging network for micro expression recognition. IEEE Transactions on Image Processing, 2019. 29: p. 1618-1627.
    [11] Wang, C., et al., Micro-attention for micro-expression recognition. Neurocomputing, 2020. 410: p. 354-362.
    [12] He, K., et al. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
    [13] Deng, J., et al. Imagenet: A large-scale hierarchical image database. in 2009 IEEE conference on computer vision and pattern recognition. 2009. Ieee.
    [14] Liong, S.-T., et al., Evaluation of the spatio-temporal features and gan for micro-expression recognition system. Journal of Signal Processing Systems, 2020: p. 1-21.
    [15] Goodfellow, I.J., et al., Generative adversarial networks. arXiv preprint arXiv:1406.2661, 2014.
    [16] Gan, Y.S., et al., OFF-ApexNet on micro-expression recognition system. Signal Processing: Image Communication, 2019. 74: p. 129-139.
    [17] Chen, B., et al., Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition. Information, 2020. 11(8): p. 380.
    [18] Reddy, S.P.T., et al. Spontaneous facial micro-expression recognition using 3D spatiotemporal convolutional neural networks. in 2019 International Joint Conference on Neural Networks (IJCNN). 2019. IEEE.
    [19] Hochreiter, S. and J. Schmidhuber, Long short-term memory. Neural computation, 1997. 9(8): p. 1735-1780.
    [20] Choi, D.Y. and B.C. Song, Facial Micro-Expression Recognition Using Two-Dimensional Landmark Feature Maps. IEEE Access, 2020. 8: p. 121549-121563.
    [21] Vaswani, A., et al., Attention is all you need. arXiv preprint arXiv:1706.03762, 2017.
    [22] Hu, J., L. Shen, and G. Sun. Squeeze-and-excitation networks. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
    [23] Woo, S., et al. Cbam: Convolutional block attention module. in Proceedings of the European conference on computer vision (ECCV). 2018.
    [24] Hou, Q., D. Zhou, and J. Feng, Coordinate attention for efficient mobile network design. arXiv preprint arXiv:2103.02907, 2021.
    [25] Liang, M. and X. Hu. Recurrent convolutional neural network for object recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
    [26] Yu, F. and V. Koltun, Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, 2015.
    [27] Chen, L.-C., et al., Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017.
    [28] King, D.E., Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research, 2009. 10: p. 1755-1758.
    [29] Bradski, G., The OpenCV Library. Dr. Dobb's Journal of Software Tools, 2000.
    [30] Lucas, B.D. and T. Kanade. An iterative image registration technique with an application to stereo vision. 1981. Vancouver, British Columbia.
    [31] Lucas, B.D., Generalized image matching by the method of differences. 1985: Carnegie Mellon University.
    [32] Farnebäck, G. Two-frame motion estimation based on polynomial expansion. in Scandinavian conference on Image analysis. 2003. Springer.
    [33] Lin, T.-Y., et al. Focal loss for dense object detection. in Proceedings of the IEEE international conference on computer vision. 2017.
    [34] Yan, W.-J., et al., CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PloS one, 2014. 9(1): p. e86041.
    [35] Davison, A.K., et al., Samm: A spontaneous micro-facial movement dataset. IEEE transactions on affective computing, 2016. 9(1): p. 116-129.
    [36] See, J., et al. Megc 2019–the second facial micro-expressions grand challenge. in 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). 2019. IEEE.
    [37] Tan, M. and Q. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. in International Conference on Machine Learning. 2019. PMLR.
    [38] Xia, Z., et al., Revealing the invisible with model and data shrinking for composite-database micro-expression recognition. IEEE Transactions on Image Processing, 2020. 29: p. 8590-8605.
    [39] Zhou, L., Q. Mao, and L. Xue. Dual-inception network for cross-database micro-expression recognition. in 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). 2019. IEEE.

    下載圖示
    2026-07-06公開
    QR CODE