簡易檢索 / 詳目顯示

回結果列表
研究生： 黃亘甫
Huang, Hsuan-Fu
論文名稱： 整合可見光影像與熱影像之AI人臉辨識熱感體溫量測系統
Integration of Infrared Thermal Image and Visible Image for AI Face Recognition and Human Temperature Taking System
指導教授： 王駿發
Wang, Jhing-Fa
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 65
中文關鍵詞： 口罩人臉辨識 、口罩偵測 、活體偵測 、體溫測量 、人臉驗證
外文關鍵詞： Masked Face Recognition, Mask Detection, Liveness Detection, Temperature Taking, Face Verification
相關次數： 點閱：156  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 自從新冠肺炎迅速傳播，人們開始重視如何有效防疫，外出時配戴口罩、隨時隨地量測體溫已變成生活中的例行事項。然而口罩遮蔽下的人臉辨識不夠成熟，有些人臉辨識系統仍要求使用者摘除口罩使用，不僅容易形成防疫破口，也降低了人臉辨識的便利性。許多場合的體溫測量還是由人力進行，不只消耗較多時間也增加近距離接觸傳染的機會。
    基於上述的問題，本研究提出整合可見光影像與熱影像之AI人臉辨識熱感體溫量測系統，能夠有效整合口罩人臉辨識、口罩偵測、體溫測量與活體偵測等任務，使防疫工作能更確實有效的落實。系統包含下列特色: (1)提出一種為未遮蔽人臉影像生成模擬口罩遮蔽的方法，以往的人臉辨識系統經常在辨識流程上改良而忽略資料庫，本系統透過使用模擬口罩遮蔽人臉資料庫來訓練人臉辨識模型，使系統可以更好地處理口罩遮蔽下的人臉。 (2)提出強弱相關人臉驗證方法，藉由無遮蔽人臉匹配與口罩遮蔽人臉匹配來提升人臉辨識的準確性，結合此方法與模擬口罩遮蔽人臉資料庫，可將VGGFace2測試中的未遮蔽測試和模擬口罩遮蔽測試的準確度分別提升 4.31% 和6.17%。 (3)使用紅外線熱像儀結合AI技術對人臉區域進行體溫測量，經過數十位使用者測試，系統可正確量測90%以上使用者的體溫。 (4)整合可見光影像與熱影像進行活體偵測，準確度可達95%。 (5)系統另可運行於NVIDIA Jetson Nano開發板與NUWA Kebbi Air機器人上，經由多位使用者在準確性、速度及便利性上進行評分，平均分數達4.57(滿分5)。
    實驗結果顯示在VGGFace2測試中非遮蔽下的人臉辨識正確率可以達到99.79%，口罩遮蔽下的人臉辨識正確率可以達到99.4%。口罩偵測正確率可以達到99.6%。系統對於照片與影片的欺騙具有相當高的識別能力。在實際應用上，本系統可以同時進行多人的口罩人臉辨識、口罩偵測與體溫量測等任務，有效完成防疫工作。

    Since the rapid spread of Novel Coronavirus, people have begun to pay attention to preventing the pandemic. Wearing a mask when going out and taking body temperature anytime and anywhere has become a routine in life. However, masked face recognition is not mature. Some face recognition systems still require users to remove the mask for use. That not only forms an infection control breach but also reduces the convenience of face recognition. On many occasions, temperature taking is still done manually, which consumes more time and increases the chance of close contact.
    Based on the above problems, we propose an Integration of Infrared Thermal Image and Visible Image for AI Face Recognition and Human Temperature Taking System. The system can effectively integrate masked face recognition, mask detection, temperature taking, and liveness detection. With the system, the preventive measures can be implemented more reliably and effectively. The system includes the following features: (1) An approach for generating simulated masked face images for unmasked face images is proposed. In the past, face recognition systems often improve the recognition process and ignore the database. This system uses simulated masked face datasets to train the face recognition model so that it can better deal with masked faces. (2) A strong and weak correlation face verification approach is proposed to improve the performance of face recognition by unmasked face matching and masked face matching. Combining the proposed face verification approach and simulated masked face datasets, the accuracy of the unmasked test and the simulated masked test in the VGGFace2 test set are improved by 4.31% and 6.17%. (3) Using a thermal camera combined with AI technology to take body temperature in the face area, after testing by dozens of users, the system can accurately take the body temperature of more than 90% of users. (4) Integrating visible images and thermal images for liveness detection, the accuracy can reach 95%. (5) The system can also run on the NVIDIA Jetson Nano development kit and the NUWA Kebbi Air robot and is rated by several users in terms of accuracy, speed, and convenience, with an average score of 4.57 out of 5.
    The experimental results show that the accuracy of unmasked face recognition can reach 99.79%, while the accuracy of masked face recognition can achieve 99.4%. The accuracy of mask detection can achieve 99.6%. The system has a high recognition capability for photo and video spoof. In practical applications, the system can perform masked face recognition, mask detection, and temperature taking of multiple people simultaneously to complete the preventive measures effectively.

    中文摘要 I Abstract III 誌謝 V Content VI Table List IX Figure List X Chapter1 Introduction 1 1.1 Background 1 1.2 Motivation 2 1.3 Objectives 2 1.4 Organization 3 Chapter2 Related Works 4 2.1 Review of Masked Face Dataset 4 2.2 Review of Occluded Face Recognition 5 2.3 Review of Liveness Detection 7 Chapter3 AI Face Recognition and Human Temperature Taking System 9 3.1 System Overview 9 3.1.1 Masked Face Image Generation Module 9 3.1.2 Face Detection Module 9 3.1.3 Face Recognition Module 9 3.1.4 Face Verification Module 10 3.1.5 Mask Detection Module 10 3.1.6 Fever & Liveness Detection Module 10 3.2 Masked Face Image Generation Module 11 3.2.1 Frame Overview 11 3.2.2 Dlib Face Detector 11 3.2.3 Face Landmark Analysis 13 3.2.4 Image Synthesis 14 3.3 Face Detection Module 15 3.3.1 Frame Overview 15 3.3.2 Multi-Face Detection Based on SSH Face Detector 15 3.3.3 Face Alignment 21 3.4 Face Recognition Module 22 3.4.1 Frame Overview 22 3.4.2 Facial Feature Extraction Based on ResNet 23 3.4.3 Identity Classification 27 3.5 Face Verification Module 29 3.5.1 Frame Overview 29 3.5.2 Strong & Weak Correlation Verification 29 3.6 Mask Detection Module 31 3.6.1 Frame Overview 31 3.6.2 Confidence in Wearing Masks Based on MobileNetV2 32 3.7 Fever & Liveness Detection Module 37 3.7.1 Frame Overview 37 3.7.2 Homography 38 3.7.3 Facial Region Extraction in Thermal Image 42 Chapter4 Realization on Jetson Nano Development Kit and Kebbi Air Robot 43 4.1 Detail of Realization on Jetson Nano Development Kit 43 4.2 Detail of Realization on Kebbi Air Robot 44 Chapter5 Experimental Results 46 5.1 Experimental Environment 46 5.2 Experimental Results of Face Recognition & Face Verification 46 5.2.1 Training Dataset 46 5.2.2 Unmasked Face Accuracy Test 47 5.2.3 Masked Face Accuracy Test 51 5.2.4 False Acceptance Rate (FAR) and False Rejection Rate (FRR) Test 54 5.3 Experimental Results of Realization on Different Platforms 57 5.3.1 Experimental Results of Realization on the Jetson Nano 57 5.3.2 Experimental Results of Realization on the Kebbi Air Robot 58 5.4 Experimental Results of Mask Detection Module 59 5.4.1 Mask Detection Dataset 59 5.4.2 Mask Detection Test 59 5.5 Experimental Results of Fever & Liveness Detection Module 60 Chapter6 Conclusions and Future Works 61 6.1 Summary 61 6.2 Contributions 61 6.3 Future Works 62 References 63

    [1]TVBS. 指紋打卡憂感染　中市交大改人臉辨識. Available: https://news.tvbs.com.tw/life/1311935.
    [2]聯合報. 台中今公告「外出全程戴口罩」違者最重罰1.5萬. Available: https://udn.com/news/story/7325/5469678.
    [3]ETtoday財經雲. 防疫破口？群創進廠得「脫口罩」人臉辨識　公司回應：已改為刷卡進入. Available: https://finance.ettoday.net/news/1985505.
    [4]Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, "Ms-celeb-1m: A dataset and benchmark for large-scale face recognition," in European conference on computer vision, 2016: Springer, pp. 87-102.
    [5]G. B. Huang, M. Mattar, T. Berg, and E. Learned-Miller, "Labeled faces in the wild: A database forstudying face recognition in unconstrained environments," in Workshop on faces in'Real-Life'Images: detection, alignment, and recognition, 2008.
    [6]D. Yi, Z. Lei, S. Liao, and S. Z. Li, "Learning face representation from scratch," arXiv preprint arXiv:1411.7923, 2014.
    [7]S. Ge, J. Li, Q. Ye, and Z. Luo, "Detecting masked faces in the wild with lle-cnns," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2682-2690.
    [8]Z. Wang et al., "Masked face recognition dataset and application," arXiv preprint arXiv:2003.09093, 2020.
    [9]D. Zeng, R. Veldhuis, and L. Spreeuwers, "A survey of face recognition techniques under occlusion," arXiv preprint arXiv:2006.11366, 2020.
    [10]W. Hariri, "Efficient masked face recognition method during the covid-19 pandemic," arXiv preprint arXiv:2105.03026, 2021.
    [11]I. Cheheb, N. Al-Maadeed, S. Al-Madeed, A. Bouridane, and R. Jiang, "Random sampling for patch-based face recognition," in 2017 5th International Workshop on Biometrics and Forensics (IWBF), 2017: IEEE, pp. 1-5.
    [12]D. S. Trigueros, L. Meng, and M. Hartnett, "Enhancing convolutional neural networks for face recognition with occlusion maps and batch triplet loss," Image and Vision Computing, vol. 79, pp. 99-108, 2018.
    [13]L. Song, D. Gong, Z. Li, C. Liu, and W. Liu, "Occlusion robust face recognition based on mask learning with pairwise differential siamese network," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 773-782.
    [14]F. Zhao, J. Feng, J. Zhao, W. Yang, and S. Yan, "Robust LSTM-autoencoders for face de-occlusion in the wild," IEEE Transactions on Image Processing, vol. 27, no. 2, pp. 778-790, 2017.
    [15]W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, "Sphereface: Deep hypersphere embedding for face recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 212-220.
    [16]H. Wang et al., "Cosface: Large margin cosine loss for deep face recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5265-5274.
    [17]J. Deng, J. Guo, N. Xue, and S. Zafeiriou, "Arcface: Additive angular margin loss for deep face recognition," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4690-4699.
    [18]Y. S. Moon, J. Chen, K. Chan, K. So, and K. Woo, "Wavelet based fingerprint liveness detection," Electronics Letters, vol. 41, no. 20, pp. 1112-1113, 2005.
    [19]G. Pan, L. Sun, Z. Wu, and S. Lao, "Eyeblink-based anti-spoofing in face recognition from a generic webcamera," in 2007 IEEE 11th international conference on computer vision, 2007: IEEE, pp. 1-8.
    [20]K. Kollreider, H. Fronthaler, and J. Bigun, "Verifying liveness by multiple experts in face biometrics," in 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008: Ieee, pp. 1-6.
    [21]J. Määttä, A. Hadid, and M. Pietikäinen, "Face spoofing detection from single images using micro-texture analysis," in 2011 international joint conference on Biometrics (IJCB), 2011: IEEE, pp. 1-7.
    [22]Y. Atoum, Y. Liu, A. Jourabloo, and X. Liu, "Face anti-spoofing using patch and depth-based CNNs," in 2017 IEEE International Joint Conference on Biometrics (IJCB), 2017: IEEE, pp. 319-328.
    [23]G. Pan, Z. Wu, and L. Sun, "Liveness detection for face recognition," Recent advances in face recognition, pp. 109-124, 2008.
    [24]J. Yang, Z. Lei, and S. Z. Li, "Learn convolutional neural network for face anti-spoofing," arXiv preprint arXiv:1408.5601, 2014.
    [25]Dlib library. Available: http://dlib.net/.
    [26]N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), 2005, vol. 1: Ieee, pp. 886-893.
    [27]C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273-297, 1995.
    [28]Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, "Vggface2: A dataset for recognising faces across pose and age," in 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), 2018: IEEE, pp. 67-74.
    [29]M. Najibi, P. Samangouei, R. Chellappa, and L. S. Davis, "Ssh: Single stage headless face detector," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 4875-4884.
    [30]K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint face detection and alignment using multitask cascaded convolutional networks," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
    [31]W. Liu et al., "Ssd: Single shot multibox detector," in European conference on computer vision, 2016: Springer, pp. 21-37.
    [32]K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
    [33]S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," arXiv preprint arXiv:1506.01497, 2015.
    [34]K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
    [35]J. Snell, K. Swersky, and R. S. Zemel, "Prototypical networks for few-shot learning," arXiv preprint arXiv:1703.05175, 2017.
    [36]M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "Mobilenetv2: Inverted residuals and linear bottlenecks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510-4520.
    [37]A. G. Howard et al., "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
    [38]G. Bradski and A. Kaehler, Learning OpenCV: Computer vision with the OpenCV library. " O'Reilly Media, Inc.", 2008.
    [39]Tencent. ncnn. Available: https://github.com/Tencent/ncnn.
    [40]Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep learning face attributes in the wild," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 3730-3738.

    下載圖示
    2026-08-10公開
    QR CODE