非接觸式心律量測和人工智慧被應用於各種領域，包括心理上的疲勞預測。麻省理工學院的電腦科學與人工智慧實驗室(CSAIL)在2012年提出的歐拉影像放大法（Eulerian Video Magnification）可放大影片中的細微顏色和位移變化，並可應用於光體積變化描記圖法（Photoplethysmography）抓取心跳間期。
在本研究中，我們結合注意區域(Region of Interest)和區域峰值偵測演算法，提高在使用歐拉影像技術時心跳預測的準確性，並透過在受試者進行連續性電腦疲勞實驗時收集心律特徵來訓練分類器-支援向量機(Support Vector Machine)模型和邏輯斯迴歸(Logistic Regression)-並預測受試者的心理疲勞。
本研究第一部分發現使用歐拉方法中的YIQ色彩光譜影像放大值利用來心律預測上，Y分量比I分量更準確；結果表示錄影時間增加對心跳預測更準確且相機幀數30比60有更好的心跳預測；結果也顯示使用GoPro高速相機幀數30進行錄影且環境亮度約為1500勒克斯時，得到之心跳與血氧計相比的錯誤率為5％內。
第二部分中，我們依據每位受試者異常心跳間隔值的刪除數量小於30者，從完整資料庫30位電腦疲勞實驗受試者中篩選出14位為數據較優的資料庫，刪除異常值可過濾部分因受試者非自主移動造成的雜訊。總體而言，當使用來自較優資料庫當輸入時，預測結果優於隨機猜測，但在所有二元分類中，16個輸入集中有3個的結果是比隨機猜測還要差；在邏輯斯分類中，我們發現心律變異性(heart rate variability)的頻域參數LF / HF是影響疲勞預測的重要參數。
未來應進一步測試多攝影機同時拍攝，且於臉部的特徵上放置有色標記，以追蹤臉部的注意區域。

Non-contact heart rate measurement and artificial intelligence have been widely utilized in multiple applications including mental fatigue detection. The Eulerian Video Magnification (EVM) algorithm introduced in 2012 can be used to magnify subtle color variations or small motions in videos and cardiac features can be extracted through the photoplethysmography (PPG) method.
In this study, we intended to improve the accuracy of heart rate estimation when using EVM methods and train a support vector machine (SVM) model to detect mental fatigue by collecting cardiac features while a subject was doing a computer fatigue experiment.
Combining region of interest and peak detection algorithm, we found out that the Y component in YIQ color space was more consistent than the I component in terms of heart rate prediction. The result also demonstrated the error rate of heart rate extraction compared to oximeter was 5% when the illumination level was higher than 1500 lux and GoPro Hero 6 was set to 30 frames per second (fps).
We collected cardiac features from 30 subjects (full set) during the fatigue experiment and selected 14 (selected set) out of the full set based on the number of deleted heartbeat intervals using outlier filtering. Overall, we achieved better than random guess predictions when using Eulerian-HRV parameters from the selected set. However, 3 out of 16 results using SVM and logistic regression were worse than random guess when using binary classification. In binary logistic regression, we found out that the LF/HF ratio is a strong identifier of fatigue state prediction.
In the future, multi-camera recording should be further tested. In addition, placing a colored mark can help a camera keep tracking the landmarks on the face.

[1] J. Healey and R. Picard, "SmartCar: detecting driver stress," in Pattern Recognition, 2000. Proceedings. 15th International Conference on, 2000, vol. 4, pp. 218-221, 236: IEEE.
[2] N. Hjortskov, D. Rissén, A. K. Blangsted, N. Fallentin, U. Lundberg, and K. Søgaard, "The effect of mental stress on heart rate variability and blood pressure during computer work," European journal of applied physiology, vol. 92, no. 1-2, pp. 84-89, 2004.
[3] J. Sztajzel, "Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system," Swiss medical weekly, vol. 134, no. 35-36, pp. 514-522, 2004.
[4] 翁根本, 何慈育, 歐善福, 林竹川, and 謝凱生, "心律變動性分析," 臺灣醫界, vol. 52, no. 6, pp. 290-293, 2009.
[5] V. Pichot, E. Bourin, F. Roche, M. Garet, J. M. Gaspoz, D. Duverney, A. Antoniadis, J. R. Lacour, and J. C. Barthelemy, "Quantification of cumulated physical fatigue at the workplace," (in English), Pflugers Archiv-European Journal of Physiology, vol. 445, no. 2, pp. 267-272, Nov 2002.
[6] M. Hassan, A. Malik, D. Fofi, N. Saad, B. Karasfi, Y. Ali, and F. Meriaudeau, "Heart rate estimation using facial video: a review," Biomedical Signal Processing and Control, vol. 38, pp. 346-360, 2017.
[7] C. Wang, T. Pun, and G. Chanel, "A Comparative Survey of Methods for Remote Heart Rate Detection From Frontal Face Videos," Frontiers in bioengineering and biotechnology, vol. 6, 2018.
[8] F. Adib, H. Mao, Z. Kabelac, D. Katabi, and R. C. Miller, "Smart homes that monitor breathing and heart rate," in Proceedings of the 33rd annual ACM conference on human factors in computing systems, 2015, pp. 837-846, 100: ACM.
[9] M. Z. Poh, D. J. McDuff, and R. W. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," (in English), Optics Express, Article vol. 18, no. 10, pp. 10762-10774, May 2010.
[10] M. Z. Poh, D. J. McDuff, and R. W. Picard, "Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam," (in English), Ieee Transactions on Biomedical Engineering, Article vol. 58, no. 1, pp. 7-11, Jan 2011.
[11] M. Villarroel, A. Guazzi, J. Jorge, S. Davis, P. Watkinson, G. Green, A. Shenvi, K. McCormick, and L. Tarassenko, "Continuous non-contact vital sign monitoring in neonatal intensive care unit," Healthcare technology letters, vol. 1, no. 3, pp. 87-91, 2014.
[12] D. McDuff, S. Gontarek, and R. Picard, "Remote measurement of cognitive stress via heart rate variability," in Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, 2014, pp. 2957-2960, 71: IEEE.
[13] E. Gambi, A. Agostinelli, A. Belli, L. Burattini, E. Cippitelli, S. Fioretti, P. Pierleoni, M. Ricciuti, A. Sbrollini, and S. Spinsante, "Heart Rate Detection Using Microsoft Kinect: Validation and Comparison to Wearable Devices," Sensors, vol. 17, no. 8, p. 1776, 2017.
[14] J. L. Alonso-Cuevas and R. E. Sanchez-Yanez, "Simplified heart rate measurement from video," in Proceedings of the 3rd International Workshop on Interactive and Spatial Computing, 2018, pp. 23-28, 0: ACM.
[15] M. Rapczynski, P. Werner, F. Saxen, and A. Al-Hamadi, "How the Region of Interest Impacts Contact Free Heart Rate Estimation Algorithms," in 2018 25th IEEE International Conference on Image Processing (ICIP), 2018, pp. 2027-2031, 0: IEEE.
[16] A. Sikdar, S. K. Behera, and D. P. Dogra, "Computer-vision-guided human pulse rate estimation: a review," IEEE reviews in biomedical engineering, vol. 9, pp. 91-105, 2016.
[17] Q. Zhang, G.-q. Xu, M. Wang, Y. Zhou, and W. Feng, "Webcam based non-contact real-time monitoring for the physiological parameters of drivers," in Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2014 IEEE 4th Annual International Conference on, 2014, pp. 648-652, 15: IEEE.
[18] S. Sanyal and K. K. Nundy, "Algorithms for monitoring heart rate and respiratory rate from the video of a user’s face," IEEE Journal of Translational Engineering in Health and Medicine, 2018.
[19] H. Rahman, M. U. Ahmed, and S. Begum, "Non-Contact Heart Rate Monitoring Using Lab Color Space," in pHealth, 2016, pp. 46-53, 3.
[20] H. Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, and W. Freeman, "Eulerian Video Magnification for Revealing Subtle Changes in the World," (in English), Acm Transactions on Graphics, Article vol. 31, no. 4, p. 8, Jul 2012, Art. no. 65.
[21] P. B. Chambino, "Android-based implementation of Eulerian Video Magnification for vital signs monitoring," 2013.
[22] N. Miljković and D. Trifunović, "Pulse rate assessment: Eulerian video magnification vs. electrocardiography recordings," in Neural Network Applications in Electrical Engineering (NEUREL), 2014 12th Symposium on, 2014, pp. 17-20, 3: IEEE.
[23] K. Alghoul, S. Alharthi, H. Al Osman, and A. El Saddik, "Heart Rate Variability extraction from videos signals: ICA vs. EVM comparison," IEEE Access, vol. 5, pp. 4711-4719, 2017.
[24] A. Sarkar, A. L. Abbott, and Z. Doerzaph, "Assessment of psychophysiological characteristics using heart rate from naturalistic face video data," in Biometrics (IJCB), 2014 IEEE International Joint Conference on, 2014, pp. 1-6, 4: IEEE.
[25] A. Sarkar, A. L. Abbott, Z. Doerzaph, and K. Sykes, "Evaluation of video magnification for nonintrusive heart rate measurement," in Control, Measurement and Instrumentation (CMI), 2016 IEEE First International Conference on, 2016, pp. 494-498, 5: IEEE.
[26] J. H. Klaessens, M. van den Born, A. van der Veen, J. Sikkens-van de Kraats, F. A. van den Dungen, and R. M. Verdaasdonk, "Development of a baby friendly non-contact method for measuring vital signs: first results of clinical measurements in an open incubator at a neonatal intensive care unit," in Proc. SPIE 8935, Advanced Biomed. and Clin. Diagnostic Syst. XII, 2014, pp. 89351p-1-89351P-7, 14.
[27] E. Niforatos, I. Elhart, and M. Langheinrich, "Public displays for monitoring and improving community wellbeing," in Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, 2015, pp. 775-778, 2: ACM.
[28] A. C. L. Ngo, J. See, and R. C. W. Phan, "Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application," (in English), Ieee Transactions on Affective Computing, Article vol. 8, no. 3, pp. 396-411, Jul-Sep 2017.
[29] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[30] M. Patel, S. Lal, D. Kavanagh, and P. Rossiter, "Applying neural network analysis on heart rate variability data to assess driver fatigue," Expert systems with Applications, vol. 38, no. 6, pp. 7235-7242, 2011.
[31] M. M. Bundele and R. Banerjee, "Detection of fatigue of vehicular driver using skin conductance and oximetry pulse: a neural network approach," in Proceedings of the 11th International Conference on Information Integration and web-based applications & services, 2009, pp. 739-744, 45: ACM.
[32] D. Wang, P. Shen, T. Wang, and Z. Xiao, "Fatigue detection of vehicular driver through skin conductance, pulse oximetry and respiration: A random forest classifier," in Communication Software and Networks (ICCSN), 2017 IEEE 9th International Conference on, 2017, pp. 1162-1166, 0: IEEE.
[33] J. Zhai and A. Barreto, "Stress detection in computer users based on digital signal processing of noninvasive physiological variables," in Engineering in Medicine and Biology Society, 2006. EMBS'06. 28th Annual International Conference of the IEEE, 2006, pp. 1355-1358, 201: IEEE.
[34] V. Sandulescu, S. Andrews, D. Ellis, N. Bellotto, and O. M. Mozos, "Stress detection using wearable physiological sensors," in International Work-Conference on the Interplay Between Natural and Artificial Computation, 2015, pp. 526-532, 21: Springer.
[35] M. A. Boksem and M. Tops, "Mental fatigue: costs and benefits," Brain research reviews, vol. 59, no. 1, pp. 125-139, 2008.
[36] J. H. Friedman, R. G. Brown, C. Comella, C. E. Garber, L. B. Krupp, J. S. Lou, L. Marsh, L. Nail, L. Shulman, and C. B. Taylor, "Fatigue in Parkinson's disease: a review," Movement disorders: official journal of the Movement Disorder Society, vol. 22, no. 3, pp. 297-308, 2007.
[37] F. Edition, "Diagnostic and statistical manual of mental disorders," Arlington: American Psychiatric Publishing, 2013.
[38] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, 2001, vol. 1, pp. I-I, 16610: IEEE.
[39] Y. Hussain and A. A. Shkara, "Speed up Eulerian Video Motion Magnification," Kurdistan Journal of Applied Research, vol. 2, no. 3, 2017.
[40] D. Van Der Linden, M. Frese, and S. Sonnentag, "The impact of mental fatigue on exploration in a complex computer task: Rigidity and loss of systematic strategies," Human Factors, vol. 45, no. 3, pp. 483-494, 2003.
[41] K. Kaida, M. Takahashi, T. Åkerstedt, A. Nakata, Y. Otsuka, T. Haratani, and K. Fukasawa, "Validation of the Karolinska sleepiness scale against performance and EEG variables," Clinical Neurophysiology, vol. 117, no. 7, pp. 1574-1581, 2006.
[42] T. Åkerstedt and M. Gillberg, "Subjective and objective sleepiness in the active individual," International Journal of Neuroscience, vol. 52, no. 1-2, pp. 29-37, 1990.
[43] P. Philip, P. Sagaspe, N. Moore, J. Taillard, A. Charles, C. Guilleminault, and B. Bioulac, "Fatigue, sleep restriction and driving performance," Accident Analysis & Prevention, vol. 37, no. 3, pp. 473-478, 2005.
[44] A. Sahayadhas, K. Sundaraj, and M. Murugappan, "Drowsiness detection during different times of day using multiple features," Australasian physical & engineering sciences in medicine, vol. 36, no. 2, pp. 243-250, 2013.
[45] C.-W. Hsu, C.-C. Chang, and C.-J. Lin, "A practical guide to support vector classification," 2003.
[46] B. W. Matthews, "Comparison of the predicted and observed secondary structure of T4 phage lysozyme," Biochimica et Biophysica Acta (BBA)-Protein Structure, vol. 405, no. 2, pp. 442-451, 1975.