研究介紹: 遠程光體積變化描記圖法（rPPG）是一種藉由從影片量測皮膚的血容量變化去提取出心跳訊號的非接觸式技術。近年來，雖然有許多rPPG方法被提出，但對於醫療用途來說皆不夠準確或強健。

研究目標: 此研究的主要目標為開發rPPG方法以改善其心跳率預測之準確度與強健性。

研究方法: 在這篇論文中，我們提出了兩種rPPG方法。第一個方法–Combination of Simple Chrominance signals (CSC) 透過De Haan and Jeanne (2013) 提出的alpha-tuning步驟去結合三個簡單的色度訊號。另一個方法–CHROM model with PRNet (CHROM-PRN)透過Feng et al. (2018) 提出的3D臉部密集對齊演算法PRNet 去追蹤臉部，然後將多個從不同的臉部區域提取出的CHROM訊號結合。此外，我們也重新實做了五種rPPG方法，並設計一組實驗來蒐集在不同動作（靜止、說話、踩腳踏車）、照明和心跳率下的資料。

研究結果: 我們總共招募到22位受試者並蒐集了他們的臉部影片與生理訊號。在靜態、說話、騎腳踏車測試中，不同方法得到的心跳率平均絕對誤差（對於絕對誤差小於10 bpm 的心跳率預測）與相對應的成功率分別為：CSC為1.14 bpm（成功率96.69%）、2.07 bpm（成功率68.26%）和1.31 bpm（成功率85.98%）；CHROM-PRN為1.14 bpm（成功率93.15%）、2.05 bpm（成功率81.09%）和1.37 bpm（成功率88.99%）；POS為1.13 bpm（成功率96.61%）、1.99 bpm（成功率69.78%）和1.28bpm（成功率87.28%）；CHROM則為1.08 bpm（成功率90.05%）、2.02 bpm（成功率81.30%）和1.34 bpm（成功率88.12%）。

研究結論: 我們所提出的兩個方法皆能達到與重新實做的CHROM和POS相似的表現。

Introduction: Remote photoplethysmography (rPPG) is a non-contact technique to extract the blood volume variation in skin from a video. Although many rPPG algorithms were proposed in recent years, none is accurate or robust enough for medical use.

Objectives: The primary objective of this study is to explore rPPG methods for improving the accuracy and robustness of heart rate (HR) estimation.

Methods: We propose two rPPG methods: Combination of Simple Chrominance signals (CSC) that combines three simple chrominance signals through the alpha-tuning process proposed by De Haan and Jeanne (2013). CHROM model with PRNet (CHROM-PRN) that combines multiple CHROM signals that are extracted from different facial regions tracked by the dense 3D face alignment algorithm PRNet. Besides, we also reimplemented five rPPG methods and designed an experiment for data collection under different motion (static, speaking, biking), illumination, and HR.

Results: We collected facial videos and physiological signals from 22 participants. The mean absolute error (MAE) of the successful HR estimates with an absolute error less than 10 bpm and the corresponding success rate (SR) are for the static, speaking, and biking tests: for CSC 1.14 bpm with 96.69% SR, 2.07 bpm with 68.26% SR, and 1.31 bpm with 85.98% SR; for CHROM-PRN 1.14 bpm with 93.15% SR, 2.05 bpm with 81.09% SR, and 1.37 bpm with 88.99% SR; for POS 1.13 bpm with 96.61% SR, 1.99 bpm with 69.78% SR, and 1.28 bpm with 87.28% SR; for CHROM 1.08 bpm with 90.05% SR, 2.02 bpm with 81.30% SR, and 1.34 bpm with 88.12% SR on average.

Conclusion: Both our methods perform similarly to the reimplemented CHROM and POS.

Aarts, L. A., Jeanne, V., Cleary, J. P., Lieber, C., Nelson, J. S., Oetomo, S. B., and Verkruysse, W. (2013). Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit—a pilot study. Early human development, 89(12):943–948.

Ahonen, T., Hadid, A., and Pietikainen, M. (2006). Face description with local binary patterns: Application to face recognition. IEEE transactions on pattern analysis and machine intelligence, 28(12):2037–2041.

Anliker, U., Ward, J. A., Lukowicz, P., Troster, G., Dolveck, F., Baer, M., Keita, F., Schenker, E. B., Catarsi, F., Coluccini, L., et al. (2004). Amon: a wearable multiparameter medical monitoring and alert system. IEEE Transactions on information technology in Biomedicine, 8(4):415–427.

Balakrishnan, G., Durand, F., and Guttag, J. (2013). Detecting pulse from head motions in video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3430–3437.

Baranoski, G. V. and Krishnaswamy, A. (2004). An introduction to light interaction with human skin. Revista de Informática Teórica e Aplicada, 11(1):33–62.

Barral, J.-P. and Croibier, A. (2011). 16 the facial artery. In Barral, J.-P. and Croibier, A., editors, Visceral Vascular Manipulations, pages 143 – 146. Churchill Livingstone, Oxford.

Basilio, J. A. M., Torres, G. A., Pérez, G. S., Medina, L. K. T., and Meana, H. M. P. (2011). Explicit image detection using ycbcr space color model as skin detection. Applications of Mathematics and Computer Engineering, pages 123–128.

Blackford, E. B. and Estepp, J. R. (2015). Effects of frame rate and image resolution on pulse rate measured using multiple camera imaging photoplethysmography. In Medical Imaging 2015: Biomedical Applications in Molecular, Structural, and Functional Imaging, volume 9417, page 94172D. International Society for Optics and Photonics.

Blackford, E. B., Estepp, J. R., Piasecki, A. M., Bowers, M. A., and Klosterman, S. L. (2016). Long-range non-contact imaging photoplethysmography: cardiac pulse wave sensing at a distance. In Optical Diagnostics and Sensing XVI: Toward Point-of-Care Diagnostics, volume 9715, page 971512. International Society for Optics and Photonics.

Blaus, B. (2014). Medical gallery of blausen medical 2014. Wiki J. Med, 1(10).

Bobbia, S., Macwan, R., Benezeth, Y., Mansouri, A., and Dubois, J. (2019). Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognition Letters, 124:82–90.

Böhm, M., Swedberg, K., Komajda, M., Borer, J. S., Ford, I., Dubost-Brama, A., Lerebours, G., Tavazzi, L., investigators, S., et al. (2010). Heart rate as a risk factor in chronic heart failure (shift): the association between heart rate and outcomes in a randomised placebo-controlled trial. The Lancet, 376(9744):886–894.

Bongiorno, D. L., Bryson, M., Dansereau, D. G., and Williams, S. B. (2013). Spectral characterization of cots rgb cameras using a linear variable edge filter. In Digital Photography IX, volume 8660, page 86600N. International Society for Optics and Photonics.

Bonmassar, G., Purdon, P. L., Jääskeläinen, I. P., Chiappa, K., Solo, V., Brown, E. N., and Belliveau, J. W. (2002). Motion and ballistocardiogram artifact removal for interleaved recording of eeg and eps during mri. Neuroimage, 16(4):1127–1141.

Bradski, G. (2000). The OpenCV Library. Dr. Dobb’s Journal of Software Tools.

Chaichulee, S., Villarroel, M., Jorge, J., Arteta, C., Green, G., McCormick, K., Zisserman, A., and Tarassenko, L. (2017). Multi-task convolutional neural network for patient detection and skin segmentation in continuous noncontact vital sign monitoring. In 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pages 266–272. IEEE.

Cheang, P. Y. and Smith, P. R. (2003). An overview of non-contact photoplethysmography. Electronic systems and control division research, pages 57–59.

Chen, W. and McDuff, D. (2018). Deepphys: Video-based physiological measurement using convolutional attention networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 349–365.

Chung, H. U., Kim, B. H., Lee, J. Y., Lee, J., Xie, Z., Ibler, E. M., Lee, K., Banks, A., Jeong, J. Y., Kim, J., et al. (2019). Binodal, wireless epidermal electronic systems with in-sensor analytics for neonatal intensive care. Science, 363(6430).

Correa, J. A. M., Abadi, M. K., Sebe, N., and Patras, I. (2018). Amigos: A dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affective Computing.

Da He, D., Winokur, E. S., and Sodini, C. G. (2011). A continuous, wearable, and wireless heart monitor using head ballistocardiogram (bcg) and head electrocardiogram (ecg). In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 4729–4732. IEEE.

Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), volume 1, pages 886–893. IEEE.

Davila, M. I., Lewis, G. F., and Porges, S. W. (2017). The physiocam: a novel non-contact sensor to measure heart rate variability in clinical and field applications. Frontiers in public health, 5:300.

De Haan, G. and Jeanne, V. (2013). Robust pulse rate from chrominance-based rppg. IEEE Transactions on Biomedical Engineering, 60(10):2878–2886.

De Haan, G. and Van Leest, A. (2014). Improved motion robustness of remote-ppg by using the blood volume pulse signature. Physiological measurement, 35(9):1913.

Donner, C. and Jensen, H. W. (2005). Light diffusion in multilayered translucent materials. ACM Transactions on Graphics (ToG), 24(3):1032–1039.

Donner, C., Weyrich, T., d’Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. (2008). A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Transactions on Graphics (TOG), 27(5):1–12.

Farnebäck, G. (2003). Two-frame motion estimation based on polynomial expansion. In Scandinavian conference on Image analysis, pages 363–370. Springer.

Feng, L., Po, L.M., Xu, X., and Li, Y. (2014). Motion artifacts suppression for remote imaging photoplethysmography. In 2014 19th International Conference on Digital Signal Processing, pages 18–23. IEEE.

Feng, Y., Wu, F., Shao, X., Wang, Y., and Zhou, X. (2018). Joint 3d face reconstruction and dense alignment with position map regression network. In Proceedings of the European Conference on Computer Vision (ECCV), pages 534–551.

Fernando, S., Wang, W., Kirenko, I., de Haan, G., Bambang Oetomo, S., Corporaal, H., and van Dalfsen, J. (2015). Feasibility of contactless pulse rate monitoring of neonates using google glass. In Proceedings of the 5th EAI International Conference on Wireless Mobile Communication and Healthcare, pages 198–201.

Fitzpatrick, T. B. (1988). The validity and practicality of sunreactive skin types i through vi. Archives of dermatology, 124(6):869–871.

Fouad, R., Omer, O. A., and Aly, M. H. (2019). Optimizing remote photoplethysmography using adaptive skin segmentation for real-time heart rate monitoring. IEEE Access, 7:76513–76528.

Gudi, A., Bittner, M., Lochmans, R., and van Gemert, J. (2019). Efficient real-time camera based estimation of heart rate and its variability. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 0–0.

Haque, M. A., Irani, R., Nasrollahi, K., and Moeslund, T. B. (2016). Heartbeat Rate Measurement from Facial Video. IEEE Intelligent Systems, 31(3):40–48.

He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778.

Hernandez-Ortega, J., Fierrez, J., Morales, A., and Tome, P. (2018). Time analysis of pulse-based face anti-spoofing in visible and nir. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 544–552.

Hertzman, A. B. (1937). Photoelectric plethysmography of the fingers and toes in man. Proceedings of the Society for Experimental Biology and Medicine, 37(3):529–534.

Heusch, G., Anjos, A., and Marcel, S. (2017). A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962.

Horn, B. K. and Schunck, B. G. (1981). Determining optical flow. In Techniques and Applications of Image Understanding, volume 281, pages 319–331. International Society for Optics and Photonics.

Hsu, G.S., Ambikapathi, A., and Chen, M.S. (2017). Deep learning with time-frequency representation for pulse estimation from facial videos. In 2017 IEEE International Joint Conference on Biometrics (IJCB), pages 383–389. IEEE.

Hu, J., He, Y., Liu, J., He, M., and Wang, W. (2019). Illumination robust heart-rate extraction from single-wavelength infrared camera using spatial-channel expansion. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 3896–3899. IEEE.

Huang, P.-W., Wu, B.-J., and Wu, B.-F. (2020). A heart rate monitoring framework for real-world drivers using remote photoplethysmography. IEEE journal of biomedical and health informatics.

Hülsbusch, M. and Rembold, B. (2008). Ein bildgestütztes, funktionelles verfahren zur optoelektronischen erfassung der hautperfusion. Technical report, Lehrstuhl und Institut für Hochfrequenztechnik.

Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2017). Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2462–2470.

Inan, O. T., Migeotte, P.-F., Park, K.-S., Etemadi, M., Tavakolian, K., Casanella, R., Zanetti, J., Tank, J., Funtova, I., Prisk, G. K., et al. (2014). Ballistocardiography and seismocardiography: A review of recent advances. IEEE journal of biomedical and health informatics, 19(4):1414–1427.

Irani, R., Nasrollahi, K., and Moeslund, T. B. (2014). Improved pulse detection from head motions using dct. In 2014 international conference on computer vision theory and applications (VISAPP), volume 3, pages 118–124. IEEE.

Jansen, B. H., Larson, B. H., and Shankar, K. (1991). Monitoring of the ballistocardiogram with the static charge sensitive bed. IEEE Transactions on Biomedical Engineering, 38(8): 748–751.

Karlen, W., Lim, J., Ansermino, J. M., Dumont, G., and Scheffer, C. (2012). Design challenges for camera oximetry on a mobile phone. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 2448–2451. IEEE.

Klaessens, J. H., Van Den Born, M., Van Der Veen, A., Sikkens-Van De Kraats, J., van den Dungen, F. A., and Verdaasdonk, R. M. (2014). Development of a baby friendly non-contact method for measuring vital signs: first results of clinical measurements in an open incubator at a neonatal intensive care unit. In Advanced Biomedical and Clinical Diagnostic Systems XII, volume 8935, page 89351P. International Society for Optics and Photonics.

Koivistoinen, T., Junnila, S., Varri, A., and Koobi, T. (2004). A new method for measuring the ballistocardiogram using emfi sensors in a normal chair. In The 26th annual international conference of the IEEE engineering in medicine and biology society, volume 1, pages 2026–2029. IEEE.

Kublanov, V. and Purtov, K. (2015). Researching the possibilities of remote photoplethysmography application to analysis of time-frequency changes of human heart rate variability. In 2015 International Conference on Biomedical Engineering and Computational Technologies (SIBIRCON), pages 87–92. IEEE.

Kyal, S. (2013). Constrained independent component analysis for nonobtrusive pulse rate measurements using a webcam.

Lee, D., Kim, J., Kwon, S., and Park, K. (2015). Heart rate estimation from facial photoplethysmography during dynamic illuminance changes. In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 2758–2761. IEEE.

Lee, H., Cho, A., Lee, S., and Whang, M. (2019). Vision-based measurement of heart rate from ballistocardiographic head movements using unsupervised clustering. Sensors, 19(15):3263.

Lee, S. P., Ha, G., Wright, D. E., Ma, Y., Sen-Gupta, E., Haubrich, N. R., Branche, P. C., Li, W., Huppert, G. L., Johnson, M., et al. (2018a). Highly flexible, wearable, and disposable cardiac biosensors for remote and ambulatory monitoring. NPJ digital medicine, 1(1):1–8.

Lee, Y., Park, J.-Y., Choi, Y.-W., Park, H.-K., Cho, S.-H., Cho, S. H., and Lim, Y.-H. (2018b). A novel non-contact heart rate monitor using impulse-radio ultrawideband (ir-uwb) radar technology. Scientific reports, 8(1):1–10.

Lewandowska, M., Rumiński, J., Kocejko, T., and Nowak, J. (2011). Measuring pulse rate with a webcam—a non-contact method for evaluating cardiac activity. In 2011 federated conference on computer science and information systems (FedCSIS), pages 405–410. IEEE.

Li, P., Benezeth, Y., Nakamura, K., Gomez, R., Li, C., and Yang, F. (2019). An improvement for video-based heart rate variability measurement. In 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), pages 435–439. IEEE.

Li, X., Alikhani, I., Shi, J., Seppanen, T., Junttila, J., Majamaa-Voltti, K., Tulppo, M., and Zhao, G. (2018). The obf database: a large face video database for remote physiological signal measurement and atrial fibrillation detection. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pages 242–249. IEEE.

Li, X., Chen, J., Zhao, G., and Pietikainen, M. (2014). Remote heart rate measurement from face videos under realistic situations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4264–4271.

Lienhart, R. and Maydt, J. (2002). An extended set of haar-like features for rapid object detection. In Proceedings. international conference on image processing, volume 1, pages I–I. IEEE.

Lin, B., Li, X., Yu, Z., and Zhao, G. (2019). Face liveness detection by rppg features and contextual patch-based cnn. In Proceedings of the 2019 3rd International Conference on Biometric Engineering and Applications, pages 61–68.

Lin, X. and de Haan, G. (2014). Using blood volume pulse vector to extract rppg signal in infrared spectrum.

Lin, Y.-C. and Lin, Y.H. (2017). A study of color illumination effect on the snr of rppg signals. In 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 4301–4304. IEEE.

Liu, S., Yuen, P. C., Zhang, S., and Zhao, G. (2016a). 3d mask face anti-spoofing with remote photoplethysmography. In European Conference on Computer Vision, pages 85–100. Springer.

Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A. C. (2016b). Ssd: Single shot multibox detector. In European conference on computer vision, pages 21–37. Springer.

Liu, Y., Jourabloo, A., and Liu, X. (2018). Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 389–398.

Liu, Y., Tai, Y., Li, J., Ding, S., Wang, C., Huang, F., Li, D., Qi, W., and Ji, R. (2019). Aurora guard: Real-time face anti-spoofing via light reflection. arXiv preprint arXiv:1902.10311.

Lucas, B. D., Kanade, T., et al. (1981). An iterative image registration technique with an application to stereo vision.

Luo, L., Xue, D., Feng, X., Yu, Y., and Wang, P. (2019). Realtime segmentation and facial skin tones grading. arXiv preprint arXiv:1912.12888.

Macwan, R., Benezeth, Y., and Mansouri, A. (2018a). Remote photoplethysmography with constrained ica using periodicity and chrominance constraints. Biomedical engineering online, 17(1):22.

Macwan, R., Benezeth, Y., Nakamura, K., Gomez, R., Wu, Y., and Mansouri, A. (2018b). Parameter-free adaptive stepsize multiobjective optimization applied to remote photoplethysmography. In 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), pages 267–270. IEEE.

Magdalena Nowara, E., Marks, T. K., Mansour, H., and Veeraraghavan, A. (2018). Sparseppg: Towards driver monitoring using camera-based vital signs estimation in near-infrared. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1272–1281.

Martinez, L. F. C., Paez, G., and Strojnik, M. (2011). Optimal wavelength selection for noncontact reflection photoplethysmography. In 22nd Congress of the International Commission for Optics: Light for the Development of the World, volume 8011, page 801191. International Society for Optics and Photonics.

McDuff, D. J., Blackford, E. B., Estepp, J. R., and Nishidate, I. (2018). A fast non-contact imaging photoplethysmography method using a tissuelike model. In Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diagnostics, volume 10501, page 105010Q. International Society for Optics and Photonics.

Mestha, L. K., Kyal, S., Xu, B., Lewis, L. E., and Kumar, V. (2014). Towards continuous monitoring of pulse rate in neonatal intensive care unit with a webcam. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 3817–3820. IEEE.

Moço, A. V., Stuijk, S., and de Haan, G. (2016). Skin inhomogeneity as a source of error in remote ppg-imaging. Biomedical optics express, 7(11):4718–4733.

Moço, A. V., Stuijk, S., and de Haan, G. (2018). New insights into the origin of remote ppg signals in visible light and infrared. Scientific reports, 8(1):8501.

Monitors, C. (2002). Heart rate meters, and alarms. ANSI/AAMI Standard EC13.

Morbiducci, U., Scalise, L., De Melis, M., and Grigioni, M. (2007). Optical vibrocardiography: A novel tool for the optical monitoring of cardiac activity. Annals of biomedical engineering, 35(1):45–58.

Mounsey, P. (1957). Praecordial ballistocardiography. British heart journal, 19(2):259.

Niu, X., Han, H., Shan, S., and Chen, X. (2018a). Synrhythm: Learning a deep heart rate estimator from general to specific. In 2018 24th International Conference on Pattern Recognition (ICPR), pages 3580–3585. IEEE.

Niu, X., Han, H., Shan, S., and Chen, X. (2018b). Vipl-hr: A multimodal database for pulse estimation from less-constrained face video. In Asian Conference on Computer Vision, pages 562–576. Springer.

Niu, X., Shan, S., Han, H., and Chen, X. (2019). Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation. IEEE Transactions on Image Processing.

Oh, T.-H., Jaroensri, R., Kim, C., Elgharib, M., Durand, F., Freeman, W. T., and Matusik, W. (2018). Learning-based video motion magnification. In Proceedings of the European Conference on Computer Vision (ECCV), pages 633–648.

Pilz, C. S., Blazek, V., and Leonhardt, S. (2019). On the vector space in photoplethysmography imaging. arXiv preprint arXiv:1906.04431.

Pilz, C. S., Krajewski, J., and Blazek, V. (2017). On the diffusion process for heart rate estimation from face videos under realistic conditions. In German Conference on Pattern Recognition, pages 361–373. Springer.

Pilz, C. S., Zaunseder, S., Krajewski, J., and Blazek, V. (2018). Local group invariance for heart rate estimation from face videos in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1254–1262.

Poh, M.-Z., Kim, K., Goessling, A., Swenson, N., and Picard, R. (2010a). Cardiovascular monitoring using earphones and a mobile device. IEEE Pervasive Computing, 11(4):18– 26.

Poh, M.-Z., McDuff, D. J., and Picard, R. W. (2010b). Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express, 18(10): 10762–10774.

Ponikowski, P., Voors, A. A., Anker, S. D., Bueno, H., Cleland, J. G., Coats, A. J., Falk, V., González-Juanatey, J. R., Harjola, V.-P., Jankowska, E. A., et al. (2016). 2016 esc guidelines for the diagnosis and treatment of acute and chronic heart failure: The task force for the diagnosis and treatment of acute and chronic heart failure of the european society of cardiology (esc). developed with the special contribution of the heart failure association (hfa) of the esc. European journal of heart failure, 18(8):891–975.

Proença, M., Grossenbacher, O., Dasen, S., Moser, V., Ostojic, D., Lemkaddem, A., Ferrario, D., Lemay, M., Wolf, M., Fauchére, J.-C., et al. (2018). Performance assessment of a dedicated reflectance pulse oximeter in a neonatal intensive care unit. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 1502–1505. IEEE.

Scalise, L., Bernacchia, N., Ercoli, I., and Marchionni, P. (2012). Heart rate measurement in neonatal patients using a webcamera. In 2012 IEEE International Symposium on Medical Measurements and Applications Proceedings, pages 1–4. IEEE.

Shafer, S. A. (1985). Using color to separate reflection components. Color Research & Application, 10(4):210–218.

Shan, L. and Yu, M. (2013). Video-based heart rate measurement using head motion tracking and ica. In 2013 6th International Congress on Image and Signal Processing (CISP), volume 1, pages 160–164. IEEE.

Shao, R., Lan, X., and Yuen, P. C. (2017). Deep convolutional dynamic texture learning with adaptive channel-discriminability for 3d mask face anti-spoofing. In 2017 IEEE International Joint Conference on Biometrics (IJCB), pages 748–755. IEEE.

Shimada, M., Yamada, Y., Itoh, M., and Yatagai, T. (2001). Melanin and blood concentration in human skin studied by multiple regression analysis: experiments. Physics in Medicine & Biology, 46(9):2385.

Soleymani, M., Lichtenauer, J., Pun, T., and Pantic, M. (2011). A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing, 3(1): 42–55.

Špetlík, R., Franc, V., and Matas, J. (2018). Visual heart rate estimation with convolutional neural network. In Proceedings of the British Machine Vision Conference, Newcastle, UK, pages 3–6.

Stricker, R., Müller, S., and Gross, H.M. (2014). Non-contact video-based pulse rate measurement on a mobile service robot. In The 23rd IEEE International Symposium on Robot and Human Interactive Communication, pages 1056–1062. IEEE.

Sun, Y., Azorin-Peris, V., Kalawsky, R., Hu, S., Papin, C., and Greenwald, S. E. (2012). Use of ambient light in remote photoplethysmographic systems: comparison between a high-performance camera and a low-cost webcam. Journal of biomedical optics, 17(3): 037005.

Sun, Y. and Thakor, N. (2016). Photoplethysmography revisited: from contact to noncontact, from point to imaging. IEEE Transactions on Biomedical Engineering, 63(3):463–477.

Takano, C. and Ohta, Y. (2007). Heart rate measurement based on a time-lapse image. Medical engineering & physics, 29(8):853–857.

Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., and Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. arXiv preprint arXiv:2001.00179. Tran, L. and Liu, X. (2018). Nonlinear 3d face morphable model. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7346–7355.

Tran, Q.-V., Su, S.-F., Sun, W., and Tran, M.-Q. (2019). Adaptive pulsatile plane for robust noncontact heart rate monitoring. IEEE Transactions on Systems, Man, and Cybernetics: Systems.

Tsou, Y.-Y., Lee, Y.-A., and Hsu, C.-T. (2020a). Multi-task learning for simultaneous video generation and remote photoplethysmography estimation. In Proceedings of the Asian Conference on Computer Vision.

Tsou, Y.-Y., Lee, Y.-A., Hsu, C.-T., and Chang, S.-H. (2020b). Siamese-rppg network: remote photoplethysmography signal estimation from face videos. In Proceedings of the 35th Annual ACM Symposium on Applied Computing, pages 2066–2073.

Tu, X., Zhao, J., Xie, M., Jiang, Z., Balamurugan, A., Luo, Y., Zhao, Y., He, L., Ma, Z., and Feng, J. (2020). 3d face reconstruction from a single image assisted by 2d face images in the wild. IEEE Transactions on Multimedia.

Tulyakov, S., AlamedaPineda, X., Ricci, E., Yin, L., Cohn, J. F., and Sebe, N. (2016). Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2396–2404.

van Gastel, M., Stuijk, S., and de Haan, G. (2015). Motion robust remote-ppg in infrared. IEEE Transactions on Biomedical Engineering, 62(5):1425–1433.

van Gastel, M., Stuijk, S., and de Haan, G. (2016). Robust respiration detection from remote photoplethysmography. Biomedical optics express, 7(12):4941–4957.

Van Gemert, M., Jacques, S. L., Sterenborg, H., and Star, W. (1989). Skin optics. IEEE Transactions on biomedical engineering, 36(12):1146–1154.

Verkruysse, W., Svaasand, L. O., and Nelson, J. S. (2008). Remote plethysmographic imaging using ambient light. Optics express, 16(26):21434–21445.

Villarroel, M., Chaichulee, S., Jorge, J., Davis, S., Green, G., Arteta, C., Zisserman, A., McCormick, K., Watkinson, P., and Tarassenko, L. (2019). Non-contact physiological monitoring of preterm infants in the neonatal intensive care unit. npj Digital Medicine, 2(1):1–18.

Villarroel, M., Guazzi, A., Jorge, J., Davis, S., Watkinson, P., Green, G., Shenvi, A., McCormick, K., and Tarassenko, L. (2014). Continuous noncontact vital sign monitoring in neonatal intensive care unit. Healthcare technology letters, 1(3):87–91.

Vogels, T., van Gastel, M., Wang, W., and de Haan, G. (2018). Fully-automatic camera-based pulse-oximetry during sleep. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1349–1357.

Wadhwa, N., Rubinstein, M., Durand, F., and Freeman, W. T. (2013). Phase-based video motion processing. ACM Transactions on Graphics (TOG), 32(4):80.

Wadhwa, N., Rubinstein, M., Durand, F., and Freeman, W. T. (2014). Riesz pyramids for fast phase-based video magnification. In 2014 IEEE International Conference on Computational Photography (ICCP), pages 1–10. IEEE.

Wang, W., Den Brinker, A. C., and De Haan, G. (2018a). Single element remote-ppg. IEEE Transactions on Biomedical Engineering.

Wang, W., Den Brinker, A. C., and De Haan, G. (2019). Discriminative signatures for remote-ppg. IEEE Transactions on Biomedical Engineering.

Wang, W., den Brinker, A. C., Stuijk, S., and de Haan, G. (2016). Algorithmic principles of remote ppg. IEEE Transactions on Biomedical Engineering, 64(7):1479–1491.

Wang, W. and Shan, C. (2020). Impact of makeup on remote-ppg monitoring. Biomedical Physics & Engineering Express, 6(3):035004.

Wang, W., Stuijk, S., and De Haan, G. (2014). Exploiting spatial redundancy of image sensor for motion robust rppg. IEEE transactions on Biomedical Engineering, 62(2):415–425.

Wang, W., Stuijk, S., and De Haan, G. (2015). A novel algorithm for remote photoplethysmography: Spatial subspace rotation. IEEE transactions on biomedical engineering, 63(9):1974–1984.

Wang, Z., Yang, X., and Cheng, K.T. (2018b). Accurate face alignment and adaptive patch selection for heart rate estimation from videos under realistic scenarios. PloS one, 13(5):e0197275.

Wiens, A. D., Etemadi, M., Roy, S., Klein, L., and Inan, O. T. (2014). Toward continuous, noninvasive assessment of ventricular function and hemodynamics: Wearable ballistocardiography. IEEE journal of biomedical and health informatics, 19(4):1435–1442.

Wieringa, F. P., Mastik, F., and van der Steen, A. F. (2005). Contactless multiple wavelength photoplethysmographic imaging: a first step toward “spo 2 camera＂technology. Annals of biomedical engineering, 33(8):1034–1041.

Wu, B.-F., Chu, Y.-W., Huang, P.-W., and Chung, M.-L. (2019a). Neural network based luminance variation resistant remotephotoplethysmography for driver＇s heart rate monitoring. IEEE Access, 7:57210–57225.

Wu, B.-F., Huang, P.-W., He, D.-H., Lin, C.-H., and Chen, K.-H. (2019b). Remote photoplethysmography enhancement with machine leaning methods. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pages 2466–2471. IEEE.

Wu, B.-F., Huang, P.-W., Lin, C.-H., Chung, M.-L., Tsou, T.-Y., and Wu, Y.-L. (2018). Motion resistant image-photoplethysmography based on spectral peak tracking algorithm. IEEE Access, 6:21621–21634.

Wu, B.-F., Yang, Y.-Y., Tsai, B.-R., Huang, P.-W., Tsai, Y.-C., and Chen, K.-H. (2019c). Remote heartrate measurement based on signal feature detection in time domain. In 2019 International Conference on System Science and Engineering (ICSSE), pages 88–93. IEEE.

Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. (2012). Eulerian video magnification for revealing subtle changes in the world.

Yu, Z., Li, X., and Zhao, G. (2019a). Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. In Proc. BMVC.

Yu, Z., Peng, W., Li, X., Hong, X., and Zhao, G. (2019b). Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement. In Proceedings of the IEEE International Conference on Computer Vision, pages 151–160.

Zhang, F., Fan, X., Ai, G., Song, J., Qin, Y., and Wu, J. (2019). Accurate face detection for high performance. arXiv preprint arXiv:1905.01585.

Zhang, G., Shan, C., Kirenko, I., Long, X., and Aarts, R. M. (2017a). Hybrid optical unobtrusive blood pressure measurements. Sensors, 17(7):1541.

Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., and Li, S. Z. (2017b). Faceboxes: A cpu real-time face detector with high accuracy. In 2017 IEEE International Joint Conference on Biometrics (IJCB), pages 1–9. IEEE.

Zhang, Y., Pintea, S. L., and Van Gemert, J. C. (2017c). Video acceleration magnification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 529–537.

Zhao, C., Lin, C.-L., Chen, W., and Li, Z. (2018). A novel framework for remote photoplethysmography pulse extraction on compressed videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1299–1308.

Zheng, J., Hu, S., Chouliaras, V., and Summers, R. (2008). Feasibility of imaging photoplethysmography. In 2008 International Conference on BioMedical Engineering and Informatics, volume 2, pages 72–75. IEEE.

Zhou, K., Krause, S., Blocher, T., and Stork, W. (2020). Enhancing remote-ppg pulse extraction in disturbance scenarios utilizing spectral characteristics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 280–281.