近年來，隨著人工智慧的再度興起，深度學習之技術蓬勃發展，相關的應用研究也越來越熱門。許多有趣的電影工業的應用，人臉置換已經成為電影後期製作的重要工具。傳統的人臉置換是透過影像編輯軟體以人工方式合成，這個方法的合成效果不但會不自然且花費的時間也較久。本篇論文提出基於自動編碼器的人臉置換類神經網路系統，透過深度學習的技術，學習兩張人臉之間的映射關係，經過不斷地訓練，神經網路能夠使置換後臉部能夠更加自然，置換速度也更快。本人臉置換系統首先利用MTCNN與Dlib函式庫的face recognition先對影片作人臉偵測並辨識，收集並建立角色A與B的臉部資料集，透過編碼器分辨人臉A與B並提取面部資訊，再透過人臉A及B解碼器分別還原其人臉，訓練完的神經網路能讓角色A與B的臉部互相置換，最後再將置換後的臉透過圖像融合貼回原影片，便完成了人臉置換。經實驗結果證明我們提出的網路架構系統可有效完成人臉置換效果。

Recently, the artificial intelligent (AI) technology has become prosperous by deep learning such the AI related researches have become more and more popular in many applications. The AI-based face swapping system has been applied to movie industry as an effective post production tool. Traditional face swapping approach is manually synthesized through image editing software such that the synthetic results are not only unnatural but also time consuming. This paper proposes a face swapping system mainly based on autoencoders. Through the technique of deep neural networks, the mapping relationship between two faces can be learned. After training, the neural networks can make the swapped faces more natural and achieve in very faster speed. In the proposed face swapping system, MTCNN and face recognition of Dlib library are first used to detect and collect the faces of character A in a video sequence and the faces of the target character B from other video sources. With the collected facial data sets of characters A and B, we can employ the shared-encoder to extract their facial information and restore their faces by decoders A and B, respectively. The trained neural networks can swap the faces of the character A to those of character B if we use shared-encoder with decoder B. Finally, by image fusing process, the swapped faces are pasted to the original video to finish the face swap procedure. The experimental results are exhibited to show the effectiveness of the proposed face swapping neural network system.

[1] P. Baldi, "Autoencoders, unsupervised learning, and deep architectures," in Proceedings of ICML workshop on unsupervised and transfer learning, 2012, pp. 37-49.
[2] V. Blanz and T. Vetter, "A morphable model for the synthesis of 3D faces," in Siggraph, 1999, vol. 99, no. 1999, pp. 187-194.
[3] D. Bitouk, N. Kumar, S. Dhillon, P. Belhumeur, and S. K. Nayar, "Face swapping: automatically replacing faces in photographs," in ACM Transactions on Graphics (TOG), 2008, vol. 27, no. 3: ACM, p. 39.
[4] K. Dale, K. Sunkavalli, M. K. Johnson, D. Vlasic, W. Matusik, and H. Pfister, "Video face replacement," in ACM Transactions on Graphics (TOG), 2011, vol. 30, no. 6: ACM, p. 130.
[5] S. Suwajanakorn, S. M. Seitz, and I. Kemelmacher-Shlizerman, "What makes tom hanks look like tom hanks," in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 3952-3960.
[6] Adobe. (2019). Adobe Photoshop [Online]. Available: https://www.adobe.com/tw/products/photoshop.html.
[7] Y. Nirkin, I. Masi, A. T. Tuan, T. Hassner, and G. Medioni, "On face segmentation, face swapping, and face perception," in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018: IEEE, pp. 98-105.
[8] I. Korshunova, W. Shi, J. Dambre, and L. Theis, "Fast face-swap using convolutional neural networks," in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3677-3685.
[9] I. Goodfellow et al., "Generative adversarial nets," in Advances in neural information processing systems, 2014, pp. 2672-2680.
[10] L. A. Gatys, A. S. Ecker, and M. Bethge, "Image style transfer using convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2414-2423.
[11] T. Wei. (2017). CycleGAN Face-off 直播換臉 [Online]. Available: https://www.youtube.com/watch?v=Fea4kZq0oFQ.
[12] P. Pérez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Transactions on graphics (TOG), vol. 22, no. 3, pp. 313-318, 2003.
[13] OpenCV. (2019). Open Source Computer Vision Library [Online]. Available: https://github.com/opencv/opencv.
[14] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[15] N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in international Conference on computer vision & Pattern Recognition (CVPR'05), 2005, vol. 1: IEEE Computer Society, pp. 886--893.
[16] A. M. A. K. Revision. (2013). Introduction to SIFT (Scale Invariant Feature Transform) [Online]. Available: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_sift_intro/py_sift_intro.html.
[17] T. Ahonen, A. Hadid, and M. Pietikainen, "Face description with local binary patterns: Application to face recognition," IEEE Transactions on Pattern Analysis & Machine Intelligence, no. 12, pp. 2037-2041, 2006.
[18] L. Bottou, "Large-scale machine learning with stochastic gradient descent," in Proceedings of COMPSTAT'2010: Springer, 2010, pp. 177-186.
[19] T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop, coursera: Neural networks for machine learning," University of Toronto, Technical Report, 2012.
[20] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
[21] reddit/deepfakes. (2017). deepfakes_faceswap [Online]. Available: https://github.com/joshua-wu/deepfakes_faceswap.
[22] W. Shi et al., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1874-1883.
[23] P. I. Wilson and J. Fernandez, "Facial feature detection using Haar classifiers," Journal of Computing Sciences in Colleges, vol. 21, no. 4, pp. 127-133, 2006.
[24] Y. Wu and X. Ai, "Face detection in color images using AdaBoost algorithm based on skin color information," in First International Workshop on Knowledge Discovery and Data Mining (WKDD 2008), 2008: IEEE, pp. 339-342.
[25] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint face detection and alignment using multitask cascaded convolutional networks," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
[26] D. King. (2019). The world's simplest facial recognition api for Python and the command line [Online]. Available: https://github.com/ageitgey/face_recognition.
[27] S. Mallick. (2015). Seamless Cloning using OpenCV ( Python , C++ ) [Online]. Available: https://www.learnopencv.com/seamless-cloning-using-opencv-python-cpp/.