近年來結構光三維成像技術已經被相當廣泛地應用在各種層面，其中基於格雷碼編碼結構光投影技術因其擁有著良好的強健性以及抗噪性被廣泛地應用於工業檢測上，能夠量測待測物體的高度分布用以進一步的深入研究運算。但格雷碼編碼技術是直接投影特定強度的編碼圖案到待測物件上，如果待測物件本身是具有強烈鏡面反射性質的高反光金屬物件的話，容易使得獲取的編碼影像有局部區域資訊喪失的結果，導致量測的三維成像有著嚴重的精度落差。為了進一步地改善高反光物件的結構光三維成像成果，本研究提出了一種嶄新的編碼影像修復技術，此技術開發基於深度學習中的生成對抗網路框架，利用神經網路去自動偵測出編碼影像中資訊喪失的區域同時進行編碼條紋的修補。
關鍵字：深度學習、高反光物件量測、結構光、影像處理

In recent years, three-dimensional 3D scanning technology has been widely used in various levels. Among them, Based on the Gray code pattern structured light projection technology has been widely used in industrial inspection. Gray code encoding technology directly projects a sequence of fringe pattern with specific intensity onto the scanned object and is used to measure height distribution of the scanned object for further in-depth research operations. However, If the scanned object itself is a highly reflective metal object with strong specular reflection surface properties, tend to cause the acquired encoded fringe image to have local area encoding information lost. As a result, the measured point clouds has a serious accuracy gap. In order to improve quality of reconstruction results of highly reflective objects, this study proposes a new encoded fringe image inpainting technology. This technology develops a fringe-inpainting system based on generative adversarial network framework.
Key words: Deep Learning, Measurement of Highly Reflective Objects,
Structured Light, Image Processing

[1] F. Blais, "Review of 20 years of range sensor development," Journal of electronic imaging, vol. 13, no. 1, pp. 231-243, 2004.
[2] A. Borji, "Pros and cons of gan evaluation measures," Computer Vision and Image Understanding, vol. 179, pp. 41-65, 2019.
[3] J.-Y. Bouguet, Visual methods for three-dimensional modeling. California Institute of Technology, 1999.
[4] S. Feng et al., "Fringe pattern analysis using deep learning," Advanced Photonics, vol. 1, no. 2, p. 025001, 2019.
[5] I. Goodfellow et al., "Generative adversarial nets," Advances in neural information processing systems, vol. 27, 2014.
[6] F. Gray, "Pulse code communication," United States Patent Number 2632058, 1953.
[7] C. Guo et al., "Zero-reference deep curve estimation for low-light image enhancement," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1780-1789.
[8] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[9] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125-1134.
[10] H. Jiang, H. Zhao, and X. Li, "High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces," Optics and Lasers in Engineering, vol. 50, no. 10, pp. 1484-1493, 2012.
[11] J. Johnson, A. Alahi, and L. Fei-Fei, "Perceptual losses for real-time style transfer and super-resolution," in European conference on computer vision, 2016: Springer, pp. 694-711.
[12] Q. Kemao, "Windowed Fourier transform for fringe pattern analysis," Applied Optics, vol. 43, no. 13, pp. 2695-2702, 2004.
[13] F. Li, Q. Li, T. Zhang, Y. Niu, and G. Shi, "Depth acquisition with the combination of structured light and deep learning stereo matching," Signal Processing: Image Communication, vol. 75, pp. 111-117, 2019.
[14] X. Liu, W. Chen, H. Madhusudanan, J. Ge, C. Ru, and Y. Sun, "Optical measurement of highly reflective surfaces from a single exposure," IEEE Transactions on Industrial Informatics, vol. 17, no. 3, pp. 1882-1891, 2020.
[15] S. Mehta, E. Mercan, J. Bartlett, D. Weaver, J. G. Elmore, and L. Shapiro, "Y-Net: joint segmentation and classification for diagnosis of breast biopsy images," in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018: Springer, pp. 893-901.
[16] S. Minaee, Y. Y. Boykov, F. Porikli, A. J. Plaza, N. Kehtarnavaz, and D. Terzopoulos, "Image segmentation using deep learning: A survey," IEEE transactions on pattern analysis and machine intelligence, 2021.
[17] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, "Spectral normalization for generative adversarial networks," arXiv preprint arXiv:1802.05957, 2018.
[18] K. Nazeri, E. Ng, T. Joseph, F. Z. Qureshi, and M. Ebrahimi, "Edgeconnect: Generative image inpainting with adversarial edge learning," arXiv preprint arXiv:1901.00212, 2019.
[19] J. Pages, J. Salvi, C. Collewet, and J. Forest, "Optimised De Bruijn patterns for one-shot shape acquisition," Image and Vision Computing, vol. 23, no. 8, pp. 707-720, 2005.
[20] Z. Qin, Q. Zeng, Y. Zong, and F. Xu, "Image inpainting based on deep learning: A review," Displays, vol. 69, p. 102028, 2021.
[21] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015: Springer, pp. 234-241.
[22] D. Scharstein and R. Szeliski, "High-accuracy stereo depth maps using structured light," in 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., 2003, vol. 1: IEEE, pp. I-I.
[23] M. Sjödahl and P. Synnergren, "Measurement of shape by using projected random patterns and temporal digital speckle photography," Applied optics, vol. 38, no. 10, pp. 1990-1997, 1999.
[24] Z. Song, R. Chung, and X.-T. Zhang, "An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-D," IEEE Transactions on Industrial Electronics, vol. 60, no. 3, pp. 1023-1032, 2012.
[25] Z. Song, H. Jiang, H. Lin, and S. Tang, "A high dynamic range structured light means for the 3D measurement of specular surface," Optics and Lasers in Engineering, vol. 95, pp. 8-16, 2017.
[26] X. Su and W. Chen, "Fourier transform profilometry:: a review," Optics and lasers in Engineering, vol. 35, no. 5, pp. 263-284, 2001.
[27] C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo, and C.-W. Lin, "Deep learning on image denoising: An overview," Neural Networks, vol. 131, pp. 251-275, 2020.
[28] D. Ulyanov, A. Vedaldi, and V. Lempitsky, "Instance normalization: The missing ingredient for fast stylization," arXiv preprint arXiv:1607.08022, 2016.
[29] F. Wang and Z. Zhao, "A survey of iterative closest point algorithm," in 2017 Chinese Automation Congress (CAC), 2017: IEEE, pp. 4395-4399.
[30] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, "High-resolution image synthesis and semantic manipulation with conditional gans," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8798-8807.
[31] K. Wu, J. Tan, H. L. Xia, and C. B. Liu, "An exposure fusion-based structured light approach for the 3D measurement of a specular surface," IEEE Sensors Journal, vol. 21, no. 5, pp. 6314-6324, 2020.
[32] W. Xiong et al., "Foreground-aware image inpainting," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5840-5848.
[33] Y. Xu and D. G. Aliaga, "An adaptive correspondence algorithm for modeling scenes with strong interreflections," IEEE Transactions on Visualization and Computer Graphics, vol. 15, no. 3, pp. 465-480, 2009.
[34] Y. Yang and X. Guo, "Generative landmark guided face inpainting," in Chinese Conference on Pattern Recognition and Computer Vision (PRCV), 2020: Springer, pp. 14-26.
[35] Y. Yang, X. Guo, J. Ma, L. Ma, and H. Ling, "Lafin: Generative landmark guided face inpainting," arXiv preprint arXiv:1911.11394, 2019.
[36] H. Yu, D. Zheng, J. Fu, Y. Zhang, C. Zuo, and J. Han, "Deep learning-based fringe modulation-enhancing method for accurate fringe projection profilometry," Optics Express, vol. 28, no. 15, pp. 21692-21703, 2020.
[37] C. Zhang, J. Xu, N. Xi, J. Zhao, and Q. Shi, "A robust surface coding method for optically challenging objects using structured light," IEEE Transactions on Automation Science and Engineering, vol. 11, no. 3, pp. 775-788, 2014.
[38] L. Zhang, Q. Chen, C. Zuo, and S. Feng, "High-speed high dynamic range 3D shape measurement based on deep learning," Optics and Lasers in Engineering, vol. 134, p. 106245, 2020.
[39] Q. Zhang, X. Su, L. Xiang, and X. Sun, "3-D shape measurement based on complementary Gray-code light," Optics and Lasers in Engineering, vol. 50, no. 4, pp. 574-579, 2012.
[40] 冯世杰, 左超, 尹维, and 陈钱, "深度学习技术在条纹投影三维成像中的应用," 红外与激光工程, vol. 49, no. 3, pp. 0303018-0303018-17, 2020.
[41] 左超, 冯世杰, 张翔宇, 韩静, and 陈钱, "深度学习下的计算成像: 现状, 挑战与未来 Invited," 光学学报, vol. 40, no. 1, p. 0111003, 2020.
[42] 张启灿 and 吴周杰, "基于格雷码图案投影的结构光三维成像技术," 红外与激光工程, vol. 49, no. 3, pp. 0303004-0303004-13, 2020.