隨著3D 技術的快速發展，越來越多應用出現來滿足人類的需求，而為了追求更好的視覺體驗，深度圖的品質扮演著非常重要的角色。由於經由深度相機拍出來的深度圖在物體邊緣會有錯誤的深度值，需要一個深度增強系統來修正上述的問題，而近幾年深度學習在影像處理方面的成果非常卓越，許多學者紛紛將深度學習運用至深度增強系統中。因此，在本論文中，我們提出一套深度增強系統能有效修正物體邊緣錯誤的深度值，運用圖像引導概念去增強錯誤的深度圖和殘差密集網路(RDN)保有輸入深度圖的架構。並提出兩個特殊的損失函數來使網路更加專注於物體邊緣附近和錯誤的點，且為了使網路可以盡早去參考彩圖的資訊，在深度分支前半部結合彩圖分支的特徵。從本邊論文的實驗結果顯示比其他先進的網路能有更好的修正效果，並從消融研究去證明所提出的損失函數和添加彩圖資訊的效果。

With the rapid development of 3D technology, more and more applications are emerging to meet human needs. In order to pursue a better visual experience, the quality of the depth map plays a very important role. Since the depth map taken by the depth camera will have the wrong depth value at the edge of the object, a depth enhancement system is needed to correct the above problem. In recent years, deep learning has achieved outstanding results in image processing, and many scholars have applied deep learning to deep enhancement systems. Therefore, in this paper, we propose a set of depth enhancement system that can effectively correct the wrong depth value of the edge of the object. The concept of image guidance is used to enhance the error depth map and the residual dense network (RDN)
maintains the structure of the input depth map. Two special loss functions are proposed to make the network focus more on the points near the edge of the object and the wrong point. In order to allow the network to refer to the information of the color map as early as possible, the features of the color map branches are combined in the first half of the depth branch. The experimental results of this paper show that it has better correction effects than other advanced networks. The proposed loss function and the effect of adding color information are demonstrated from the ablation study.

[1] S. Kumari, R. R. Jha, A. Bhavsar and A. Nigam, "AUTODEPTH: Single Image Depth Map Estimation via Residual CNN Encoder-Decoder and Stacked Hourglass," Proc. of 2019 IEEE International Conference on Image Processing (ICIP), 2019, pp. 340-344, doi: 10.1109/ICIP.2019.8803006.
[2] Dong C., Loy C.C., He K., Tang X. (2014) Learning a Deep Convolutional Network for Image Super-Resolution. In: Fleet D., Pajdla T., Schiele B., Tuytelaars T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8692. Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_13
[3] C. Dong, C. C. Loy, K. He and X. Tang, "Image Super-Resolution Using Deep Convolutional Networks," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 1 Feb. 2016, doi: 10.1109/TPAMI.2015.2439281.
[4] Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual dense network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2472-2481).
[5] Y. -. Zhou, R. Chellappa, A. Vaid and B. K. Jenkins, "Image restoration using a neural network," in IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, no. 7, pp. 1141-1151, July 1988, doi: 10.1109/29.1641.
[6] K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising," in IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, July 2017, doi: 10.1109/TIP.2017.2662206.
[7] H. C. Burger, C. J. Schuler and S. Harmeling, "Image denoising: Can plain neural networks compete with BM3D?," Proc. of 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2392-2399, doi: 10.1109/CVPR.2012.6247952.
[8] K. R. Vijayanagar, M. Loghman and Joohee Kim, "Refinement of depth maps generated by low-cost depth sensors," Proc. of 2012 International SoC Design Conference (ISOCC), 2012, pp. 355-358, doi: 10.1109/ISOCC.2012.6407114.
[9] O. P. Gangwal and B. Djapic, "Real-time implementation of depth map post-processing for 3D-TV in dedicated hardware," 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE), 2010, pp. 173-174, doi: 10.1109/ICCE.2010.5418730.
[10] Johannes Kopf, Michael F. Cohen, Dani Lischinski, and Matt Uyttendaele. 2007. Joint bilateral upsampling. ACM Trans. Graph. 26, 3 (July 2007), 96–es. DOI:https://doi.org/10.1145/1276377.1276497
[11] C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," Proc. of Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271), 1998, pp. 839-846, doi: 10.1109/ICCV.1998.710815.
[12] X. Zhang and R. Wu, "Fast depth image denoising and enhancement using a deep convolutional network," Proc. of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 2499-2503, doi: 10.1109/ICASSP.2016.7472127.
[13] Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual dense network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2472-2481). [14] C.-W. Hsu, C.-C. Chang, and C.-J. Lin, "A practical guide to support vector classification," 2003.
[14] J. Zhu, J. Zhang, Y. Cao and Z. Wang, "Image guided depth enhancement via deep fusion and local linear regularizaron," Proc. of 2017 IEEE International Conference on Image Processing (ICIP), 2017, pp. 4068-4072, doi: 10.1109/ICIP.2017.8297047.
[15] Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2980-2988).
[16] K. He, J. Sun and X. Tang, "Guided Image Filtering," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1397-1409, June 2013, doi: 10.1109/TPAMI.2012.213.
[17] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," Proc. of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90.
[18] A. Krizhevsky, I. Sutskever, & G. E. Hinton. (2012). "Imagenet classification with deep convolutional neural networks." In Advances in Neural Information Processing Systems (pp. 1097-1105).
[19] Lim, B., Son, S., Kim, H., Nah, S., & Mu Lee, K. (2017). Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 136-144).
[20] G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, "Densely Connected Convolutional Networks," Proc. of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 2261-2269, doi: 10.1109/CVPR.2017.243.
[21] Tong, T., Li, G., Liu, X., & Gao, Q. (2017). Image super-resolution using dense skip connections. In Proceedings of the IEEE international conference on computer vision (pp. 4799-4807).
[22] S. Ren, , K. He, R.Girshick, & J. Sun (2015). "Faster r-cnn: towards real-time object detection with region proposal networks." In Advances in Neural Information Processing Systems (pp. 91-99).
[23] Dai, J., Li, Y., He, K., & Sun, J. (2016). R-fcn: Object detection via region-based fully convolutional networks. arXiv preprint arXiv:1605.06409.
[24] J. Redmon, S. Divvala, R. Girshick, & A. Farhadi (2016). "You only look once: unified, real-time object detection." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 779-788).
[25] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, & A. C. Berg (2016, October). "Ssd: single shot multibox detector." In Proc. of European Conference on Computer Vision (pp. 21-37). Springer, Cham.
[26] N. Mayer et al., "A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation," Proc. of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 4040-4048, doi: 10.1109/CVPR.2016.438.
[27] W. Zhou, X. Li and D. Reynolds, "Guided deep network for depth map super-resolution: How much can color help?," Proc. of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 1457-1461, doi: 10.1109/ICASSP.2017.7952398.