本論文提出一個基於紋理相似度之深度圖上採樣演算法，其含基於紋理相似度之插值法及邊緣引導濾波器。紋理相似度插值法採用贏者全拿的策略搭配紋理的資訊來強化邊緣處理的效果。邊緣引導濾波器利用從索貝爾邊緣偵測法得到的深度邊緣圖，並沿著邊緣進行濾波處理，以消除邊緣的細微鋸齒與失真，深度圖品質得以大量提升。
根據實驗結果顯示，使用本論文提出之深度圖上採樣演算法之峰值訊號雜訊比以及結構相似度皆獲提升，且能更近似於真實深度圖。此外，本論文亦提出此演算法之硬體架構，並以Altera之FPGA加以實現，此影像置中深度解封裝硬體架構共使用4,697個邏輯元件，且運作速度可達到240.38 MHz。

In this thesis, a texture similarity based depth up-sampling algorithm is proposed, where it majorly consists of a texture similarity based interpolation and an edge guided filter. The texture similarity based interpolation uses the winner-takes-all strategy, and adopts the color information of the texture to further improve the interpolated effect along the edges. The edge guided filter adopts the depth edge map obtained from Sobel edge detector to perform filtering operation along the edge boundaries. After filtering, the depth quality along the edges is greatly enhanced because the tiny saw teeth and other aliasing can be eliminated. Moreover, the hardware architecture of the proposed texture similarity based interpolation algorithm is designed and realized in Altera FPGA.
The experimental results show that the proposed method can achieve higher PSNR and SSIM qualities and have a better approximation to the ground truth. Besides, the design of CTDP architecture can achieve 240.38 MHz with 4,697 logic elements on Altera FPGA.

[1] D. Zhou, X. Shen, and W. Dong, “Image zooming using directional cubic convolution interpolation,” IET Image Process., vol. 6, pp. 627–634, Aug. 2012.
[2] X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. Image Process., vol. 10, pp. 1521–1527, Oct. 2001.
[3] Y. Luo, S. Liu, and H. Zhu, “Edge-directed interpolation based on canny detector,” in Proc. IEEE Int. Conf. Mechatron. Autom.(ICMA), Beijing, China, Aug. 2011, pp. 698–702.
[4] S. Fifman, “Digital rectification of ERTS multispectral imagery,” in Proc. Symp. Significant Results Obtained from ERTS-1, Nov. 1995, pp. 1131–1142.
[5] R. C. Gonzalez and R. E. Woods, Digital Image Processing. Reading, MA: Addison-Wesley, 1992.
[6] H. S. Hou and H. Andrews, “Cubic splines for image interpolation and digital filtering,” IEEE Trans. Acoust., Speech, Signal Process., vol. 26, pp. 508–517, Dec. 1978.
[7] P. Y. Chen, C. Y. Lien, and C. P. Lu, “VLSI implementation of an edge-oriented image scaling processor,” IEEE Trans. Very Large Scale Integr. Syst., vol. 17, pp.1275–1284, Sep. 2009.
[8] Y. S. Wu, “A super resolution algorithm based on iterative edge-directional predictions,” M.S. thesis, National Cheng Kung University, Taiwan, July 2014.
[9] J. Allebach and P. W. Wong, “Edge-directed interpolation,” in Proc IEEE Int Conf. Image Process. (ICIP), Lausanne, Switzerland, Sep. 1996, pp. 707–710.
[10] C. S. Wong and W. C. Siu, “Adaptive directional window selection for edge-directed interpolation,” in Proc IEEE Int Conf. Comput. Commun. and Netw, commun. (ICCCN), Zurich, Switzerland, Aug. 2010, pp. 1–6.
[11] H. Deng, L. Yu, J. Qiu, and J. Zhang, “A joint texture/depth edge-directed up-sampling algorithm for depth map coding,” in Proc. IEEE Int. Conf. Multimedia Expo. (ICME), Melbourne, Australia, Jul. 2012, pp. 646–650.
[12] M. O. Wildeboer, T. Yendo, M. Panahpour Tehrani, T. Fujii, and M. Tanimoto, “Color based depth up-sampling for depth compression,” in Proc. Picture Coding Symp. (PCS), Nagoya, Japan, Dec. 2010, pp. 170–173.
[13] S. Liu, P. Lai, D. Tian, and C. W. Chen, “New depth coding techniques with utilization of corresponding video,” IEEE Trans. Broadcast., vol. 57, pp. 551–561, June 2011.
[14] J. Park, H. Kim, Y. W. Tai, M. S. Brown, and I. S. Kweon, “High quality depth map upsampling and completion for RGB-D cameras,” IEEE Trans. Image Process., vol. 23, pp. 5559–5572, Dec. 2014.
[15] J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele, “Joint bilateral upsampling,” ACM Trans. Grap., vol. 26, p. 96, 2007.
[16] K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, pp. 1397–1409, June 2013.
[17] D. Ferstl, C. Reinbacher, R. Ranftl, M. Ruther, and H. Bischof, “Image guided depth upsampling using anisotropic total generalized variation,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sydney, Australia, Dec. 2013, pp. 993–1000.
[18] J. Li, G. Zeng, R. Gan, H. Zha, and L. Wang, “A Bayesian approach to uncertainty-based depth map super resolution,” in Proc. Asian Conf. Comput Vis. (ACCV), Daejeon, South Korea, Nov. 2012, pp. 205–216.
[19] O. Mac Aodha, N. D. Campbell, A. Nair, and G. J. Brostow, “Patch based synthesis for single depth image super-resolution,” in Proc. Eur. Conf. Comput. Vis. (ECCV), Firenze, Italy, Oct. 2012, pp. 71–84.
[20] J. Park, H. Kim, Y. W. Tai, M. S. Brown, and I. Kweon, “High quality depth map upsampling for 3D-TOF cameras,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Barcelona, Spain, Nov. 2011, pp. 1623–1630.
[21] J. Xie, R. S. Feris, S. S. Yu, and M. T. Sun, “Joint super resolution and denoising from a single depth image,” IEEE Trans. Multimedia, vol. 17, pp. 1525–1537, Sep. 2015.
[22] Q. Yang, R. Yang, J. Davis, and D. Nister, “Spatial-depth super resolution for range images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), Minneapolis, USA, June 2007, pp. 1–8.
[23] X. Zhang and X. Wu, “Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation,” IEEE Trans. Image Process., vol. 17, pp. 887–896, June 2008.
[24] C. C. Lin, M. H. Sheu, H. K. Chiang, W. K. Tsai, and Z. C. Wu, “Real-time FPGA architecture of extended linear convolution for digital imagescaling”, in Proc. IEEE Int. Conf. Field-Programmable Technol. (ICECE), Taipei, Taiwan, Dec. 2008, pp. 381–384.
[25] S. L. Chen, “VLSI implementation of a low cost high quality image scaling processer”, IEEE Trans. Circuits Syst.II, Exp. Briefs, vol. 60, pp. 31–35, Jan. 2013.
[26] C. H. Kim, S. M. Seong, J. A. Lee, and L. S. Kim, “Winscale: animage-scaling algorithm using an area pixel model”, IEEE Trans. Circuits Syst. Video Technol., vol. 13, pp. 549–553, June 2003.
[27] I. Andreadis and A. Amanatiadis, “Digital imagescaling”, in Proc. IEEE Instrum. Meas. Technol. Conf. (IMTC), Ottawa, Canada, May 2005, pp. 2028–2032.

[28] J. F. Yang, H. M. Wang, and A. T. Chiang, “2D backwards compatible centralized color-depth packing,” Joint Collaborative Team on 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, the 6th Meeting: Document: JCT3V-F0087, Geneva, Nov. 2013.
[29] J. Canny, “A computational approach to edge detection,” IEEE Trans. PAMI, vol. 8, pp. 679–698, Nov. 1986.
[30] N. Kanopoulos, N. Vasanthavada, and R. L. Baker, “Design of an image edge detection filter using the Sobel operator,” IEEE J. Solid-State Circuits, vol. 23, pp. 358–367, Apr. 1998.
[31] G. N. Chaple, R. D. Daruwala, and M. S. Gofane, “Comparisions of Robert, Prewitt, Sobel operator based edge detection methods for real time uses on FPGA,” in Proc. Int. Conf. Technol. Sustain. Develop. (ICTSD), Mumbai, India, Feb. 2015, pp. 1–4.
[32] C. Soell, L. Shi, J. Roeber, M. Reichenbach, R. Weigel, and A. Hagelauer, “Low-power analog smart camera sensor for edge detection,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Phoenix, USA, Sep. 2016, pp. 4408–4412.
[33] A. Anand, S. S. Tripathy, and R. S. Kumar, “An improved edge detection using morphological Laplacian of Gaussian operator,” in Proc. 2nd Int. Conf. Signal Process. Integr. Netw. (SPIN), Noida, India, Feb. 2015, pp. 532–536.