圖像大小重調，除了要維持幾何，也必須考慮圖像內容，Avidan and Shamir [2]提出了一個稱為隙縫移除(seam carving)的圖像縮放演算法。透過定義一個像素的能量公式，計算出每條細縫的能量，seam carving法按能量從低到高將隙縫逐一移除來達成說小影像寬(或高)度的目的。在圖像主體不夠明顯、圖像組成太過複雜等情況下，seam carving演算法容易破壞圖像，導致主體破碎的結果。

為了改善seam carving的缺陷，我們提出了一個結合seam carving, content-aware cropping and standard scaling的演算法，首先二值化saliency map，找出圖像中重要的區域，在保存區域寬度與區域間距離的情況下利用seam carving 移除圖像中相對能量較低的seam，當區域寬度與區域間距離降低到低於臨界值，我們以content-aware cropping法取代seam carving將圖像兩側不重要的區域移除。最後我們以standard scaling將圖像等比例縮小至目標尺寸。此演算法結合了三種影像縮放技術的優點，定義出適當的轉換時機，藉此得到最佳的輸出結果。

As an image is resized not only the geometry of objects but also the image content musted be maintained. Avidan and Shamir [2] proposed an image resizing algorithm called seam carving. By defining an energy function for a seam, the seam carving method reduce the width of an image by removing seams in every increasing order. Since the seams removed contain lower energy the content of the image is well preserved. But there are still some problems in seam carving. If objects in the image is not protected or the image composition is complex, the seam carving method may generate broken images.
In order to solve the problem, we propose a content-aware image resizing algorithm which resizes an image by combining seam carving, content-aware cropping and standard scaling. In the method, important regions are identified and the within region width and between region distance are calculated to set the condition of terminating the seam carving procedure. After the seam carving step, the content-aware cropping is used to remove the unimportant boundaries according to saliency map. In the final step, uniform scaling is employed to scale the image into its targeted size. The proposed algorithm benefits by taking the advantages of the three image resizing techniques employed.

References

[1] R. Achanta, S. Süsstrunk, “Saliency detection using maximum symmetric surround,” IEEE International Conference on Image Processing, 2010, pp. 2653 – 2656.

[2] S. Avidan, A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph., vol. 26, no. 3, pp. 1–10, 2007.

[3] T. Basha, Y. Moses, S. Avidan, “Geometrically consistent stereo seam carving,” IEEE International Conference Computer Vision, 2011, pp. 1816 – 1823.

[4] S. Battiato, G. Maria Farinella, G. Puglisi, D. Ravì, “Saliency-Based Selection of Gradient Vector Flow Paths for Content Aware Image Resizing,” IEEE Transactions on Image Processing, vol. 23, no. 5, pp. 2081–2095, 2014.

[5] G. Ciocca, C. Cusano, F. Gasparini, R. Schettini, “Self-adaptive image cropping for small displays,” IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1622–1627, 2007.

[6] Dong, W., Zhou, N., Paul, J., Zhang, X. “Optimized Image Resizing Using Seam Carving and Scaling,” ACM Trans. Graph., vol. 28, no. 5, Article 125, 2009.

[7] S. Goferman, L. Zelnik-Manor, A. Tal, 2010, “Context-aware saliency detection,” IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2376 – 2383.

[8] L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254 - 1259, 1998.

[9] M. Kumar, D.D. Conger, R.L. Miller, J. Luo, H. Radha, “A distortion-sensitive seam carving algorithm for content-aware image resizing,” Journal of Signal Processing Systems, vol. 65, no. 2, pp. 159–169, 2011.

[10] K, -C, Lien, M, Turk, “On Preserving Structure in Stereo Seam Carving,” IEEE International Conference on 3D Vision, 2015, pp. 571 – 579.

[11] Y. –W. Miao, H. Lin, “Visual Saliency Guided Global and Local Resizing for 3D Models,” IEEE International Conference on Computer-Aided Design and Computer Graphics, 2013, pp. 212 – 219.

[12] M. Nishiyama, T. Okabe, Y. Sato, I. Sato, “Sensation-based photo cropping,” ACM international conference on Multimedia, pp. 669–672, 2009.

[13] M. Rubinstein, A. Shamir, S. Avidan, “Improved seam carving for video retargeting,” ACM Trans. Graph., vol. 27, no 3, Article 16, 2008.

[14] M. Rubinstein, A. Shamir, S. Avidan, “Multi-operator media retargeting,” ACM Trans. Graph., vol. 28, no 3, Article 23, 2009.

[15] F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, R. Fu, “An Energy-Constrained Video Retargeting Approach for Color-Plus-Depth 3D Video,” IEEE Journal of Display Technology, vol. 12, no. 5, pp. 491 – 499, 2016.

[16] F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, R. Fu, “Stereoscopic Visual Attention Guided Seam Carving for Stereoscopic Image Retargeting,” IEEE Journal of Display Technology, vol. 12, no. 5, pp. 491 - 499, 2016.

[17] B. Suh, H. Ling, B.B. Bederson, D.W. Jacobs, 2003, “Automatic thumbnail cropping and its effectiveness,” 16th Annual ACM Symposium on User Interface Software and Technology, pp. 95–104, 2003.

[18] Q. Wang, P. Yan, Y. Yuan, X. Li, 2013, “Multi-spectral saliency detection,” Pattern Recognition Letters. pp. 34 – 41, 2013.

[19] Y.-S. Wang, C.-L. Tai, O. Sorkine, T.-Y. Lee, “Optimized scale-and-stretch for image resizing,” ACM Trans. Graph., vol. 27, no. 5, Article 118, 2008.

[20] Q. Wang, Y. Yuan, P. Yan, 2013, “Visual saliency by selective contrast,” IEEE Trans. Circuits Syst. Video Technol. vol. 23, no. 7, pp. 1150 – 1155, 2013.

[21] Q. Wang, G. Zhu, Y. Yuan, “Multi-spectral dataset and its application in saliency detection,” Computer Vision and Image Understanding. vol. 117, pp. 1748 – 1754, 2013.

[22] V. Setlur, S. Takagi, R. Raskar, M. Gleicher, and B. Gooch, “Automatic image retargeting,” ACM SIGGRAPH Technical Sketch, page 4, 2004.

[23] F. Shao, W. Lin, W. Lin, “Stereoscopic Visual Attention Guided Seam Carving for Stereoscopic Image Retargeting,” IEEE Journal of Display Technology, vol. 12, no. 1, pp. 22-30, 2016.

[24] P. C. Su, Y, C Chou, “An Image Retargeting Scheme with Content-based Cropping and Local Significance Aware Seam Carving,” Signal and Information Processing Association Annual Summit and Conference, 2015.

[25] Q. Zeng, K. Li, B. Bare, B. Yan, “Improved seam carving for image retargeting with sift feature preservation,” IEEE International Conference on Signal and Information Processing, 2015.

[26] X. Zhang, S. Zhong, Z. Pan, K. Wong, R. Yun (Eds.), “Optimal Bi-directional Seam Carving for Content-Aware Image Resizing,” Entertainment for Education. Digital Techniques and Systems, vol. 6249, pp. 456-467, 2010.