近年來由於螢幕比例大小日益月新，我們提出影片重新縮放方法，針對不同拍攝手法及動態內容影像等複雜性較高之影片，皆能達到高品質任意比例縮放。之前研究中，影片重新縮放大部分強調內容於空間的關係，維持重要物體於影片中都能有一致地外觀比例，藉由移除或扭曲較不重要背景內容。然而，基本上影片上空間有限，一旦有移動物體，那將使得前景與背景變得密不可分，因而減少可移除或扭曲變形的空間。本論文提出新穎方法解決影片重新縮放問題，明確地利用物體移動資訊且將扭曲失真分佈於空間及時間維度上。我們結合裁切與不等比例縮放；利用裁切移除去重複出現內容，並使用不等比例縮放將較不重要區域進行扭曲變形，並且可維持移動中物體形狀。最後我們利用最佳化運算，於不等比例縮放和裁切上尋找使用比例平衡點，並可使用在具有複雜動作、眾多突顯前景物體、以及任意景深變化的影片。最後，我們的結果與其他最新影片重新縮放系統進行使用者調查，由結果可以了解我們的技術獲得廣泛支持。

We introduce a video retargeting method that achieves high-quality resizing to arbitrary aspect ratios for complex videos containing diverse camera and dynamic motions. Previous content-aware retargeting methods mostly concentrated on spatial considerations, attempting to preserve the shape of salient objects in each frame by removing or distorting homogeneous background content. However, sacrificeable space is fundamentally limited in video, since object motion makes foreground and background regions correlated, causing waving and squeezing artifacts. We solve the retargeting problem by explicitly employing motion information and by distributing distortion in both spatial and temporal dimensions. We combine novel cropping and warping operators, where the cropping removes temporally-recurring contents and the warping utilizes available homogeneous regions to mask deformations while preserving motion. Variational optimization allows to find the best balance between the two operations, enabling retargeting of challenging videos with complex motions, numerous prominent objects and arbitrary depth variability. Our method compares favorably with state-of-the-art retargeting systems, as demonstrated in the examples and widely supported by the conducted user study.

[1] AVIDAN, S., AND SHAMIR, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3, 10.

[2] BARNES, C., SHECHTMAN, E., FINKELSTEIN, A., AND GOLDMAN, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3.

[3] BUATOIS, L., CAUMON, G., AND L´E VY, B. 2009. Concurrent number cruncher: a GPU implementation of a general sparse linear solver. Int. J. Parallel Emerg. Distrib. Syst. 24, 3, 205–223.

[4] CHEN, L. Q., XIE, X., FAN, X., MA, W. Y., ZHANG, H. J., AND ZHOU, H. Q. 2003. A visual attention model for adapting images on small displays. ACM Multimedia Systems Journal 9, 4, 353–364.

[5] CHO, T. S., BUTMAN, M., AVIDAN, S., AND FREEMAN, W. T. 2008. The patch transform and its applications to image editing. In CVPR ’08.

[6] DAVID, H. A. 1963. The Method of Paired Comparisons. Charles Griffin & Company.

[7] DESELAERS, T., DREUW, P., AND NEY, H. 2008. Pan, zoom, scan: Time-coherent, trained automatic video cropping. In CVPR.

[8] DONG, W., ZHOU, N., PAUL, J.-C., AND ZHANG, X. 2009. Optimized image resizing using seam carving and scaling. ACM Trans. Graph. 28, 5, 1–10.

[9] FAN, X., XIE, X., ZHOU, H.-Q., AND MA, W.-Y. 2003. Looking into video frames on small displays. In Multimedia ’03, 247–250.

[10] GAL, R., SORKINE, O., AND COHEN-OR, D. 2006. Featureaware texturing. In EGSR ’06, 297–303.

[11] ITTI, L., KOCH, C., AND NIEBUR, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 11, 1254–1259.

[12] KARNI, Z., FREEDMAN, D., AND GOTSMAN, C. 2009. Energy-based image deformation. Comput. Graph. Forum 28, 5, 1257–1268.

[13] KR¨AHENB¨U HL, P., LANG, M., HORNUNG, A., AND GROSS, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28, 5.

[14] LIU, F., AND GLEICHER, M. 2006. Video retargeting: automating pan and scan. In Multimedia ’06, 241–250.

[15] LIU, H., XIE, X., MA, W.-Y., AND ZHANG, H.-J. 2003. Automatic browsing of large pictures on mobile devices. In Proceedings of ACM International Conference on Multimedia, 148–155.

[16] PRITCH, Y., KAV-VENAKI, E., AND PELEG, S. 2009. Shift-map image editing. In ICCV’09.

[17] RASHEED, Z., AND SHAH, M. 2003. Scene detection in Hollywood movies and TV shows. In CVPR ’03, vol. 2, II–343–8.

[18] RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3.

[19] RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S. 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3, 23.

[20] SANTELLA, A., AGRAWALA, M., DECARLO, D., SALESIN, D., AND COHEN, M. 2006. Gaze-based interaction for semiautomatic photo cropping. In Proceedings of CHI, 771–780.

[21] SHAMIR, A., AND SORKINE, O. 2009. Visual media retargeting. In ACM SIGGRAPH Asia Courses.

[22] SIMAKOV, D., CASPI, Y., SHECHTMAN, E., AND IRANI, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR ’08.

[23] SUH, B., LING, H., BEDERSON, B. B., AND JACOBS, D. W. 2003. Automatic thumbnail cropping and its effectiveness. In Proceedings of UIST, 95–104.

[24] VIOLA, P., AND JONES, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57, 2, 137–154.

[25] WANG, Y.-S., TAI, C.-L., SORKINE, O., AND LEE, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5, 118.

[26] WANG, Y.-S., FU, H., SORKINE, O., LEE, T.-Y., AND SEIDEL, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Trans. Graph. 28, 5.

[27] WERLBERGER, M., TROBIN, W., POCK, T., WEDEL, A., CREMERS, D., AND BISCHOF, H. 2009. Anisotropic Huber-L1 optical flow. In Proceedings of the British Machine Vision Conference (BMVC).

[28] WOLF, L., GUTTMANN, M., AND COHEN-OR, D. 2007. Non-homogeneous content-driven video-retargeting. In ICCV ’07.

[29] ZHANG, Y.-F., HU, S.-M., AND MARTIN, R. R. 2008. Shrinkability maps for content-aware video resizing. In PG ’08.

[30] ZHANG, G.-X., CHENG, M.-M., HU, S.-M., AND MARTIN, R. R. 2009. A shape-preserving approach to image resizing. Computer Graphics Forum 28, 7, 1897–1906.