高品質影片縮放技術的關鍵為保留重要物體的形狀並保持該物體在時間軸的一致性。現有的方法很難在這兩者上都達成，必須在這兩者中捨去一個，這樣的結果造成物體有波浪狀的扭曲。近年來為達成重要物體在時間軸的一致性有學者提出針對整個影片最佳化的方法。這大幅地增進縮放影片的品質，但是在來源影片的解析度與時間上並沒有很好的延展性，這是一個難以解決的問題。因此我們提出在沒有降低縮放影片的品質下增強影片的延展性的演算法。我們主要著重在時間與空間兩部分，首先我們對單一圖框重新縮放達到保留重要物體的形狀，接著我們最佳化利用視覺流量所找出的特徵點路徑線達到時間軸的一致性。這讓我們在最佳化影片時分成許多子問題，這些子問題的大小是與單一圖框的解析度成正比，並可以被平行地運算。我們並解說了如何加入裁切至系統，裁切用於當影片有多個重要物體時，只使用不等比例縮放會變成線性縮放。我們的結果可以跟目前最好的方法一樣好，並同時大幅地降低計算時間與記憶體消耗量，使得內容為主的影片縮放技術有延展性與實際性。

The key to high-quality video resizing is preserving the shape and motion of visually salient objects while remaining temporally-coherent. These spatial and temporal requirements are difficult to reconcile, typically leading existing video retargeting methods to sacrifice one of them and causing distortion of waving artifacts. Recent work enforces temporal coherent of content-aware video warping by solving global optimization problem over the entire video cube. This significantly improves the results but does not scale well with the resolution and length of the input video and quickly becomes intractable. We purpose a new method that solves the scalability problem without compromising the resizing quality. Our method factors the problem into spatial and time/motion components: we first resize each frame independently to preserve the shape of salient regions, and then we optimize their motion using a reduced model for each pathline of the optical flow. This factorization decomposes the optimization of the video cube into sets of sub-problems whose size is proportional to a signal frame’s resolution and which can be solved in parallel. We also show how to incorporate cropping into our optimization, which is useful for scenes with numerous salient objects where warping alone would degenerate to linear scaling. Our results match the quality of the-state-of-the-art retargeting methods while dramatically reducing the computation time and memory consumption, making content-aware video resizing scalable and practical.

[1] AVIDAN, S., AND SHAMIR, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3, 10.
[2] BARNES, C., SHECHTMAN, E., FINKELSTEIN, A., AND GOLDMAN, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3.
[3] BUATOIS, L., CAUMON, G., AND L´E VY, B. 2009. Concurrent number cruncher: a GPU implementation of a general sparse linear solver. Int. J. Parallel Emerg. Distrib. Syst. 24, 3, 205–223.
[4] CHEN, L. Q., XIE, X., FAN, X., MA, W. Y., ZHANG, H. J., AND ZHOU, H. Q. 2003. A visual attention model for adapting images on small displays. ACM Multimedia Systems Journal 9, 4, 353–364.
[5] CHO, T. S., BUTMAN, M., AVIDAN, S., AND FREEMAN, W. T. 2008. The patch transform and its applications to image editing. In CVPR ’08.
[6] DESELAERS, T., DREUW, P., AND NEY, H. 2008. Pan, zoom, scan: Time-coherent, trained automatic video cropping. In CVPR.
[7] DONG, W., ZHOU, N., PAUL, J.-C., AND ZHANG, X. 2009. Optimized image resizing using seam carving and scaling. ACM Trans. Graph. 28, 5, 1–10.
[8] GAL, R., SORKINE, O., AND COHEN-OR, D. 2006. Featureaware texturing. In EGSR ’06, 297–303.
[9] GLEICHER, M. L., AND LIU, F. 2008. Re-cinematography: Improving the camerawork of casual video. ACM Trans. Multimedia Comput. Common. Appl. 5, 1, 1-28.
[10] KARNI, Z., FREEDMAN, D., AND GOTSMAN, C. 2009. Energy based image deformation. Comput. Graph. Forum 28, 5, 1257–1268.
[11] KR¨AHENB¨U HL, P., LANG, M., HORNUNG, A., AND GROSS, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28, 5.
[12] LIU, F., AND GLEICHER, M. 2006. Video retargeting: automating pan and scan. In Multimedia ’06, 241–250.
[13] LIU, H., XIE, X., MA, W.-Y., AND ZHANG, H.-J. 2003. Automatic browsing of large pictures on mobile devices. In Proceedings of ACM International Conference on Multimedia, 148–155.
[14] NIU, Y., LIU, F., LI, X. AND GLEICHER, M. 2010. Warp propagation for video resizing. CVPR 537-544.
[15] PRITCH, Y., KAV-VENAKI, E., AND PELEG, S. 2009. Shift-map image editing. In ICCV’09.
[16] RASHEED, Z., AND SHAH, M. 2003. Scene detection in Hollywood movies and TV shows. In CVPR ’03, vol. 2, II–343–8.
[17] RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3.
[18] RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S. 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3, 23.
[19] SANTELLA, A., AGRAWALA, M., DECARLO, D., SALESIN, D., AND COHEN, M. 2006. Gaze-based interaction for semiautomatic photo cropping. In Proceedings of CHI, 771–780.
[20] SHAMIR, A., AND SORKINE, O. 2009. Visual media retargeting. In ACM SIGGRAPH Asia Courses.
[21] SIMAKOV, D., CASPI, Y., SHECHTMAN, E., AND IRANI, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR ’08.
[22] SUH, B., LING, H., BEDERSON, B. B., AND JACOBS, D. W. 2003. Automatic thumbnail cropping and its effectiveness. In Proceedings of UIST, 95–104.
[23] VIOLA, P., AND JONES, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57, 2, 137–154.
[24] WANG, Y.-S., TAI, C.-L., SORKINE, O., AND LEE, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5, 118.
[25] WANG, Y.-S., FU, H., SORKINE, O., LEE, T.-Y., AND SEIDEL, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Trans. Graph. 28, 5.
[26] WANG, Y.-S., LIN, H-C., SORKINE, O., LEE, T.-Y. 2010. Motion-based video retargeting with optimal crop-and-warp. ACM Trans. Graph. 29, 4, article no, 90.
[27] WERLBERGER, M., TROBIN, W., POCK, T., WEDEL, A., CREMERS, D., AND BISCHOF, H. 2009. Anisotropic Huber-L1 optical flow. In Proceedings of the British Machine Vision Conference (BMVC).
[28] WOLF, L., GUTTMANN, M., AND COHEN-OR, D. 2007. Non-homogeneous content-driven video-retargeting. In ICCV ’07.
[29] Wu, H., WANG, Y.-S., FENG, K.-C., WONG, T.-T., LEE T.-Y., AND HENG, P.-A. 2010. Resizing by symmetry-summarization. ACM Trans. Graph. 29, 6, 159:1-159:9.
[30] ZHANG, Y.-F., HU, S.-M., AND MARTIN, R. R. 2008. Shrinkability maps for content-aware video resizing. In PG ’08.
[31] ZHANG, G.-X., CHENG, M.-M., HU, S.-M., AND MARTIN, R. R. 2009. A shape-preserving approach to image resizing. Computer Graphics Forum 28, 7, 1897–1906.