為了提升電視顯示器的播放品質，我們提出一個包含提高畫面更新率（frame rate up-conversion）、畫面縮放（spatial scaling）、及適應性畫面邊界銳化（adaptively edge sharpening）的整合架構。我們提出將運動向量(motion vector)和平均絕對差值（mean absolute difference）的資訊運用在運動補償（motion compensation）和適應性邊界銳化上，實驗證明此演算法可提升畫面品質。在此，我們採用雙向運動估測（bi-directional motion estimation）的技術決定運動向量，並且不只考慮時間上的相關性，亦考慮空間上相關性，在連續兩張連續畫面中間內插出新的一張畫面。利用時間及空間上的相關性，我們將因運動補償造成的空洞及重疊區域(holes and overlapped areas) 分類，並將適當的畫素(pixel)填入這些區域內。在適應性畫面邊界銳化上，我們利用運動向量及平均絕對差值的大小決定銳化的權重，以增進畫面品質。我們利用系統層級設計的觀念，在虛擬平台(virtual platform)上實現及驗證此整合系統並藉由軟硬體共同設計的觀念設計架構，發揮軟體及硬體的優點，使更有效率的執行。 在模擬結果裡，我們提出的方法具有較高的峰值雜訊比（Peak Signal to Noise Ratio）比及畫面品質。

To improve video quality in TV display, we propose an architecture including frame rate up-conversion (FRUC), spatial scaling, and adaptively edge sharpening. The proposed scheme is that the information of motion vector (MV) and mean absolute difference (MAD) are applied in motion compensation and edge sharpening to improve the image quality. In FRUC, the bi-directional motion estimation is adopted and we not only consider the temporal correlation but also the spatial correlation to interpolate a new frame between two successive frames. The holes and overlapped areas created by motion compensation can be classified by considering the spatial correlation and inserted by the appropriate pixels. In the edge sharpening, the adaptively weighting is determined by the magnitude of MV and MAD of current block to improve image quality. We implement and verify our architecture by system level design on a virtual platform. By the hardware/software co-design, our architecture can be improved to get efficient performance. Simulation results show that our method has better PSNR and image quality.

[1] A. M. Tourapis, O. C. Au, and M. L. Liou, “Predictive motion vector field
adaptive search technique (PMVFAST)- enhancing block based motion
estimation,” in Proc. Visual Communications and Image Processing, Jan. 2001,
pp. 883-892.
[2] C.Y. Kuang, A. Vetro, S. Huifang and S.Y. Kung, “Frame-rate up-conversion
using transmitted true motion vectors,” in Proc. 2nd IEEE International
Workshop on Multimedia Signal Processing, Redondo Beach, Calif, USA,
December 1998, pp. 622-627.
[3] C. K. Wong, O.C. Au, “Fast Motion Compensated Temporal Interpolation for
Video,” in Proc. SPIE Visual Communication and Image Processing, May 1995,
pp.1108-1118.
[4] J. Zhai, K. Yu, J. Li, and S. Li, “A Low Complexity Motion Compensated
Frame Interpolation Method,” in Proc. IEEE 2005 Int. Conf. Circuits and
Systems, May 2005, pp. 4927-4930.
[5] K. Kawaguchi and S. K. Mitra, “Frame rate up-conversion considering multiple
motion,” in Proc. IEEE 1997 Int. Conf. Image Processing, Oct. 1997,
pp.727-730.
[6] E. G. T. Jaspers, P. H. N. de With, “A Generic 2D Sharpness Enhancement
Algorithm for Luminance Signals,” in Proc. IEEE Int. conf. Image Proc. &
Appl., July 1997, pp. 269-273.
[7] F. Gottardo, J. Mir, G. Parladori and G. L. Sicuranza, “nonlinear preprocessing
for mpeg-2 video sequences,” in Proc. IEEE-EURASIP 1999 Wrokshop on
Nonlinear Signal and Image Processing, June 1999, pp. 43-47.
[8] M. Janssen, J. G. Stessen, H. J. de With, “An advanced sampling rate conversion
technique for video and graphics signals,” in Proc. IEEE Int. Conf. Image
Processing and Its Applications, July 1997, pp. 14-17.
[9] R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE
Trans. on Acoustics, Speech, and Signal Processing, vol. 29, Dec 1981. pp. 1153
-1160.
[10] C. E. Duchon, “Lanczos Filtering in One and Two Dimensions,” J. Appl.
Meteor., 1979, pp. 1016-1022.
[11] D. P. Mitchell and A. N. Netravali, “Reconstruction Filters in Computer
Graphics,” in Proc. SIGGRAPH 1998, Aug. 1998, pp. 221-228.
[12] E. Hogenauer, “An economical class of digital filters for decimation and
interpolation,” IEEE Trans. on Acoustics, Speech and Signal Processing, Apr.
1981, pp.155-162.[13] C. K. Cheung, L. M. Po, “Normalized partial distortion search algorithm for
block motion estimation,” IEEE Trans. on Circuits and Systems for Video
Technology, vol. 10, pp. 417-422, April 2000.
[14] Y. Xiaoquan, L. Nam, “Improved partial distortion search algorithm for rapid
block motion estimation via dual-halfway-stop,” in Proc. IEEE Int. Conf.
Acoustics, Speech, and Signal Processing, March 2005, pp.917-920.
[15] C. H. Cheung, L. M. Po, “Adjustable partial distortion search algorithm for fast
block motion estimation,” IEEE Trans. on Circuits and Systems for Video
Technology, vol. 13, pp. 100-110, Jan. 2003.
[16] L. S.-H., S. Yang, “Adaptive motion-compensated frame rate up-conversion,”
Electronics Lett. , vol. 38, pp. 451-452, May 2002.
[17] M.A Nuno-Maganda, M.O. Arias-Estrada, “Real-time FPGA-based architecture
for bicubic interpolation: an application for digital image scaling,” in Proc.
IEEE Int. Conf. Reconfigurable Computing and FPGAs, Sept. 2005.
[18] R. Castagno, P. Haavisto, G. Ramponi, “A method for motion adaptive frame
rate up-conversion,” IEEE Trans. on Circuits and Systems for Video Technology,
vol. 6, pp. 436-446, Oct. 1996.
[19] H Taehyeun., L. Seongjoo, K. Jaeseok, “Motion compensated frame
interpolation by new block-based motion estimation algorithm,” IEEE Trans. on
Consumer Electronics, vol. 50, pp.752-759, May 2004.
[20] T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion
compensated interframe coding for videoconferencing,” in Proc. Nat.
Telecommun. Conf., New Orleans, LA, Dec’81. , pp. G5.3.1-G5.3.5
[21] L. Renxiang, Z. Bing, L. Ming, “A new three-step search algorithm for block
motion estimation,” IEEE Trans. on circuits and systems for video technology,
vol. 4, pp. 438-442, Aug. 1994.
[22] S. Zhu and K.K. Ma, “A new diamond search algorithm for fast block matching
motion estimation,” in Proc. IEEE Int. Conf. Communications and Signal
Processing, 1997, pp.292-6.
[23] P.I. Hosur and K.K. Ma, “Motion Vector Field Adaptive Fast Motion
Estimation,” in Proc. IEEE Int. Conf. Communications and Signal Processing,
Singapore, 7-10 Dec 1999.
[24] G. Ramponi, N. Strobel, S. K. Mitra, and T. Yu, “Nonlinear unsharp masking
methods for image contrast enhancement,” J. Electron. Imag, pp. 353-366, July
1996.
[25] MPEG4-AVC/H.264 JVT document archives: ftp://ftp.imtcfiles.org/jvtexperts
[26] http://www.coware.com
[27] http://www.arm.com/[28] http://www.cadence.com
[29] http://www.mentor.com
[30] L.M.Po, W.C.Ma, “A novel four-step search algorithm for fast block motion
estimation,” IEEE Trans. on CSVT, vo1.6, pp.313-317, June 1996
[31] Z. Ce, L. Xiao, and C. Lap-hi, “Hexagon-Based Search Patten for Fast Block
Motion Estimation,” IEEE Trans. on CSVT, vol.2, pp.349-355, May 2002
[32] C. Zhibo, Z. Peng, H. Yun, “Fast Integer Pel and Fractional Pel Motion
Estimation for JVT,” JVT-F017.doc, 6th Meeting. Awaji, Island, JP, pp.5-13,
December 2002
[33] C. Zhibo, Z. Peng, H. Yun, “Fast Motion Estimation for JVT,” JVT-G016.doc,
7th Meeting. Pattaya ll, Thailand, ppl.7-14, March, 2002
[34] K. Hilman, P. W. Hyun, K. Yongmin, “Using motion-compensated frame-rate
conversion for the correction of 3:2 pulldown artifacts in video sequences,”
IEEE Trans. on Circuits and Systems for Video Technology, vol. 10, pp. 869-877,
Sept. 2000.
[35] S. H. Lee, B. S. Hur, S. H. Kim, R.H. Park, “Weighted-adaptive
motion-compensated frame rate up-conversion,” in Proc. IEEE Int. Conf.
Consumer Electronics, 17-19 June 2003, pp,342-343
[36] S. Fujiwara, A. Taguchi, “Motion-compensated frame rate up-conversion based
on block matching algorithm with multi-size blocks,” in Proc. IEEE Int. Symp.
Intelligent Signal Processing and Communication Systems, 13-16 Dec. 2005, pp.
353-356
[37] C. H. Kim, S. M. Seong, J. A. Lee; Lee-Sup Kim;, “Winscale: an image-scaling
algorithm using an area pixel model,” IEEE Trans. on Circuits and Systems for
Video Technology, vol. 13, pp.549-553, June 2003
[38] I. Andreadis, A. Amanatiadis, “Digital Image Scaling,” in Proc. IEEE 2005
Instrumentation and Measurement Technology Conference, 16-19 May 2005,
pp.2028-2032.
[39] G. Ramponi, “A rational edge-preserving smoother,” in Proc. IEEE Int. Conf.
Image Processing, 23-26 Oct. 1995, pp.151-154.
[40] K. M. Nam, J. S. Kim, R. H. Park, Y. S. Shim, “A Fast Hierarchical Motion
Vector Estimation Algorithm Using Mean Pyramid,” IEEE Trans. on Circuits
and Systems for Video technology, vol. 5, no.4, pp. 344-351, August 1995.
[41] T. M. Oh, Y. R. Kim, W. G. Hong, S.J. Ko, “A fast full search motion estimation
algorithm using the sum of partial norms,” in Proc. IEEE Int. Conf. Consumer
Electronics, Dig. Tech. Papers, pp. 236-237, 13-15 June 2000.
[42] J.N. Kim, D.K. Kang, S.C. Byun, I.L. Lee, B. H. Ahn, “A fast full search
motion estimation algorithm using sequential rejection of candidates fromhierarchical decision structure,” IEEE Trans. on Broadcasting, vol. 48, pp.43-46,
March 2002.
[43] M. J. Chen, L.G. Chen, T. D. Chiueh, Y. P. Lee, “One-dimensional full search
motion estimation algorithm for video coding,” IEEE Trans. on Circuits and
Systems for Video Technology, vol.4, pp. 504-509, October 1994.
[44] B. Kienhuis, E. Deprettere, K. Vissers, P. Van Der Wolf, “An approach for
quantitative analysis of application-specific dataflow architectures,” in Proc.
IEEE Int. Conf. Application-Specific Systems, Architectures and Processors,
14-16 July 1997, pp.338-349.
[45] ISO/IEC MPEG and ITU-T VCEG, “Draft ITU-T Recommendation and Final
Draft International Standard of Joint Video Specification,” May 2003.
[46] T. Y. Kuo, C.-C. J. Kuo, “Motion-compensated interpolation for low-bit-rate
video quality enhancement,” SPIE Annual Meeting 1998 Applications of Digital
Image Processing XXI, vol. 3460, pp. 277-288, July 1998.
[47] S. Liu, J. Kim, and C.-C. J. Kuo, “MCI-embedded motion compensated
prediction for quality enhancement of frame interpolation,” presented at the
SPIE Int. Symp. Voice, Video, Data Communications, Multimedia Syst. Appl.,
Boston, MA, Nov. 2000.
[48] B.D. Choi, J.W. Han, C.S. Kim, S.J. Ko, “Frame rate up-conversion using
perspective transform,” IEEE Trans. on Consumer Electronics, vol. 52, pp.
975-982, Aug. 2006
[49] S. C. Han, J.W. Woods, “Frame-rate up-conversion using transmitted motion
and segmentation fields for very low bit-rate video coding,” in Proc. IEEE Int.
Conf. Image Processing, 26-29 Oct. 1997, pp. 747-750.
[50] A.M. Huang, T. Nguyen, “A Novel Motion Compensated Frame Interpolation
Based on Block-Merging and Residual Energy,” 2006 IEEE 8th Workshop on
Multimedia Signal Processing, pp. 395-398, Oct. 2006.
[51] C. Tao, “Adaptive temporal interpolation using bidirectional motion estimation
and compensation,” in Proc. IEEE Int. Conf. Image Processing, 22-25 Sept.
2002, pp. II-313 - II-316
[52] Z. Jianning, S. Lifeng, Y.Shiqiang, Z. Yuzhuo, “Position prediction
motion-compensated interpolation for frame rate up-conversion using temporal
modeling,” in Proc. IEEE Int. Conf. Image Processing, 11-14 Sept. 2005, pp. I -
53-6
[53] J.K. Su, R.M. Mersereau, “Motion-compensated interpolation of untransmitted
frames in compressed video,” in Proc. IEEE Int. Conf. Signals, Systems and
Computers, 3-6 Nov. 1996, pp. 100-104.
[54] Y. T. Yang, Y. S. Tung, J.L. Wu, C.Y. Weng, “Low-Bitrate Video QualityEnhancement by Frame Rate Up-Conversion and Adaptive Frame Encoding,”
PCM (2), pp.841-853, 2005
[55] S.H. Lee, O. Kwon, R.H. Park, “Weighted-adaptive motion-compensated frame
rate up-conversion,” IEEE Trans. on Consumer Electronics, vol. 49, pp.
485-492, Aug. 2003
[56] S. H. Lee, Y. C. Shin, S. Yang, H. H. Moon, R. H. Park, “Adaptive
motion-compensated interpolation for frame rate up-conversion,” IEEE Trans.
on Consumer Electronics, vol. 48, pp.444-450 , Aug. 2002
[57] S. H. Lee, Y. C. Shin, S. Yang, H. H. Moon, R. H. Park, “Adaptive
motion-compensated interpolation for frame rate up-conversion,” in Proc. IEEE
Int. Conf. Consumer Electronics, Dig. Tech. Papers, pp.68-69, 18-20 June 2002.
[58] H.Sasai, S.Kondo, S.Kadono, “Frame-rate up-conversion using reliable analysis
of transmitted motion information,” in Proc. IEEE Int. Conf. Acoustics, Speech
and Signal Processing, 17-21 May 2004, pp.257-260.
[59] D.Alfonso, D. Bagni, D. Moglia, “Bi-directionally motion-compensated
frame-rate up-conversion for H.264/AVC decoders,” in Proc. 47th Int. Symp.
ELMAR, 8-10 Jun. 2005, pp.41-44.
[60] G. Dane, K. El-Maleh, L. Yen-Chi, “Encoder-Assisted Adaptive Video Frame
Interpolation,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal
Processing, 18-23 March 2005, pp. 349-352
[61] T. Y. Kuo, J. W. Kim, C.C.J. Kuo, “Motion-compensated frame interpolation
scheme for H.263 codec,” in Proc. IEEE Int. Symp. Circuits and Systems, June
1999, pp.491-494.
[62] R.B.Wittebrood, G. de Haan, R. Lodder, “Tackling occlusion in scan rate
conversion systems,” in Proc. IEEE Int. Conf. Consumer Electronics, 17-19
June 2003, pp.344-345
[63] D.O Kim, R.H. Park, “Recognition of 3-D objects with curved surfaces based
on the cross entropy between shape histograms,” in Proc. SPIE 2000 Visual
Communications and Image Processing, vol. 4067, pp. 1155-1163, June 2000.
[64] D.Alfonso, D.Bagni, D.Moglia, “Motion-compensated frame-rate up-conversion
for H.264/AVC decoders,” in Proc. IEEE Int. Conf. Consumer Electronics,
Dig. Tech. Papers, pp. 439 – 440, 8-12 Jan. 2005
[65] M.E. Al-Mualla, “Motion field interpolation for frame rate conversion,” in Proc.
IEEE Int. Symp. Circuits and Systems, 25-28 May 2003, pp. II-652 - II-655.
[66] M. E. Al-Mualla, C. N. Canagarajah, D. R.Bull, “Motion field interpolation for
improved motion compensation and frame-rate Conversion,” in Proc. IEEE Int.
Conf. Visual Information Engineering, 7-9 July 2003, pp. 17-20.
[67] S. H. Lee, O. Kwon, R. H. Park, “Motion vector correction based on thepattern-like image analysis,” IEEE Trans. on Consumer Electronics, , vol.
49, pp. 479 – 484, Aug. 2003.
[68] N.Grammalidis, T D.zovarns, M.G. Strintzis, “Temporal frame interpolation for
stereoscopic sequences using object-based motion estimation and occlusion
detection,” in Proc. IEEE Int. Conf. Image Processing, 23-26 Oct. 1995, pp.
382-385.
[69] C.Auyeung, Kosmach, J.James , T.Orchard, T. Kalafatis, “Overlapped Block
Motion Compensation,” in Proc. SPIE, Visual Communications and Image
Processing, vol. 1818, pp. 561-572, 1992
[70] J. Ascenso, C. Brites, and F. Pereira, “Improving frame interpolation with
spatial motion smoothing for pixel domain distributed video coding,” in Proc.
European Association for Signal 2005, Speech and Image Processing, July 2005
[71] E.A. Lee, T.M. Parks, “Dataflow process networks,” in Proc. IEEE, vol.
83, pp.773-801, May 1995
[72] P. Rieder, G. Scheffler, “New concepts on denoising and sharpening of video
signals,” IEEE Trans. on Consumer Electronics, vol. 47, pp.666-671, Aug. 2001
[73] R. Giovanni, “A cubic unsharp masking technique for contrast enhancement,” in
Proc. IEEE Int. Conf. Signal Processing, 1998, pp. 211-222
[74] S. K. Mitra, H. Li, I. S. Lin, T.-H.Yu, “A new class of nonlinear filters for image
enhancement,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal
Processing, pp.2525 – 2528, 14-17 April 1991
[75] H. D. Cheng, X. J. Shi, “A simple and effective histogram equalization
approach to image enhancement,” in Proc. IEEE Int. Conf. Digital Signal
Process, pp. 158-170, 2003.
[76] R. Bernstein, M. Moore, S. Mitra, “Adjustable quadratic filters for image
enhancement,” in Proc. IEEE Int. Conf. Image Processing, 26-29 Oct. 1997,
pp.287-290
[77] A. Polesel, G. Ramponi, V. J. Mathews, “Adaptive unsharp masking for contrast
enhancement,” in Proc. IEEE Int. Conf. Image Processing, 26-29 Oct. 1997,
pp.267-270
[78] S. Carrato, L. Tenze, “A high quality 2×image interpolator,” in Proc. IEEE
Signal Processing Lett., vol. 7, pp. 132-134, June 2000.
[79] Y. Shuai, M. Abe, A. Taguchi, M. Kawamata, “High accuracy WaDi image
interpolation with local gradient features,” in Proc. IEEE Int. Symp. Intelligent
Signal Processing and Communication Systems, 13-16 Dec. 2005, pp. 85–88
[80] L. Xin, M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. on
Image Processing, vol. 10, Issue 10, pp.1521-1527, Oct. 2001
[81] S. Carrato, G. Ramponi, S. Marsi, “A simple edge-sensitive image interpolationfilter,” in Proc. IEEE Int. Conf. Image Processing, 16-19 Sept. 1996, pp.
711-714.
[82] S. Oraintara, T. Q. Nguyen, “M-th band filter design based on cosine
modulation,” in Proc. IEEE Int. Symp. Circuits and Systems, 31 May June 1998,
pp.37-40.
[83] C. F. Alaya, L. Khriji, M. Gabbouj, G. Ramponi, “Color Image Interpolation
Using Vector Rational Filters,” in Proc. SPIE Conf. Nonlinear Image
Processing IX, vol.3304-26, 24-30 January 1998.
[84] B. Wuensche, E. Tempero, “A Comparison and Evaluation of Interpolation
Methods for Visualising Discrete 2D Survey Data,” in Proc. Australasian symp.
Information Visualisation, pp.1-7, February 2004.
[85] C. Y. Chen, S. Y. Chien, W. M. Chao, Y. W. Huang, L. G. Chen, “Hardware
architecture for global motion estimation for MPEG-4 Advanced Simple
Profile,” in Proc. IEEE Int. Symp. Circuits and Systems, 23-26 May 2004. pp.
301-4.