傳統的立體匹配演算法僅利用單一帧裡的空間資訊計算視差，受到紋理重複或遮蔽的影響下，很容易計算錯誤並且產生雜訊。在對序列影像計算視差時，由於物體的移動，每幀的雜訊情況可能會不一樣，在產生深度圖後播放時就會有深度跳動的現象。在這篇論文，我們提出了一個基於十字的時空區域性立體匹配演算法，希望在計算視差時，藉由加入時間上資訊來達到深度的一致性。方法一開始我們建立屬於每個像素的支持區域，並分別利用顏色、周遭紋理和支持區域的形狀來比較兩像素的相似性。接下來在計算每幀像素視差時，透過像素的搜尋，追蹤每個像素在上一幀的位置並且參考其資訊，再利用懲罰的限制讓兩幀計算出的視差一致。在方法的最後，我們執行左右一致性檢查和序列深度一致性等後處理，對於空間和時間上的深度進一步的修補和平滑化。由實驗的結果顯示，我們的方法可以有效的避免紋理重複可能造成的錯誤，同時減少很多時間上深度的跳動，並且能夠有效的維持物體的深度邊界。

Traditional stereo matching algorithm estimates disparity by using spatial information in one pair of images, but it is easy to estimate error and generate noise due to repeated textures or occlusion. When we calculate the disparity for sequential images, the noise situation in each frame may be different due to the movement of objects, and flickering-artifacts will be produced when you play the depth map. In this paper, we propose a cross-based spatio-temporal local stereo matching. By adding temporal information when we estimate disparity, we hope to achieve depth consistency. In the beginning, we construct a support region for each pixel, then we find the similarities in their colors, surrounding textures and the shape of support regions of two pixels. Next, when we estimate the disparity of each pixel, we track the location of each pixel in the preceding frame and reference their information, and we use the penalty restriction to make the disparity computed by two frames consistent. In the end, we perform post-processing methods, such as left-right consistency check and temporal consistency of depth sequence to repair and smooth the spatial and temporal depth. The experimental results show that our method can effectively avoid the errors caused by repeated textures, while reducing the flickering-artifacts and maintaining the depth boundary of the object.

[1] Kuk-Jin Yoon and In-So Kweon, “Adaptive Support-Weight Approach for Correspondence Search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.28(4), pp. 650-656, 2006.

[2] Ke Zhang, Jiangbo Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images,” IEEE Transactions on Circuits and Systems for Video Technology, vol.19(7), pp. 1073-1079, 2009.

[3] Asmaa Hosni, Michael Bleyer, Margrit Gelautz, and Christoph Rhemann, “Local Stereo Matching Using Geodesic Support Weights,” IEEE International Conference on Image Processing, pp. 2093-2096, 2009.

[4] Yuri Boykov, Olga Veksler, and Ramin Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.23(11), pp. 1222-1239, 2001.

[5] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions Using Graph Cuts,” IEEE International Conference on Computer Vision, vol.2,
pp. 508-515, 2001.

[6] Jian Sun, Nan-Ning Zheng, and Heung-Yeung Shum, “Stereo Matching Using Belief Propagation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25(7), pp. 787-800, 2003.

[7] Qingxiong Yang, Liang Wang, Ruigang Yang, Henrik Stewe´ nius, and David Niste´ r, “Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.31(3), pp. 492-504, 2009.

[8] Li Zhang, Brian Curless, and Steven M. Seitz, “Spacetime Stereo: Shape Recovery for Dynamic Scenes,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 367-374, 2003.

[9] James Davis, Diego Nehab, Ravi Ramamoorthi, and Szymon Rusinkiewicz, “Spacetime Stereo: A Unifying Framework for Depth from Triangulation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.27(2), pp. 296-302, 2005.

[10] Michael Bleyer and Margrit Gelautz, “Temporally Consistent Disparity Maps from Uncalibrated Stereo Videos,” Image and Signal Processing and Analysis, pp. 383-387, 2009.

[11] E. Scott Larsen, Philippos Mordohai, Marc Pollefeys, and Henry Fuchs, “Temporally consistent reconstruction from multiple video streams using enhanced belief propagation,” IEEE International Conference on Computer Vision, pp. 1-8, 2007.

[12] Cuong Cao Pham, Vinh Dinh Nguyen, and Jae Wook Jeon, “Efficient spatio-temporal local stereo matching using information permeability filtering,” IEEE International Conference on Image Processing, pp. 2965-2968, 2012.

[13] Ramin Zabih and John Wood Ll, “Non-parametric Local Transforms for Computing Visual Correspondence,” European conference on Computer Vision, pp. 151-158, 1994.

[14] Christian Richardt, Douglas Orr, Ian Davies, Antonio Criminisi and Neil A. Dodgson, “Real-Time Spatiotemporal Stereo Matching Using the Dual-Cross-Bilateral Grid,” European conference on ComputerVision, pp. 510-523, 2010.

[15] Wan-Yu Chen, Yu-Lin Chang, Shyh-Feng Lin, Li-Fu Ding and Liang-Gee Chen, “Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation,” IEEE International Conference on Multimedia and Expo, pp. 1314-1317, 2005.

[16] Xun Sun, Xing Mei, Shaohui Jiao, Mingcai Zhou and Haitao Wang, “Stereo Matching with Reliable Disparity Propagation,” International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pp. 132-139, 2011.

[17] Li Hong and George Chen, “Segment-based stereo matching using graph cuts,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol.1, pp.I-74 - I-81, 2004.

[18] Andreas Klaus, Mario Sormann and Konrad Karner, “Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure,” International Conference on Pattern Recognition, vol.3, pp. 15-18, 2006.

[19] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” International Journal of Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.

[20] Heiko Hirschmu¨ ller, “Stereo Processing by Semiglobal Matching and Mutual Information,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, 2008.