簡易檢索 / 詳目顯示

回結果列表
研究生： 黃明哲
Huang, Ming-Zhe
論文名稱： 以單目魚眼相機進行距離測量之研究
A Research of Distance Prediction Based on Monocular Fisheye Camera
指導教授： 莊智清
Juang, Jyh-Ching
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2026
畢業學年度： 114
語文別： 英文
論文頁數： 78
中文關鍵詞： 距離預測 、語意分割 、尺度缺失 、單目相機
外文關鍵詞： Distance Prediction, Semantic Segmentation, Scale Ambiguity, Monocular Camera
相關次數： 點閱：3  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 隨著自動駕駛與穿戴式輔助載具的快速發展，周遭環境的深度感知與絕對距離預測已成為攸關系統安全性的核心技術。而關於做到深度感知，現行方案當中，通常可藉由多重感測器來輔助，例如光達、慣性測量單元、雙目相機等，此類方法能夠有較單目相機多的絕對距離資訊，有利於得出物體絕對距離，但通常成本相對高昂或不利攜帶。與之相對的是單目相機測距，由於只有平面影像，導致產生尺度不確定性，而解決方法主要有兩種路線：第一種是深度學習預測模型，預測圖片中個別畫素的絕對距離，但模型的泛化能力是一大挑戰；第二種則是結合視覺特徵，並基於先驗知識的假設做推算，在此種方式中，其中一種主要的形式是先透過視覺同步地圖定位建出相對尺度的周遭環境點，接著藉由先驗知識得出的比例因素回推出絕對距離，但視覺特徵的擷取易受到紋理複雜度影響。
    在此前提下，本研究提出一種基於單目魚眼相機之絕對尺度之距離預測系統。首要先以單目魚眼相機，結合視覺同步地圖定位，建立出周遭環境的地圖點。接著以擷取地面點為目標，分別從語意感知與幾何約束出發：前者透過語意分割模型作為圖片地面點的篩選，並結合魚眼模型幾何還原出多視角校正圖片進行輸入；後者則是利用Delaunay三角化對所有畫面中可見環境點做初步關係連接，並使用深度一致性對每組相鄰點作驗證，初步篩選出可能地面點，最後做二次三角化，挑出最可能的地面參考點集。並以地面點與相機的位姿關係與推算出當下相機距離地面高度。最後結合先驗知識來得出相對與絕對距離的尺度關係，用以還原出視野中目標的絕對距離，達成以單目相機測距之需求。

    With the rapid growth of autonomous driving and wearable devices, depth sensing and absolute distance prediction of surrounding has become a core technology related to system safety. Several methods have been proved to achieve depth sensing. One is done by using multi-sensor (e.g. LiDAR, IMU, stereo camera, etc.), this type of method has additional metric distance information compared to monocular camera, which can be leveraged to compute metric distance, but the cost is relatively high or not portable-friendly. The thesis addresses the case of monocular camera and discuss the associated scale ambiguity issue. To solve it, there are usually two ways: one is deep learning model, it can predict per-pixel distance from image, but the generality of the model is a challenge; another is creating surrounding map points by visual SLAM (Simultaneous Mapping And Localization), then inferring metric distance with scale factor from priori information, but visual features is prone to be affected by texture.
    With this insight, this thesis proposes a metric distance prediction system based on monocular fisheye camera. First, it creates map points of surrounding using visual SLAM and monocular fisheye camera. Next is to extract ground point, which is divided in two parallel ways: semantic sensing and geometric constraint. The former filter ground point on image plane through semantic segmentation model, and undistort fisheye image with multiple view as input, and the latter leverages Delaunay Triangulation to set up relation of all visible map point on image plane, and validate each pair of point by distance consistency to extract map point candidates for the first time, performs Delaunay Triangulation again to select most likely map points, and infers camera height point from final ground point and camera position, and recover metric distance of target with scale factor from priori information, achieving the request for metric distance prediction with monocular camera.

    摘要 I Abstract III Acknowledgment V Contents VI List of Tables IX List of Figures X List of Abbreviations XIII Chapter 1. Introduction 1 1-1 Background 1 1-2 Literature Review 3 1-3 Contributions 5 1-4 Organization of Context 6 Chapter 2. System Overview 7 2-1 Main Hypothesis 8 2-2 Visual SLAM 9 Chapter 3. Proposed Method 12 3-1 Ground Point Extraction By Segmentation 12 3-1.1 Segmentation for Ground Region 12 3-1.2 Ground Point Matching 17 3-2 Ground Point Extraction By Geometric Method 18 3-2.1 First Selection Based on Distance Consistency 18 3-2.2 Second Selection based on Plane Checking 23 3-2.3 Final Selection Based on Grouping 25 3-3 Height Computation and Scale Recovery 27 3-3.1 Constant Kalman Filter 28 3-3.2 Rolling model in Unscented Kalman Filter 29 3-4 Distance Computation 31 Chapter 4. Performance Evaluation 34 4-1 Datasets and Settings 34 4-1.1 OpenLORIS-Scene Dataset 34 4-1.2 Hand-held Recorded Dataset 36 4-1.3 Environment and System Flow 37 4-2 Plane Fitting Validation 40 4-2.1 Evaluation Metric 40 4-2.2 Validation 41 4-3 Height Estimation 45 4-3.1 Relative Height 45 4-3.2 Filtering 49 4-4 Distance Prediction Evaluation 52 Chapter 5. Conclusion and Discussion 57 5-1 Conclusion 57 5-2 Future Work 57 Reference 59

    [1] Carlos Campos, Richard Elvira, Juan J. Gómez Rodríguez, José M. M. Montiel, and Juan D. Tardós. Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874–1890, 2021.
    [2] J. Kannala and S.S. Brandt. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1335–1340, 2006.
    [3] OPT Corporation. 360° megapixel camera. http://www.optnagano.co.jp/EN/product/nm_camera.html. (Accessed: 2026-06-25).
    [4] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779–788, 2016.
    [5] Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lv, Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. PaddleOCR 3.0 Technical Report. Technical report, PaddlePaddle, 2025.
    [6] Zilong Huang Zhen Zhao Xiaogang Xu Jiashi Feng Hengshuang Zhao Lihe Yang, Bingyi Kang. "depth anything v2". In NeurIPS 2024, 2024.
    [7] Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun. "depth pro: Sharp monocular metric depth in less than a second". In International Conference on Learning Representations, 2025.
    [8] Tomoyuki Shiozaki and Gamini Dissanayake. Eliminating scale drift in monocular slam using depth from defocus. IEEE Robotics and Automation Letters, 3(1):581–587, 2018.
    [9] Georg Klein and David Murray. Parallel tracking and mapping for small ar workspaces. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 225–234, 2007.
    [10] Andrew J. Davison, Ian D. Reid, Nicholas D. Molton, and Olivier Stasse. MonoSLAM: Real-Time Single Camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6):1052–1067, 2007.
    [11] Dingfu Zhou, Yuchao Dai, and Hongdong Li. Reliable scale estimation and correction for monocular visual odometry. In 2016 IEEE Intelligent Vehicles Symposium (IV), pages 490–495, 2016.
    [12] Sunglok Choi, Jaehyun Park, and Wonpil Yu. Resolving scale ambiguity for monocular visual odometry. In 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pages 604–608, 2013.
    [13] Rui Tian, Yunzhou Zhang, Delong Zhu, Shiwen Liang, Sonya Coleman, and Dermot Kerr. Accurate and robust scale recovery for monocular visual odometry based on plane geometry. CoRR, abs/2101.05995, 2021.
    [14] Johannes Gräter, Tobias Schwarze, and Martin Lauer. Robust scale estimation for monocular visual odometry using structure from motion and vanishing points. In 2015 IEEE Intelligent Vehicles Symposium (IV), pages 475–480, 2015.
    [15] Hui Zhang, Xiangwei Wang, Xiaochuan Yin, Mingxiao Du, Chengju Liu, and Qijun Chen. Geometry-constrained scale estimation for monocular visual odometry. IEEE Transactions on Multimedia, 24:3144–3156, 2022.
    [16] Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. ORB: An efficient alternative to SIFT or SURF. In 2011 International Conference on Computer Vision, pages 2564–2571, 2011.
    [17] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision – ECCV 2018, pages 334–349, Cham, 2018. Springer International Publishing.
    [18] Jonathan Richard Shewchuk. Triangle: Engineering a 2d quality mesh generator and delaunay triangulator. In Selected Papers from the Workshop on Applied Computational Geormetry, Towards Geometric Engineering, FCRC ’96/WACG ’96, page 203–222, Berlin, Heidelberg, 1996. Springer-Verlag.
    [19] R. E. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1):35–45, 03 1960.
    [20] Yash Vyas, Mike Allenspach, Christian Lanegger, Rol Siegwart, and Marco Tognon. Modelling and estimation of human walking gait for physical human-robot interaction. In 2021 Aerial Robotic Systems Physically Interacting with the Environment (AIRPHARO), pages 1–4, 2021.
    [21] E.A. Wan and R. Van Der Merwe. The unscented Kalman filter for nonlinear estimation. In Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373), pages 153–158, 2000.
    [22] Xuesong Shi, Dongjiang Li, Pengpeng Zhao, Qinbin Tian, Yuxin Tian, Qiwei Long, Chunhao Zhu, Jingwei Song, Fei Qiao, Le Song, Yangquan Guo, Zhigang Wang, Yimin Zhang, Baoxing Qin, Wei Yang, Fangshi Wang, Rosa H. M. Chan, and Qi She. Are we ready for service robots? the OpenLORIS-Scene datasets for lifelong SLAM. In 2020 International Conference on Robotics and Automation (ICRA), pages 3139–3145, 2020.
    [23] Intel Corporation. Intel® realsense™ depth camera d435i. https://www.intel.com/content/www/us/en/products/sku/190004/intel-realsense-depth-camera-d435i/specifications.html. (Accessed: 2026-06-25).
    [24] Intel Corporation. Intel® realsense™ tracking camera t265. https://www.intel.com/content/www/us/en/products/sku/192742/intel-realsense-tracking-camera-t265/specifications.html. (Accessed: 2026-06-25).
    [25] Guoping Wen Sergey Dorodnicov, Anders Grunnet-Jepsen. Projection, texture-mapping and occlusion with realsense™ depth cameras. https://dev.realsenseai.com/docs/projection-texture-mapping-and-occlusion-with-intel-realsense-depth-cameras/. (Accessed: 2026-06-25).
    [26] Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot operating system 2: Design, architecture, and uses in the wild. Science Robotics, 7(66):eabm6074, 2022.
    [27] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
    [28] Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision, 127(3):302–321, 2019.
    [29] Nihui. ncnn: A high-performance neural network inference framework optimized for the mobile platform. https://github.com/Tencent/NCNN. (Accessed: 2026-06-25).
    [30] G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.
    [31] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 10892–10902, 2021.

    QR CODE