| 研究生: |
林緯程 Lin, Wei-Cheng |
|---|---|
| 論文名稱: |
以三維特徵地圖輔助相機定位之適應性與精度分析 Feasibility and Accuracy Analysis of Camera Localization Aided by a 3D Feature Map |
| 指導教授: |
饒見有
Rau, Jiann-Yeou |
| 學位類別: |
碩士 Master |
| 系所名稱: |
工學院 - 測量及空間資訊學系 Department of Geomatics |
| 論文出版年: | 2022 |
| 畢業學年度: | 110 |
| 語文別: | 中文 |
| 論文頁數: | 133 |
| 中文關鍵詞: | ORB-SLAM 、三維特徵地圖 、坐標轉換 |
| 外文關鍵詞: | ORB-SLAM, 3D Feature Map, Coordinate Transformation |
| 相關次數: | 點閱:106 下載:28 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
SLAM (Simultaneous Localization And Mapping)做為一個在未知環境同時進行感測器定位以及環境地圖建立的概念,常被用以處理無可靠GNSS訊號環境之定位問題。基於影像特徵點之視覺(Visual) SLAM(V-SLAM),透過重複的三維特徵地圖建立、影像特徵至三維特徵地圖的匹配和空間後方交會的過程來達成相機定位目的。倘若特徵點法V-SLAM建立之三維特徵地圖,能做為場景先驗控制重複使用,則無可靠GNSS訊號環境之定位問題即可獲解決。然而基於影像灰度值計算的影像特徵點萃取技術,其特徵萃取和匹配結果常受環境光照條件影響。因此對於不同光照條件下所建立之三維特徵地圖,其光照條件的改變對相機定位之適應性值得被進一步探討。
得益於ORB-SLAM中的地圖再利用功能,本研究選擇其做為測試用之V-SLAM 系統。首先,本研究透過單眼相機,分別於不同光照條件下,以手持錄影的方式環繞位於戶外之測試場域,拍攝場景之影片。再將拍攝之影片,輸入ORB-SLAM中進行處理,建立不同光照條件下的三維特徵地圖。最後,再將前述步驟建立之三維特徵地圖輸入ORB-SLAM,以純定位模式(Localization Mode)對不同光照條件下拍攝之影片進行交叉定位測試,而輸出之相機外方位參數將和以SfM (Structure from Motion)方法建立之參考相機軌跡比較,進行精度分析。然而本研究所使用之相機為單眼相機,使輸出成果缺乏尺度資訊,又ORB-SLAM之輸出為稀疏點雲,難以精確標記控制點進行座標轉換。為此,本研究基於混合模式平差及三維正形轉換的概念,提出藉由控制點以及相機軌跡進行坐標轉換的兩種坐標轉換模式,所提出之轉換結果,亦將和SLAM領域中常用的Umeyama’s Method進行比較。
本研究之實驗結果顯示,以ORB-SLAM建立之三維特徵地圖輔助相機定位,在相機定位部分,其定位精度主要決定於建立該特徵地圖之ORB-SLAM處理成果之精度,和光照條件的改變並無明顯關聯;而旋轉角部分,ORB-SLAM亦可以獲得穩定的成果,然而其回復之旋轉角正確程度,則取決於後方交會時特徵點之分布涵蓋影像之面積比例。
SLAM(Simultaneous Localization and Mapping), as a concept of localizing the sensors and mapping the scenes concurrently, is often used to localize the sensors in the GNSS-denied environment. One of the most common SLAM systems is the Visual SLAM (V-SLAM) system based on optical sensors. The feature-based V-SLAM, which uses cameras as the sensor, extracts and matches feature points on imported images and generates a 3D feature map simultaneously during the localization processes. By reusing this 3D feature map, it can be considered as a control field to solve the GNSS-denied localization problem. This study demonstrates the effect of different lighting conditions on the camera localization aided by a 3D feature map and proposes a strategy for evaluating localization accuracy of V-SLAM.
As the test V-SLAM system, ORB-SLAM system is applied. First, we capture the videos of the scene under different lighting conditions by using a monocular video camera. Second, the captured videos are imported into ORB-SLAM system to establish the 3D feature maps. Finally, the 3D feature maps generated from the previous step and the videos from different lighting conditions are imported into ORB-SLAM Localization Mode together to cross test the feasibility of camera localization. However, since we adopt a monocular video camera for capturing several image sequences, the output trajectories are lack of the scale information. To cope with this problem, we propose two coordinate transformation methods based on 3D conformal transformation, which are based on the camera trajectories and the control points triangulated by the EOPs from ORB-SLAM. The experimental results show that different lighting conditions do not significantly affect the localization accuracy of ORB-SLAM aided by 3D feature maps, while the geometry between the cameras and the scene is more critical. With proper video capturing strategy, the 3D feature map might be a potential solution of the GNSS-denied localization problem.
Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S.M., Szeliski, R., 2011. Building rome in a day. Communications of the ACM 54, 105-112.
Alcantarilla, P.F., Bartoli, A., Davison, A.J., 2012. KAZE features, European conference on computer vision. Springer, pp. 214-227.
Andrew, A.M., 2001. Multiple view geometry in computer vision. Kybernetes.
Arun, K.S., Huang, T.S., Blostein, S.D., 1987. Least-squares fitting of two 3-D point sets. IEEE Transactions on pattern analysis and machine intelligence, 698-700.
Bay, H., Tuytelaars, T., Van Gool, L., 2006. Surf: Speeded up robust features, European conference on computer vision. Springer, pp. 404-417.
Bianco, S., Ciocca, G., Marelli, D., 2018. Evaluating the performance of structure from motion pipelines. Journal of Imaging 4, 98.
Bonali, F.L., Tibaldi, A., Marchese, F., Fallati, L., Russo, E., Corselli, C., Savini, A., 2019. UAV-based surveying in volcano-tectonics: An example from the Iceland rift. Journal of Structural Geology 121, 46-64.
Brown, D.C., 1968. Advanced methods for the calibration of metric cameras. DBA SYSTEMS INC MELBOURNE FL.
Calonder, M., Lepetit, V., Strecha, C., Fua, P., 2010. Brief: Binary robust independent elementary features, European conference on computer vision. Springer, pp. 778-792.
Carmen Pérez, M., Gualda, D., Vicente-Ranera, J., Villadangos, J.M., Ureña, J., 2019. Review of UAV positioning in indoor environments and new proposal based on US measurements, IPIN (Short Papers/Work-in-Progress Papers), pp. 267-274.
Caselitz, T., Steder, B., Ruhnke, M., Burgard, W., 2016. Monocular camera localization in 3d lidar maps, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 1926-1931.
Chan, B., Guan, H., Jo, J., Blumenstein, M., 2015. Towards UAV-based bridge inspection systems: a review and an application perspective. Structural Monitoring and Maintenance 2, 283-300.
Chiabrando, F., Nex, F., Piatti, D., Rinaudo, F., 2011. UAV and RPV systems for photogrammetric surveys in archaelogical areas: two tests in the Piedmont region (Italy). Journal of Archaeological Science 38, 697-710.
Chien, H.-J., Chuang, C.-C., Chen, C.-Y., Klette, R., 2016. When to use what feature? SIFT, SURF, ORB, or A-KAZE features for monocular visual odometry, 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ). IEEE, pp. 1-6.
Colomina, I., Tecnologia, P.M., 2008. Towards a new paradigm for high-resolution low-cost photogrammetryand remote sensing, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, ISPRS Congress, Beijing, China, XXXVII. Part B, pp. 1201-1206.
Conde, B., Ramos, L.F., Oliveira, D.V., Riveiro, B., Solla, M., 2017. Structural assessment of masonry arch bridges by combination of non-destructive testing techniques and three-dimensional numerical modelling: Application to Vilanova bridge. Engineering Structures 148, 621-638.
Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O., 2007. MonoSLAM: real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell 29, 1052-1067.
Dorafshan, S., Maguire, M., 2018. Bridge inspection: human performance, unmanned aerial systems and automation. Journal of Civil Structural Health Monitoring 8, 443-476.
Endres, F., Hess, J., Sturm, J., Cremers, D., Burgard, W., 2013. 3-D mapping with an RGB-D camera. IEEE transactions on robotics 30, 177-187.
Engel, J., Koltun, V., Cremers, D., 2018. Direct Sparse Odometry. IEEE Trans Pattern Anal Mach Intell 40, 611-625.
Engel, J., Schöps, T., Cremers, D., 2014. LSD-SLAM: Large-scale direct monocular SLAM, European conference on computer vision. Springer, pp. 834-849.
Fiala, M., Shu, C., 2005. Fully automatic camera calibration using self-identifying calibration targets. Techn. Rep 48306, 26.
Forster, C., Zhang, Z., Gassner, M., Werlberger, M., Scaramuzza, D., 2017. SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems. IEEE Transactions on Robotics 33, 249-265.
Fuentes-Pacheco, J., Ruiz-Ascencio, J., Rendón-Mancha, J.M., 2015. Visual simultaneous localization and mapping: a survey. Artificial intelligence review 43, 55-81.
Geiger, A., Lenz, P., Stiller, C., Urtasun, R., 2013. Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research 32, 1231-1237.
Hedborg, J., Forssén, P.-E., Felsberg, M., Ringaby, E., 2012. Rolling shutter bundle adjustment, 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp. 1434-1441.
Hedborg, J., Ringaby, E., Forssén, P.-E., Felsberg, M., 2011. Structure and motion estimation from rolling shutter video, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE, pp. 17-23.
Horn, B.K., 1987. Closed-form solution of absolute orientation using unit quaternions. Josa a 4, 629-642.
Iglhaut, J., Cabo, C., Puliti, S., Piermattei, L., O’Connor, J., Rosette, J., 2019. Structure from Motion Photogrammetry in Forestry: a Review. Current Forestry Reports 5, 155-168.
Jiang, S., Jiang, C., Jiang, W., 2020. Efficient structure from motion for large-scale UAV images: A review and a comparison of SfM tools. ISPRS Journal of Photogrammetry and Remote Sensing 167, 230-251.
Jung, S., Choi, D., Song, S., Myung, H., 2020. Bridge inspection using unmanned aerial vehicle based on HG-SLAM: Hierarchical graph-based SLAM. Remote Sensing 12, 3022.
Kerl, C., Sturm, J., Cremers, D., 2013. Robust odometry estimation for RGB-D cameras, 2013 IEEE international conference on robotics and automation. IEEE, pp. 3748-3754.
Kim, Y., Jeong, J., Kim, A., 2018. Stereo camera localization in 3d lidar maps, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 1-9.
Klein, G., Murray, D., 2007. Parallel tracking and mapping for small AR workspaces, 2007 6th IEEE and ACM international symposium on mixed and augmented reality. IEEE, pp. 225-234.
Koch, C., Paal, S.G., Rashidi, A., Zhu, Z., König, M., Brilakis, I., 2014. Achievements and challenges in machine vision-based inspection of large concrete structures. Advances in Structural Engineering 17, 303-318.
Lee, S., Kalos, N., Shin, D.H., 2014. Non-destructive testing methods in the U.S. for bridge inspection and maintenance. KSCE Journal of Civil Engineering 18, 1322-1331.
Lowe, D.G., 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60, 91-110.
Luhmann, T., Robson, S., Kyle, S., Boehm, J., 2013. Close-Range Photogrammetry and 3D Imaging. Walter de Gruyter.
Mur-Artal, R., Montiel, J.M.M., Tardos, J.D., 2015. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE transactions on robotics 31, 1147-1163.
Mur-Artal, R., Tardos, J.D., 2017. ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras. IEEE Transactions on Robotics 33, 1255-1262.
Nobis, F., Papanikolaou, O., Betz, J., Lienkamp, M., 2020. Persistent Map Saving for Visual Localization for Autonomous Vehicles: An ORB-SLAM 2 Extension, 2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER). IEEE, pp. 1-9.
Remondino, F., Fraser, C., 2006. Digital camera calibration methods: considerations and comparisons. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 36, 266-272.
Rosten, E., Drummond, T., 2006. Machine Learning for High-Speed Corner Detection, Computer Vision – ECCV 2006, pp. 430-443.
Rosten, E., Porter, R., Drummond, T., 2008. Faster and better: A machine learning approach to corner detection. IEEE transactions on pattern analysis and machine intelligence 32, 105-119.
Rublee, E., Rabaud, V., Konolige, K., Bradski, G., 2011. ORB: An efficient alternative to SIFT or SURF, 2011 International conference on computer vision. Ieee, pp. 2564-2571.
Seshu, D.R., Murthy, N.R.D., 2013. Non Destructive Testing of Bridge Pier - A Case Study. Procedia Engineering 54, 564-572.
Shi, T., Shen, S., Gao, X., Zhu, L., 2019. Visual localization using sparse semantic 3D map, 2019 IEEE international conference on image processing (ICIP). IEEE, pp. 315-319.
Snavely, N., Seitz, S.M., Szeliski, R., 2007. Modeling the World from Internet Photo Collections. International Journal of Computer Vision 80, 189-210.
Strasdat, H., Montiel, J., Davison, A.J., 2010. Scale drift-aware large scale monocular SLAM. Robotics: Science and Systems VI 2, 7.
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D., 2012. A benchmark for the evaluation of RGB-D SLAM systems, 2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp. 573-580.
Sun, C.-Z., Zhang, B., Wang, J.-K., Zhang, C.-S., 2021. A Review of Visual SLAM Based on Unmanned Systems, 2021 2nd International Conference on Artificial Intelligence and Education (ICAIE). IEEE, pp. 226-234.
Surber, J., Teixeira, L., Chli, M., 2017. Robust visual-inertial localization with weak GPS priors for repetitive UAV flights, 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 6300-6306.
Taheri, H., Xia, Z.C., 2021. SLAM; definition and evolution. Engineering Applications of Artificial Intelligence 97.
Tang, R., Fritsch, D., 2013. Correlation Analysis of Camera Self-Calibration in Close Range Photogrammetry. The Photogrammetric Record 28, 86-95.
Umeyama, S., 1991. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence 13, 376-380.
Wolcott, R.W., Eustice, R.M., 2014. Visual localization within lidar maps for automated urban driving, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 176-183.
Yang, N., Wang, R., Gao, X., Cremers, D., 2018. Challenges in Monocular Visual Odometry: Photometric Calibration, Motion Bias, and Rolling Shutter Effect. IEEE Robotics and Automation Letters 3, 2878-2885.
Yoo, Y., Im, J., Paik, J., 2015. Low-light image enhancement using adaptive digital pixel binning. Sensors 15, 14917-14931.
Zhou, H., Ni, K., Zhou, Q., Zhang, T., 2016. An SfM Algorithm With Good Convergence That Addresses Outliers for Realizing Mono-SLAM. IEEE Transactions on Industrial Informatics 12, 515-523.
尤瑞哲, 2019. 基礎測量平差法. 滄海書局, 台中市.
王仲宇, 張國鎮, 張荻薇, 黃俊鴻, 姚乃嘉, 陳賜賢, 123, 123, 123, 123, 123, 123, 123, 123, 巫柏蕙, 2012. 橋梁政策白皮書之研擬(MOTC-IOT-98-EEB012). 交通部運輸研究所, 台北市.
吳明諺, 2016. 以無人飛行載具拍攝影像建置三維橋梁模型之初步研究, 營建管理研究所. 國立中央大學, 桃園縣, p. 150.
呂岳霖, 2021. 應用全卷積神經網路及Deeplab V3+於橋梁劣化及影像模糊區之偵測, 測量及空間資訊學系. 國立成功大學, 台南市, p. 96.
张剑清, 胡安文, 2007. 多基线摄影测量前方交会方法及精度分析.
施旻汶, 2015. 自動化無人飛行載具輔助擷取橋梁影像之研究, 營建管理研究所. 國立中央大學, 桃園縣, p. 119.
孫浩庭, 2020. 無人飛行載具於橋梁檢測之研究-以黎霧橋及蘭陽大橋為例, 土木工程學系碩士班. 國立宜蘭大學, 宜蘭縣, p. 69.
曾志煌, 許書耕, 巫柏蕙, 姚乃嘉, 陳明正, 葉啟章, 蔡欣局, 廖先格, 2011. 「橋梁目視檢測評估手冊(草案)」 之研擬(MOTC-IOT-99-EEB006). 交通部運輸研究所, 台北市.
蕭凱文, 饒見有, 王瑞麟, 2017. 利用多軸無人機影像萃取橋梁劣化區三維空間資訊, 航測及遙測學刊, pp. 205-211.