簡易檢索 / 詳目顯示

回結果列表
研究生： 詹鈞評
Jhan, Jyun-Ping
論文名稱： 無人機多鏡頭相機系統之穩健自適應波段套合與影像拼接法
Robust and Adaptive Band Co-Registration and Image Stitching Methods of UAS Multi-lens Camera System
指導教授： 饒見有
Rau, Jiann-Yeou
學位類別： 博士
Doctor
系所名稱： 工學院 - 測量及空間資訊學系
Department of Geomatics
論文出版年： 2017
畢業學年度： 105
語文別： 英文
論文頁數： 90
中文關鍵詞： 多鏡頭相機系統 、波段套合 、影像拼接 、無人飛行系統
外文關鍵詞： Multi-lens Camera System, Band Co-registration, Image Stitching, UAS
相關次數： 點閱：124  下載：17 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 應用無人飛行系統(Unmanned Aerial System, UAS)搭載多鏡頭相機系統(Multi-lens camera system)具有高機動性與便利性之優點，適合蒐集航空影像以供攝影測量與遙感探測應用之需求。多相機系統採用多感器幾何架構蒐集多視角之影像與多光譜資訊，適用於三維建模、大範圍製圖與植生調查等應用。根據系統架構之設計，多鏡頭相機系統可以分為三大類別，包含能同時蒐集垂直與傾斜影像進行三維仿真數碼城市重建之三維建模相機系統(3D Mapping Camera, 3DMC)，利用相機陣列擴展整體拍攝視場角(Field of View, FOV)與提升單張影像解析度之大像幅相機系統(Large-format Mapping Camera, LFMC)，以及利用每個鏡頭搭載不同濾鏡以獲取不同波譜資訊之多光譜相機系統(Multispectral Camera, MSC)。由於多鏡頭相機系統之鏡頭觀測視角、透鏡畸變與焦距長之不一致現象，因此需要透過影像套合(Image Registration)技術將多感測器幾何轉換成單一感測器幾何以供後續的影像處理與分析需求。
    對於多光譜相機而言，雖然每個鏡頭之成像平面彼此接近平行且透視中心也盡可能接近，但仍因不同的透鏡畸變差、些微的視角與透視中心之差異，使得原始獲得的多光譜影像存在顯著的波段錯位現象。為了符合高精度的遙感探測需求，需要利用影像套合技術進行原影像的波段套合(Band Co-registration)以獲取精確的地物波譜資訊。同樣的在大像幅相機系統中，其透過不同視角之鏡頭以提升拍攝影像之整體視場角，因此需要透過影像套合進行多影像拼接(Image Stitching)，以便將多張小像幅的影像拼接成單張大像幅的影像，如此便能擁有大面積調查能力、減少飛行航帶數與提升影像處理效率之優點。至於三維建模相機系統，因其目的在於利用垂直與傾斜視角獲取建物的屋頂與牆面資訊進行仿真數碼城市的重建，且因影像間幾乎沒有重疊區，故無影像套合之需求。
    由於多光譜相機系統與大像幅相機系統皆需進行影像套合前處理，以便將多感測幾何轉換至單一感測器幾何以符合攝影測量與遙感探測之後續應用。據此，本研究提出一穩健自適應波段套合與影像拼接法(Robust and Adaptive Band-to-Band Image Transform, RABBIT)，可以符合各式多光譜相機的波段套合以及大像幅相機的影像拼接需求。該方法透過修改的透視投影轉換(Modified Projective Transformation, MPT)以將多感測器幾何轉換為單一感測器幾何。轉換過程中所需的各項係數主要透過相機系統率定推導求得，包含每個鏡頭的內方位參數(Interior Orientation Parameters, IOPs)，以及其他影像相對於某一參考影像之相對方位參數(Relative Orientation Parameters, ROPs)。據此，便能透過修改的透視投影轉換修正透鏡畸變差、不同鏡頭之焦距差與觀測視角差異，並將所有影像投影至相同之參考面進行套合。然而，由於內方位與相對方位參數率定之不確定性，導致影像經修改的透視投影轉換後仍存在些許的系統性誤差現象。為了獲得高精確的波段套合與影像拼接成果，本研究針對各項修改的透視投影轉換所使用的參數進行模擬，以了解轉換後的系統性誤差分布，進而推導出穩健與自適應改正(Robust and Adaptive Correction, RAC)方程式，能修正轉換後的系統性誤差並獲得優於次像元的套合精度。
    在本研究中，三種不同類型的多光譜相機(包含Tetracam Miniature Multiple Camera Array (MiniMCA)、Micasense Rededge和Parrot Sequoia)，以及一組自製之大像幅相機系統被用來評估RABBIT波段套合與影像拼接的可行性與精確度。本研究一共採用六組多光譜資料，由不同相機在不同時間、地點與拍攝距離獲得，並用以進行波段套合以評估RABBIT的精確度、可靠性和應用性。從多光譜相機系統的波段套合成果來看，各組資料皆能獲得優於次像元的套合精度，且套合後的影像亦能用以高效率產製正射影像以供後續的遙測分析。另一方面，RABBIT在多鏡頭相機系統之影像拼接中亦能獲得相似成果，其拼接後影像之精度分析亦經由傳統之空三平差、立體製圖分析與數值地形模型的產製進行比較，並由成果證明拼接後之大像幅影像能符合大範圍面積製圖之需求。

    Utilizing multi-lens camera systems by mounting them on an Unmanned Aerial System (UAS) has the benefits of convenience and flexibility for collecting aerial imagery for photogrammetry and remote sensing applications. A multi-lens camera system adopts multiple image geometry for collecting multi-view imagery and multispectral information that is suitable for three-dimensional modeling, large area mapping, and vegetation investigation. According to the structure design, multi-lens camera systems can be categorized into the three-dimensional modeling camera (3DMC), large-format mapping camera (LFMC), and multispectral camera (MSC). However, due to the inconsistence of viewing angles, focal length difference, and lens distortion effects among lenses, it requires performing an image registration technique in order to transform them into one sensor geometry for subsequent image analysis.
    For an MSC, several lenses with different filters are mounted parallel to and as close as possible to each other to acquire a multispectral cube. However, the slight differences in lens distortion, mounting positions, and viewing angles among lenses mean that the acquired original multispectral images have significant band misregistration errors, which requires image registration to perform band co-registration for obtaining accurate spectral response. An LFMC adopts several lenses with different viewing angles to expand the total field of view (FOV) in a single shot. In order to obtain one sensor geometry, several small-format images are stitched into a large-format image through image registration, which thereby has the ability of large area investigation and large FOV. As for a 3DMC, since the purpose of multi-lens structure adoption is to acquire both nadir and oblique images for building façade acquisition and the overlap ratio is very small, no image registration is necessary.
    In this study, we have developed a robust and adaptive band-to-band image transform (RABBIT) method for dealing with the band co-registration issues of various types of MSCs, and that can be applied for the image stitching purpose of a LFMC. The RABBIT utilizes modified projective transformation (MPT) to transfer the multiple image geometry of a multi-lens imaging system to one sensor geometry. As a result, the difference of lens distortion, focal length, and viewing angle effects are eliminated by projecting them into same image frame. All the necessary coefficients of MPT are derived from a camera system calibration procedure, including the interior orientation parameters (IOPs) of each lens and the relative orientation parameters (ROPs) among one selected reference imaging sensor and the others. However, due to the uncertainty of calibration results, some systematic errors persist after MPT. In order to understand those systematic effects, we simulated several factors of the camera calibration uncertainty and proposed a robust and adaptive correction (RAC) procedure to correct several systematic errors after MPT to obtain band co-registration and image stitching accuracy better than 0.5 pixels.
    Three state-of-the-art MSCs (Tetracam Miniature Multiple Camera Array (MiniMCA), Micasense Rededge, and Parrot Sequoia) and one customized LFMC are applied to evaluate the performance of the proposed method. Six multispectral datasets acquired at different target distances, dates, and locations are adopted to evaluate the accuracy and performance of RABBIT. The results from MSCs show that 0.5 pixels accuracy of band co-registration can be achieved, and the obtained co-registered images are available for generating ortho-images with high efficiency for subsequent remote sensing analysis. On the other hand, results also show that RABBIT can work on image stitching of LFMC and has similar performance. The accuracy assessment of stitched images are conducted through conventional aerial triangulation, stereo-plotting, and digital surface model (DSM) generation procedures, and it proves that the stitched images are feasible for large area mapping applications.

    摘要 I ABSTRACT III 致謝 V ACKNOWLEDGEMENTS VI CONTENTS VII LIST OF TABLES X LIST OF FIGURES XII LIST OF ACRONYMS XVI CHAPTER 1: INTRODUCTION 1 1.1. Research Background 1 1.2. Motivation and Objectives 4 1.3. Research Methods and Experiments 5 1.4. Dissertation Structure 6 CHAPTER 2: LITERATURE REVIEW 8 2.1. Multi-Lens Camera System 8 2.1.1. 3D Modeling Cameras (3DMCs) 8 2.1.2. Large-Format Mapping Cameras (LFMCs) 10 2.1.3. Multispectral Cameras (MSCs) 12 2.1.4. Summary 18 2.2. Image Registration 19 2.2.1. Band Co-registration of MSCs 21 2.2.2. Image Stitching of LFMCs 27 2.2.3. Summary 28 CHAPTER 3: RESEARCH MATERIAL 31 3.1. Datasets for Band Co-registration 31 3.1.1. Band Misregistration Effects 33 3.1.2. Examination Conditions 34 3.2. Dataset for Image Stitching 36 3.2.1. Customized LFMC 36 3.2.2. UAS 38 3.2.3. Study Site 39 CHAPTER 4: METHODOLOGY 41 4.1. RABBIT 41 4.2. Camera System Calibration 42 4.2.1. Calibration Approaches 42 4.2.2. Interior Orientation Parameters (IOPs) 44 4.2.3. Relative Orientation Parameters (ROPs) 45 4.3. Modified Projective Transformation (MPT) 46 4.4. Robust and Adaptive Correction (RAC) 47 4.4.1. Simulation of MPT Error Distributions 48 4.4.2. Mathematical forms of RAC 51 4.5. Sequential Image Analysis (SIA) 52 4.5.1. Image Matching Sequence 53 4.5.2. Practical Examples of RAC with SIA 53 4.6. Batch Image Processing 55 CHAPTER 5: RESULTS AND ANALYSIS 57 5.1. Band Co-registration for MSCs 57 5.1.1. Results of Camera System Calibration 59 5.1.2. Results of Band Co-registration 62 5.1.3. Accuracy Assessment 63 5.1.4. Reliability Evaluations 67 5.1.5. Close-range Applications 68 5.1.6. Results of Batch Image Processing 69 5.1.7. Summary 69 5.2. Image Stitching for Customized LFMC 70 5.2.1. Results of Camera System Calibration 71 5.2.2. Internal accuracy of Image Stitching 74 5.2.3. Aerial Triangulation Accuracy Assessment 75 5.2.4. Stereo-Plotting Accuracy Assessment 76 5.2.5. DSM Comparison 77 5.2.6. Summary 80 CHAPTER 6: CONCLUSIONS AND FUTURE WORK 82 REFERENCES 84

    1. Agapiou, A., Hadjimitsis, D., Alexakis, D., 2012. Evaluation of broadband and narrowband vegetation indices for the identification of archaeological crop marks. Remote Sensing 4, 3892.
    2. Ahmadabadian, A.H., Robson, S., Boehm, J., Shortis, M., Wenzel, K., Fritsch, D., 2013. A comparison of dense matching algorithms for scaled surface reconstruction using stereo camera rigs. ISPRS Journal of Photogrammetry and Remote Sensing 78, 157-167.
    3. Baluja, J., Diago, M.P., Balda, P., Zorer, R., Meggio, F., Morales, F., Tardaguila, J., 2012. Assessment of vineyard water status variability by thermal and multispectral imagery using an unmanned aerial vehicle (UAV). Irrigation Science 30, 511-522.
    4. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L., 2008. Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding 110, 346-359.
    5. Brown, L.G., 1992. A survey of image registration techniques. ACM Comput. Surv. 24, 325-376.
    6. Chiabrando, F., Nex, F., Piatti, D., Rinaudo, F., 2011. UAV and RPV systems for photogrammetric surveys in archaelogical areas: two tests in the Piedmont region (Italy). Journal of Archaeological Science 38, 697-710.
    7. Cho, W., Schenk, T., 1992. Resampling Digital Imagery to Epipolar Geometry, IAPRS International Archives of Photogrammetry and Remote Sensing, pp. 404-408.
    8. Colomina, I., Molina, P., 2014. Unmanned aerial systems for photogrammetry and remote sensing: a review. ISPRS Journal of Photogrammetry and Remote Sensing 92, 79-97.
    9. Dörstel, C., 2003. DMC-Practical Experiences and Photogrammetric System Performance, in: Fritsch D.(Ed.), Photogrammetric Week 2003. Citeseer.
    10. Eisenbeiß, H., 2009. UAV photogrammetry. Doctoral Dissertation. University of Technology Dresden.
    11. Fraser, C.S., 1997. Digital Camera Self-Calibration. ISPRS Journal of Photogrammetry and Remote Sensing 52, 149-159.
    12. Fraser, C.S., Edmundson, K.L., 2000. Design and implementation of a computational processing system for off-line digital close-range photogrammetry. ISPRS Journal of Photogrammetry and Remote Sensing 55, 94-104.
    13. Furukawa, Y., Ponce, J., 2010. Accurate, dense, and robust multiview stereopsis. IEEE Transactions On Pattern Analysis And Machine Intelligence 32, 1362-1376.
    14. Garcia-Ruiz, F., Sankaran, S., Maja, J.M., Lee, W.S., Rasmussen, J., Ehsani, R., 2013. Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees. Computers and Electronics in Agriculture 91, 106-115.
    15. Giordan, D., Manconi, A., Tannant, D.D., Allasia, P., 2015. UAV: Low-cost remote sensing for high-resolution investigation of landslides, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 5344-5347.
    16. Gruber, M., Ladstädter, R., 2011. Results from Ultracam Monolithic Stitching, ASPRS Annual Conference, Milwaukee, WI.
    17. Gruen, A., 1985. Adaptive least squares correlation: a powerful image matching technique. South African Journal of Photogrammetry, Remote Sensing and Cartography 14, 175-187.
    18. Haala, N., 2013. The landscape of dense image matching algorithms, in: Fritsch, D. (Ed.), Photogrammetric Week '13. Wichmann, Berlin/Offenbach, pp. 271-284.
    19. Hiep, V.H., Keriven, R., Labatut, P., Pons, J.P., 2009. Towards high-resolution large-scale multi-view stereo, 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, pp. 1430 - 1437.
    20. Holtkamp, D.J., Goshtasby, A.A., 2009. Precision registration and mosaicking of multicamera images. IEEE Transactions on Geoscience and Remote Sensing 47, 3446-3455.
    21. Huang, Y., Thomson, S.J., Lan, Y., Maas, S.J., 2010. Multispectral imaging systems for airborne remote sensing to support agricultural production management. International Journal of Agricultural and Biological Engineering 3, 50-62.
    22. Jhan, J.-P., Rau, J.-Y., Huang, C.-Y., 2016. Band-to-band registration and ortho-rectification of multilens/multispectral imagery: a case study of MiniMCA-12 acquired by a fixed-wing UAS. ISPRS Journal of Photogrammetry and Remote Sensing 114, 66-77.
    23. Jhan, J., Li, Y., Rau, J., 2015. A Modified Projective Transformation Scheme for Mosaicking Multi-Camera Imaging System Equipped on a Large Payload Fixed-Wing UAS. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 40, 87.
    24. Kelcey, J., Lucieer, A., 2012. Sensor Correction of a 6-band multispectral imaging sensor for UAV remote sensing. Remote Sensing 4, 1462-1493.
    25. Kim, J., Lee, S., Ahn, H., Seo, D., Park, S., Choi, C., 2013. Feasibility of employing a smartphone as the payload in a photogrammetric UAV system. ISPRS Journal of Photogrammetry and Remote Sensing 79, 1-18.
    26. Ladstädter, R., Gruber, M., Wiechert, A., 2010. Monolithic Stitching: One Sensor Geometry for Multiple Sensor Cameras, ASPRS Annual Conference, San Diego, CA.
    27. Laliberte, A.S., Goforth, M.A., Steele, C.M., Rango, A., 2011. Multispectral remote sensing from unmanned aircraft: image processing workflows and applications for rangeland environments. Remote Sensing 3, 2529-2551.
    28. Lewis, J.P., 1995. Fast template matching, Vision interface, pp. 15-19.
    29. Li, H., Zhang, A., Hu, S., 2015. A multispectral image creating method for a new airborne four-camera system with different bandpass filters. Sensors 15, 17453.
    30. Lowe, D., 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60, 91-110.
    31. Micheletti, N., Chandler, J.H., Lane, S.N., 2015. Structure from motion (SFM) photogrammetry.
    32. Nex, F., Remondino, F., 2014. UAV for 3D mapping applications: a review. Applied Geomatics 6, 1-15.
    33. Niethammer, U., James, M.R., Rothmund, S., Travelletti, J., Joswig, M., 2012. UAV-based remote sensing of the Super-Sauze landslide: Evaluation and results. Engineering Geology 128, 2-11.
    34. Nijland, W., de Jong, R., de Jong, S.M., Wulder, M.A., Bater, C.W., Coops, N.C., 2014. Monitoring plant condition and phenology using infrared sensitive consumer grade digital cameras. Agricultural and Forest Meteorology 184, 98-106.
    35. Pajares, G., 2015. Overview and Current Status of Remote Sensing Applications Based on Unmanned Aerial Vehicles (UAVs). Photogrammetric Engineering & Remote Sensing 81, 281-329.
    36. Petrie, G., Walker, A.S., 2007. Airborne digital imaging technology: a new overview. The Photogrammetric Record 22, 203-225.
    37. Rau, J.-Y., Jhan, J.-P., Huang, C., 2015a. Ortho-Rectification of Narrow Band Multi-Spectral Imagery Assisted by DSLR RGB Imagery Acquired by a Fixed-Wing Uas. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 40, 67.
    38. Rau, J.-Y., Yeh, P.-C., 2012. A Semi-Automatic Image-Based Close Range 3D Modeling Pipeline Using a Multi-Camera Configuration. Sensors 12, 11271.
    39. Rau, J.Y., Jhan, J.P., Hsu, Y.C., 2015b. Analysis of oblique Aerial Images for Land Cover and Point Cloud Classification in an Urban Environment. IEEE Transactions on Geoscience and Remote Sensing 53, 1304-1319.
    40. Rau, J.Y., Jhan, J.P., Li, Y.T., 2016. Development of a large-format UAS imaging system with the construction of a one sensor geometry from a multicamera array. IEEE Transactions on Geoscience and Remote Sensing, 1-10.
    41. Rau, J.Y., Jhan, J.P., Rau, R.J., 2014. Semiautomatic Object-Oriented Landslide Recognition Scheme From Multisensor Optical Imagery and DEM. IEEE Transactions on Geoscience and Remote Sensing 52, 1336-1349.
    42. Remondino, F., 2011. Heritage Recording and 3D Modeling with Photogrammetry and 3D Scanning. Remote Sensing 3, 1104.
    43. Rouse Jr, J., 1974. Monitoring the vernal advancement and retrogradation (green wave effect) of natural vegetation, NASA technical report.
    44. Saalfeld, S., Fetter, R., Cardona, A., Tomancak, P., 2012. Elastic volume reconstruction from series of ultra-thin microscopy sections. Nat Meth 9, 717-720.
    45. Sankaran, S., Khot, L.R., Espinoza, C.Z., Jarolmasjed, S., Sathuvalli, V.R., Vandemark, G.J., Miklas, P.N., Carter, A.H., Pumphrey, M.O., Knowles, N.R., Pavek, M.J., 2015. Low-altitude, high-resolution aerial imaging systems for row and field crop phenotyping: a review. European Journal of Agronomy 70, 112-123.
    46. Sulik, J.J., Long, D.S., 2016. Spectral considerations for modeling yield of canola. Remote Sensing of Environment 184, 161-174.
    47. Suárez, L., Zarco-Tejada, P.J., Berni, J.A.J., González-Dugo, V., Fereres, E., 2009. Modelling PRI for water stress detection using radiative transfer models. Remote Sensing of Environment 113, 730-744.
    48. Szeliski, R., 2006. Image Alignment and Stitching: A Tutorial. Foundations and Trends® in Computer Graphics and Vision 2, 1-104.
    49. Tommaselli, A., Galo, M., de Moraes, M., Marcato, J., Caldeira, C., Lopes, R., 2013. Generating Virtual Images from Oblique Frames. Remote Sensing 5, 1875.
    50. Torres-Sanchez, J., Lopez-Granados, F., De Castro, A.I., Pena-Barragan, J.M., 2013. Configuration and specifications of an Unmanned Aerial Vehicle (UAV) for early site specific weed management. PloS one 8, e58210.
    51. Toth, C., Jóźków, G., 2016. Remote sensing platforms and sensors: a survey. ISPRS Journal of Photogrammetry and Remote Sensing 115, 22-36.
    52. Turner, D., Lucieer, A., Wallace, L., 2014. Direct Georeferencing of Ultrahigh-Resolution UAV Imagery. IEEE Transactions on Geoscience and Remote Sensing 52, 2738-2745.
    53. Vakalopoulou, M., Karantzalos, K., 2014. Automatic descriptor-based co-registration of frame hyperspectral data. Remote Sensing 6, 3409-3426.
    54. Westoby, M.J., Brasington, J., Glasser, N.F., Hambrey, M.J., Reynolds, J.M., 2012. ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology 179, 300-314.
    55. Wikipedia, 2017. Comparison of photo stitching software. Wikipedia, The Free Encyclopedia.
    56. Wu, C., 2011. VisualSFM: A visual structure from motion system.
    57. Yang, C., Westbrook, J., Suh, C., Martin, D., Hoffmann, W., Lan, Y., Fritz, B., Goolsby, J., 2014. An airborne multispectral imaging system based on two consumer-grade cameras for agricultural remote sensing. Remote Sensing 6, 5257-5278.
    58. Ye, Y., Shan, J., 2014. A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differences. ISPRS Journal of Photogrammetry and Remote Sensing 90, 83-95.
    59. Zeitler, W., Doerstel, C., Jacobsen, K., 2002. Geometric Calibration of the DMC Method and Results, Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS.
    60. Zhang, J., Yang, C., Song, H., Hoffmann, W., Zhang, D., Zhang, G., 2016. Evaluation of an airborne remote sensing platform consisting of two consumer-grade cameras for crop identification. Remote Sensing 8, 257.
    61. Zitová, B., Flusser, J., 2003. Image registration methods: a survey. Image and Vision Computing 21, 977-1000.

    下載圖示 校內：2022-08-30公開
    校外：2022-08-30公開
    QR CODE