有別於傳統 RGB-T 融合方法依賴兩套完整且對稱的平行骨幹網路，本研究提出 GRF-YOLO 模型，透過將熱紅外線影像作為穩定的主要信號，並依據場景可靠性自適應地調控 RGB 影像的貢獻，在與單一模態模型相近的運算成本下，實現可靠的雙模態偵測效能。
GRF-YOLO 模型整合了一個閘控殘差融合（gated residual fusion, GRF）模組，此模組插入於骨幹網路的淺層特徵階段，以熱紅外線特徵串流作為主要模態，並將 RGB 外觀特徵視為條件式補充輸入。由於兩種模態所攜帶的資訊具有互補性，融合閘能夠動態地抑制另一模態中不可靠的信號，且無需承擔將 RGB-T 特徵於網路深層進行整合之完整平行設計的額外運算開銷。考量實際部署限制，熱紅外線串流採用 320×256 的解析度進行處理，與商用無人機平台上常見之低成本熱紅外線相機所支援的解析度相符。
在 DroneVehicle 資料集上的評估中，GRF-YOLO 以 2.50M 參數量及 6.63 GFLOPs 的運算量，達到 mAP50 82.2% 及 mAP50-95 62.5% 的標準物件偵測效能。其表現高於 2.58M 參數的熱紅外線單一模態基準模型，相較於採用雙特徵提取網路、具 3.79M 參數的 RGB-T 基準模型，也在較低模型成本下取得更好的 mAP50。結果顯示，以熱影像為基礎之閘控融合策略可在輕量模型規模內保留雙模態偵測效益，具備用於無人機邊緣運算平台推論的條件。

Unlike conventional RGB-T fusion approaches that rely on two full parallel and symmetric backbone networks, this paper proposes a GRF-YOLO model that achieves robust detection by treating thermal IR as a stable primary signal and gating RGB contributions adaptively based on scene reliability, delivering effective dual modality detection at a computational cost comparable to a single modality model. The GRF-YOLO model integrates a gated residual fusion (GRF) module, inserted at a shallow layer, that casts the thermal IR stream as a primary modality while incorporating RGB appearance cues as a conditional supplement. Since the two modalities carry complementary information, the fusion gate dynamically suppresses unreliable signals from the other modality without the computational overhead of a fully parallel RGB-T design that fuses features at a late stage of the network. To reflect practical deployment constraints, the thermal IR stream is intentionally downsampled to 320×256, consistent with cost effective lower resoulution thermal IR cameras commonly integrated on commercial UAV platforms. Evaluated on the DroneVehicle dataset, the GRF-YOLO achieves 82.2% and 62.5% in standard object detection performance metrics mAP50 and mAP50-95 at merely 2.50M parameters and resulting in 6.63 GFLOPs, surpassing both the thermal IR only baseline at 2.58M parameters and the RGB-T baseline with two feature extraction networks at 3.79M parameters.

[1] S. Hayat, E. Yanmaz, and R. Muzaffar, “Survey on unmanned aerial vehicle networks for civil applications: A communications viewpoint,” IEEE communications surveys & tutorials, vol. 18, no. 4, pp. 2624–2661, 2016.
[2] H. Shakhatreh, A. H. Sawalmeh, A. Al-Fuqaha, Z. Dou, E. Almaita, I. Khalil, N. S. Othman, A. Khreishah, and M. Guizani, “Unmanned aerial vehicles (uavs): A survey on civil applications and key research challenges,” IEEE access, vol. 7, pp. 48 572–48 634, 2019.
[3] S. Srivastava, S. Narayan, and S. Mittal, “A survey of deep learning techniques for vehicle detection from uav images,” Journal of Systems Architecture, vol. 117, p. 102152, 2021.
[4] S. Mittal, “A survey on optimized implementation of deep learning models on the nvidia jetson platform,” Journal of Systems Architecture, vol. 97, pp. 428–442, 2019.
[5] Y. Chen, B. Zheng, Z. Zhang, Q. Wang, C. Shen, and Q. Zhang, “Deep learning on mobile and embedded devices: State-of-the-art, challenges, and future directions,” ACM Computing Surveys (CSUR), vol. 53, no. 4, pp. 1–37, 2020.
[6] X. Xia, S. M. M. Fattah, and M. A. Babar, “A survey on uav-enabled edge computing: Resource management perspective,” ACM Computing Surveys, vol. 56, no. 3, pp. 1–36, 2023.
[7] Y. Sun, B. Cao, P. Zhu, and Q. Hu, “Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 10, pp. 6700–6713, 2022.
[8] R. Gade and T. B. Moeslund, “Thermal cameras and applications: a survey,” Machine vision and applications, vol. 25, no. 1, pp. 245–262, 2014.
[9] A. W. Browne, E. Deyneka, F. Ceccarelli, J. K. To, S. Chen, J. Tang, A. N. Vu, and P. F. Baldi, “Deep learning to enable color vision in the dark,” PloS one, vol. 17, no. 4, p. e0265185, 2022.
[10] M. Li, X. Chen, C. Peng, S. Du, and Y. Li, “Modeling the occlusion problem in thermal imaging to allow seeing through mist and foliage,” Journal of the Optical Society of America A, vol. 36, no. 2, pp. A67–A76, 2019.
[11] H. Li, S. Wen, S. Li, H. Wang, X. Geng, S. Wang, J. Zhai, and W. Zhang, “The research on infrared radiation affected by smoke or fog in different environmental temperatures,” Scientific reports, vol. 14, no. 1, p. 14410, 2024.
[12] S. G. Narasimhan and S. K. Nayar, “Vision and the atmosphere,” International journal of computer vision, vol. 48, no. 3, pp. 233–254, 2002.
[13] M. Brenner, N. H. Reyes, T. Susnjak, and A. L. Barczak, “Rgb-d and thermal sensor fusion: A systematic literature review,” Ieee Access, vol. 11, pp. 82 410–82 442, 2023.
[14] Z. Lin, B. Lin, and Y. Guo, “Label-guided real-time fusion network for rgb–t semantic segmentation,” in Proc. Brit. Mach. Vis. Conf., 2023, pp. 1–11.
[15] Y. Zhang, H. Yu, Y. He, X. Wang, and W. Yang, “Illumination-guided rgbt object detection with inter-and intra-modality fusion,” IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1–13, 2023.
[16] P. Zhu, L. Wen, D. Du, X. Bian, H. Fan, Q. Hu, and H. Ling, “Detection and tracking meet drones challenge,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 11, pp. 7380–7399, 2021.
[17] X. Ying, C. Xiao, W. An, R. Li, X. He, B. Li, X. Cao, Z. Li, Y. Wang, M. Hu et al., “Visible-thermal tiny object detection: A benchmark dataset and baselines,” IEEE transactions on pattern analysis and machine intelligence, vol. 47, no. 7, pp. 6088–6096, 2025.
[18] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149), vol. 2. IEEE, 1999, pp. 246–252.
[19] A. Gaszczak, T. P. Breckon, and J. Han, “Real-time people and vehicle detection from uav imagery,” in Intelligent robots and computer vision XXVIII: algorithms and techniques, vol. 7878. SPIE, 2011, pp. 71–83.
[20] Y. Iwasaki, M. Misumi, and T. Nakamiya, “Robust vehicle detection under various environmental conditions using an infrared thermal camera and its application to road traffic flow monitoring,” Sensors, vol. 13, no. 6, pp. 7756–7773, 2013.
[21] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol. 1. IEEE, 2005, pp. 886–893.
[22] C.-W. Hsu, C.-C. Chang, C.-J. Lin et al., “A practical guide to support vector classification,” 2003.
[23] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, vol. 1. Ieee, 2001, pp. I–I.
[24] S. Hwang, J. Park, N. Kim, Y. Choi, and I. So Kweon, “Multispectral pedestrian detection: Benchmark dataset and baseline,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1037–1045.
[25] K. Song, X. Xue, H. Wen, Y. Ji, Y. Yan, and Q. Meng, “Misaligned visible-thermal object detection: A drone-based benchmark and baseline,” IEEE Transactions on Intelligent Vehicles, 2024.
[26] J. Liu, S. Zhang, S. Wang, and D. N. Metaxas, “Multispectral deep neural networks for pedestrian detection,” arXiv preprint arXiv:1611.02644, 2016.
[27] X. Zhu, Y. Wu, H. Hu, X. Zhuang, J. Yao, D. Ou, W. Li, M. Song, N. Feng, and D. Xu, “Medical lesion segmentation by combining multimodal images with modality weighted unet,” Medical physics, vol. 49, no. 6, pp. 3692–3704, 2022.
[28] C. Li, X. Liang, Y. Lu, N. Zhao, and J. Tang, “Rgb-t object tracking: Benchmark and baseline,” Pattern Recognition, vol. 96, p. 106977, 2019.
[29] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.
[30] H. Xu, M. Gong, X. Tian, J. Huang, and J. Ma, “Cufd: An encoder–decoder network for visible and infrared image fusion based on common and unique feature decomposition,” Computer Vision and Image Understanding, vol. 218, p. 103407, 2022.
[31] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.
[32] J. Li, J. Chen, J. Liu, and H. Ma, “Learning a graph neural network with cross modality interaction for image fusion,” in Proceedings of the 31st ACM international conference on multimedia, 2023, pp. 4471–4479.
[33] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
[34] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
[35] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19.
[36] Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, and Q. Hu, “Eca-net: Efficient channel attention for deep convolutional neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 534–11 542.
[37] Y. Cao, J. Bin, J. Hamari, E. Blasch, and Z. Liu, “Multimodal object detection by channel switching and spatial attention,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 403–411.
[38] J. Arevalo, T. Solorio, M. Montes-y Gómez, and F. A. González, “Gated multimodal units for information fusion,” arXiv preprint arXiv:1702.01992, 2017.
[39] W. El Ahmar, Y. Massoud, D. Kolhatkar, H. AlGhamdi, M. Alja＇Afreh, R. Hammoud, and R. Laganiere, “Enhanced thermal-rgb fusion for robust object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 365–374.
[40] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
[41] G. Jocher and J. Qiu, “Ultralytics yolo11,” https://github.com/ultralytics/ultralytics, 2024, accessed: Jun. 16, 2026.
[42] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125.
[43] Q. Hou, D. Zhou, and J. Feng, “Coordinate attention for efficient mobile network design,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 13 713–13 722.
[44] R. Groenendijk, L. Dorst, and T. Gevers, “Morphpool: Efficient non-linear pooling & unpooling in cnns,” arXiv preprint arXiv:2211.14037, 2022.
[45] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
[46] J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International journal of computer vision, vol. 129, no. 6, pp. 1789–1819, 2021.
[47] Z. Yang, Z. Li, M. Shao, D. Shi, Z. Yuan, and C. Yuan, “Masked generative distillation,” in European conference on computer vision. Springer, 2022, pp. 53–69.
[48] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision. Springer, 2014, pp. 740–755.
[49] D. Wan, R. Lu, Y. Fang, X. Lang, S. Shu, J. Chen, S. Shen, T. Xu, and Z. Ye, “Yolov11-rgbt: Towards a comprehensive single-stage multispectral object detection framework,” arXiv preprint arXiv:2506.14696, 2025.
[50] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618–626.