電腦視覺演算法像是物件偵測、影像辨識技術已經被導入許多日常生活的應用中，然而這些演算法通常沒有辦法在實際情形都運作的很完善，原因在於實際景象中，無法預測的影像品質損害情形時常會發生，例如：雜訊、曝光度，以及多變的氣候情況。常見的氣候情況像是下雨、下雪、沙塵暴都會嚴重地影響許多電腦視覺演算法的表現。
本論文提出一個基於循環神經網路的雨紋去除演算法來逐步地去除影像中的雨紋。一個預測雨紋的子網路用來預測一張有雨影像雨紋的部分，提供額外的輔助資訊來移除雨紋。一個結合通道注意機制的殘差密集模塊被應用在主要的循環神經網路來加強雨紋去除的能力。實驗結果顯示，本論文的方法在比較的方法當中，得到了更自然的紋理與細節。

Computer vision algorithms like object detection, image classification have changed our life significantly in various applications. However, these algorithms cannot usually have good performance in practical applications due to the fact that the unpredictable degradations often occur in realistic scene, for instance, noise, illumination, and severe weather conditions. Commonly seen weather conditions such as rain, snow, and sandstorm can adversely affect the performance of many computer vision tasks.
In this Thesis, a rain removal algorithm based on recurrent neural network is proposed to remove rain streak stage by stage. A Rain Streak Prediction Network is proposed to predict the rain streak part of a rainy image, providing more information to deraining. A residual dense block combining with channel attention mechanism, called RDCAB is used to enhance the ability of deraining. Experimental results show that the proposed method gets more nature textures and details compared with available methods.

[1] K. Garg and S. K. Nayar. Detection and removal of rain from videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pages I–I,2004.
[2] Zhang, X., Li, H., Qi, Y., Leow, W.K., Ng, T.K. Rain removal in video by combining temporal and chromatic properties. In IEEE ICME, pages 461–464, 2006.
[3] P. Barnum, S. Narasimhan, T. Kanade. Analysis of rain and snow in frequency space. International Journal of Computer Vision 86 (2-3) (2010) 256–274.
[4] J. Bossu, N. Hautiere, and J.-P. Tarel. Rain or snow detection in image sequences through use of a histogram of orientation of streaks. IJCV, 93(3):348–367, 2011.
[5] C. Stauffer, W. Grimson, Learning patterns of activity using realtime tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (8) (2000) 747–757.
[6] Santhaseelan, V., Asari, V.K.: Utilizing local phase information to remove rain from video. International Journal of Computer Vision 112(1) (2015) 71–89.
[7] Li-Wei Kang, Chia-Wen Lin, Yu-Hsiang Fu. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition. In IEEE Transactions on Image Processing, vol. 21, no. 4, 2013.
[8] Yi-Lei Chen, Chiou-Ting Hsu. A Generalized Low-Rank Appearance Model for Spatio-Temporally Correlated Rain Streaks. In IEEE International Conference on Computer Vision (ICCV), 2013.
[9] Y. Li, R. Tan, X. Guo, J. Lu, and M. Brown. 2016. Rain Streak Removal Using Layer Priors. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[10] Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, John Paisley. Removing rain from single images via a deep detail network. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[11] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[12] Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, Shuicheng Yan. Deep Joint Rain Detection and Removal from a Single Image. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[13] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, Hongbin Zha. Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. arXiv preprint arXiv:1807.05698, resca2018.
[14] Fisher Yu, Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, 2015.
[15] Jie Hu, Li Shen, Samuel Albanie, Gang Sun, Enhua Wu. Squeeze-and-excitation networks. arXiv preprint arXiv:1709.01507, 2017.
[16] H. Zhang, V. A. Sindagi, and V. M. Patel. Image de-raining using a conditional generative adversarial network. arXiv preprint arXiv:1701.05957, 2017.
[17] M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
[18] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Neural Information Processing Systems (NIPS), 2014.
[19] K. Garg and S. K. Nayar. Vision and rain. International Journal of Computer Vision, 75(1):3–27, 2007.
[20] L. W. Kang, C. W. Lin, and Y. H. Fu, “Automatic singleimage-based rain streaks removal via image decomposition,” IEEE Trans. on Image Processing, vol. 21, no. 4, pp. 1742– 1755, April 2012.
[21] A. Tripathi and S. Mukhopadhyay, “Video post processing: low-latency spatiotemporal approach for detection and removal of rain,” IET image processing, vol. 6, no. 2, pp. 181–196, 2012.
[22] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, ´ A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802, 2016.
[23] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[24] Z. Yi, H. Zhang, T. Gong, Tan, and M. Gong. Dualgan. Unsupervised dual learning for image-to-image translation. In IEEE International Conference on Computer Vision (ICCV), 2017.
[25] J.-Y. Zhu, R. Zhang, D. Pathak, T. Darrell, A. A. Efros, O. Wang, and E. Shechtman. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems (NIPS), 2017.
[26] D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[27] Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. arXiv preprint arXiv:1804.07723, 2018.
[28] J. Wang, X. Li, and J. Yang. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[29] R. Qian, R. T. Tan, W. Yang, J. Su, and J. Liu, “Attentive generative adversarial network for raindrop removal from a single image,” in Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition, June 2018.
[30] X. Fu, B. Liang, Y. Huang, X. Ding, and J. Paisley, “Lightweight pyramid networks for image deraining,” IEEE Trans. on Neural Networks and Learning Systems, pp. 1–14, 2019
[31] X. Fu, J. Huang, X. Ding, Y. Liao, and J. Paisley. Clearing the skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing, 26(6):2944–2956, 2017.
[32] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997
[33] K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, ¨ F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014
[34] W. Yang, R. T. Tan, J. Feng, J. Liu, S. Yan, and Z. Guo, “Joint rain detection and removal from a single image with contextualized deep networks,” IEEE Trans. on Pattern Analysis and Machine Intelligence, pp. 1–1, 2019.
[35] R. Li, L.-F. Cheong, and R. T. Tan, “Heavy rain image restoration: Integrating physics model and conditional adversarial learning,” in Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition, June 2019.
[36] X. Hu, C.-W. Fu, L. Zhu, and P.-A. Heng, “Depth-attentional features for single-image rain removal,” in Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition, June 2019.
[37] Youzhao Yang and Hong Lu, "Single image deraining using a recurrent multi-scale aggregation and enhancement network", Proceedings of the IEEE International Conference on Multimedia and Expo, 2019.
[38] Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR (2018)
[39] G. Huang, Z. Liu, K. Q. Weinberger, and L. Maaten. Densely connected convolutional networks. In CVPR, 2017.
[40] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[41] J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. 2016.
[42] D. Ren, W. Zuo, Q. Hu, P. Zhu, and D. Meng, “Progressive image deraining networks: a better and simpler baseline,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3937–3946