這篇論文介紹了一個利用前饋式(Feed forward)的深度學習方法進行的地圖藝術風格轉換。使用者可以自行選擇肖像與地圖。並將其貼到任意地圖上。我們都可以將其轉換成與地圖藝術相似的風格。我們的方法修改了現有的損失函數與網路架構，改良了風格轉換的方法。並且改善了之前需要將肖像從地圖上提取的缺點。使用者可以更加便利的使用此系統。此外，我們除了單張影像更針對影片進行訓練。因此我們的方法可以將輸入的影片與地圖結合後輸出風格轉換後的影片。由於地圖相關的影片資料集蒐集不易。因此我們自行將蒐集到的影片利用影像切割將主體與背景分離後與大量地圖進行合成，利用此方法來進行風格轉換的訓練。而我們的網路架構也改善了以往需要耗費大量時間計算的問題，使影片的風格轉換可以達到即時轉換的效果。

This paper introduces a map art style conversion using feed forward deep learning method. Users can choose the portrait and map by themselves. And paste it on any map. We can all transform it into a style similar to map art. Our method modifies the existing loss function and network architecture and improves the style conversion method. And it improves the shortcomings of the previous need to extract the portrait from the map. Users can use this system more conveniently. In addition, we train for videos in addition to single images. Therefore, our method can combine the input video with the map and output the converted video. It is not easy to collect map-related video data collection. Therefore, we use image cutting to separate the main body and the background and synthesize it with a large number of maps and use this method to train the style conversion. And our network architecture has also improved the problem that required a lot of time to calculate in the past, so that the style conversion of the video can achieve the effect of real-time conversion.

[1] C.-Y. Shih, “Human Geography Art Map Generation,” 2017.
[2] E. Praun, H. Hoppe, M. Webb and A. Finkelstein, "Real-Time Hatching," in Proc. ACM SIGGRAPH, 2001.
[3] O. Deussen and T. StrothotteProc, "Computer-Generated Pen-and-Ink Illustration of Trees," in ACM SIGGRAPH, 2000.
[4] B. Meier, "Painterly Rendering for Animation," in Proc. ACM SIGGRAPH, 1996.
[5] D. Cornish, A. Rowan and D. Luebke, "View-Dependent Particles for Interactive Non-Photorealistic Rendering," in Proc. Graphics Interface (GI ’01), 2001.
[6] R. Kalnins, L. Markosian, B. Meier, M. Kowalski, J. Lee, P. Davidson, M. Webb, J. Hughes and A. Finkelstein, "WYSIWYG NPR: Drawing Strokes Directly on 3D Models," in ACM Trans. Graphics vol. 21, no. 3, 2002.
[7] M.-T. Chi and T.-Y. Lee, "Stylized and Abstract Painterly Rendering System Using a Multiscale Segmented Sphere Hierarchy," in Special Issue - Anatomic Rendering and Visualization, IEEE Trans. Visualization and Computer Graphics, vol. 12, no. 1, 2006.
[8] H. Xu, N. Gossett and B. Chen, "Pointworks: Abstraction and Rendering of Sparsely Scanned Outdoor Environments," in Proc. 15th Eurographics Symp. Rendering (EGSR ’04), 2004.
[9] C.-R. Yan, M.-T. Chi, T.-Y. Lee and W.-C. Lin, "Stylized Rendering Using Samples of a Painted Image," in IEEE Transactions on Visualization and Computer Graphics, March/April,Vol. 14, No. 2, 2008.
[10] T.-Y. Lee, C.-R. Yen and M.-T. Chi, "Stylized Rendering for Anatomic Visualization," in IEEE Computing in Science & Engineering, vol. 9, no. 1, 2007.
[11] W. Zhang, C. Cao, S. Chen, J. Liu and X. Tang, "Style Transfer Via Image Component Analysis," IEEE Transactions on Multimedia, 11 2013.
[12] Z. Song and S. Liu, "Sufficient Image Appearance Transfer Combining Color and Texture," IEEE Transactions on Multimedia, vol. 19, no. 4, pp. 702-711, 4 2017.
[13] J. Liu, W. Yang, X. Sun and W. Zeng, "Photo Stylistic Brush: Robust Style Transfer via Superpixel-Based Bipartite Graph," in Proc. ACM Int’l Conf. Multimedia, 2016.
[14] Z. Hu, S. Liu, J. Jiang, R. Hong, M. Wang and S. Yan, "PicWords: Render a Picture by Packing Keywords," in IEEE TRANSACTIONS ON MULTIMEDIA, 2014.
[15] S.-S. Lin, M.-C. Hu, C.-H. Lee and T.-Y. Lee, "Efficient QR Code Beautification With High Quality Visual Content," IEEE Trans. Multimedia, pp. 1515-1524, 17 9 2015.
[16] S.-S. Lin, I.-C. Yeh, C.-H. Lin and T.-Y. Lee, "Patch-Based Image Warping for Content-Aware Retargeting.," IEEE Trans. Multimedia, pp. 359-368, 15 2 2013.
[17] L. A. Gatys, A. S. Ecker and M. Bethge, "Image style transfer using convolutional neural networks," in In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[18] Z. K. and S. A., "Very deep convolutional networks for large-scale image recognition," in ICLR, 2015.
[19] F. Luan, S. Paris, E. Shechtman and K. Bala, "Deep Photo Style Transfer," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[20] C. Li and M. Wand., "Combining markov random fields and convolutional neural networks for image synthesis," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 27-30, 6 2016.
[21] F. Luan, S. Paris, E. Shechtman and K. Bala, "Deep Painterly Harmonization," Computer Graphics Forum 37(4), pp. 95-106, 4 2018.
[22] J. Zhu, T. Park, P. Isola and A. A. Efros, "Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks," in 2017 IEEE International Conference on Computer Vision (ICCV), 2017.
[23] A. Selim, M. Elgharib and L. Doyle, "Painting Style Transfer for Head Portraits using Convolutional Neural Networks," ACM Trans. Graph 35, 4, Article 129, 2016.
[24] M. Lu, F. Xu, H. Zhao, A. Yao, Y. Chen and L. Zhang, "Exemplar-Based Portrait Style Transfer," IEEE Access ( Volume: 6 ), p. 58532 – 58542, 8 8 2018.
[25] D. Lin, Y. Wang, G. Xu, J. Li and K. Fu, "Transform a simple sketch to a Chinese painting by a multiscale deep neural network," Algorithms, 1 2018.
[26] D. Chen, J. Liao, L. Yuan, N. Yu and G. Hua, "Coherent Online Video Style Transfer," in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017.
[27] J. Johnson, A. Alahi and L. Fei-Fei, "Perceptual losses for real-time style transfer and super-resolution," in European Conference on Computer Vision, 2016.
[28] C. Gao, D. Gu, F. Zhang and Y. Yu, "ReCoNet: Real-time Coherent Video Style Transfer Network.," in ArXiv abs/1807.01197 (2018), 2018.
[29] X. Huang and S. Belongie, "Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization," in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017.
[30] Y. Li, C. Fang, J. Yang, Z. Wang, X. Lu and M.-H. Yang, "Universal Style Transfer via Feature Transforms," in NIPS, 2017.
[31] X. Li, S. Liu, J. Kautz and M.-H. Yang, "Learning Linear Transformations for Fast Image and Video Style Transfer," in CVPR, 2019.
[32] Y. Z. a. C. F. a. Y. W. a. Z. W. a. Z. L. a. Y. F. a. J. Yang, "Multimodal Style Transfer via Graph Cuts," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5942-5950, 2019.
[33] Y. Jing, Y. Yang, Z. Feng, J. Ye, Y. Yu and M. Song, "Neural Style Transfer: A Review," arXiv preprint arXiv:1705.04058, 2017.
[34] J. P.-T. F. Perazzi, B. McWilliams, L. V. Gool, M. Gross and A. Sorkine-Hornung, "A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016.
[35] Pinterest, "https://www.pinterest.com/".
[36] E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy and T. Brox, "FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks," in 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[37] Ruder, Manuel, A. Dosovitskiy and T. Brox, "Artistic Style Transfer for Videos," in GCPR, 2016.
[38] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen and T. Aila, "Analyzing and Improving the Image Quality of StyleGAN," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[39] S. Singh and S. Krishnan, "Filter Response Normalization Layer: Eliminating Batch Dependence in the," in CVPR 2020, 2020.
[40] M.-C. Yeh and S. Tang, "Improved Style Transfer by Respecting Inter-layer Correlations," ArXiv abs/1801.01933 (2018), 2018.
[41] D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," in 3rd International Conference for Learning Representations, San Diego, 2015.
[42] W.-S. Lai, J.-B. Huang, O. Wang, E. Shechtman, E. Yumer and M.-H. Yang, "Learning Blind Video Temporal Consistency," in ECCV 2018, 2018.