簡易檢索 / 詳目顯示

研究生: 杜庭瑜
Du, Ting-Yu
論文名稱: 藉由實作探索一種後處理的調整方法用以創建更多種神經風格化的影像
A practice-based exploration of post-processing adjustment method for creating variety in neural stylized images
指導教授: 卓彥廷
Cho, Yen-Ting
學位類別: 碩士
Master
系所名稱: 規劃與設計學院 - 創意產業設計研究所
Institute of Creative Industries Design
論文出版年: 2018
畢業學年度: 106
語文別: 英文
論文頁數: 109
中文關鍵詞: 神經風格轉換深度學習影像融合亮度-色相-飽和度比值變換混合技術
外文關鍵詞: Neural style transfer (NST), deep learning, image fusion, intensity-hue-saturation (IHS), Brovey transform (BT), the blending technique
相關次數: 點閱:156下載:17
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 神經風格轉換是一種基於深度學習的卷積神經網路的應用,它可將一般的生活照片藉由選擇的藝術風格圖片來創建出具有此藝術風格的生活圖片。近三年來,神經風格轉換已廣泛地使用在藝術教育和工業應用上面,例如MoMA和Prisma等網站。雖然目前的研究主要集中在抽象繪畫的製作,但其轉換風格的效果往往在視覺上令人驚艷。然而,在神經風格化圖片的生成過程中仍有三個本質上的問題是使用者希望能改進的:即配色的方式,風格筆觸的強弱和對比度的調整。為了讓使用者能夠調整神經風格轉換後的圖片,作者在現有神經風格轉換方法的實驗基礎上,整合了遙測影像處理中的影像融合、對比度增強與混合技術等三種方法,設計了一個快速的後處理方法來滿足使用者的需求。以下為本論文的研究問題:1) 如何整合BT,ICDDS和混合技術等技術來應用到神經風格的轉換並藉由調整參數以提供多重選擇給使用者參考? 2) 對於藝術創作者與藝術教育者,神經風格轉移的價值為何? 本論文為實作型的研究,研究步驟則分為三個階段:初步研究,實驗與評估。初步研究是藉由專案、文獻探討和實務反思來進行;同時針對設計學院的學生進行初步訪談。實驗則包括設計的方法對神經風格轉換與自動渲染機的測試。最後的評估則是以專家的深入訪談來探討如何在藝術創作和藝術教育上的價值;結果顯示對藝術家在收集創作材料時的確可藉由作者提出的方法來啟發意想不到的靈感,同時也有助於學生發展其個人的獨特畫風。鑒於此,由於本研究所提出的方法對神經風格轉換三個本質上的問題可提供多樣的選擇且無需再訓練,在藝術教育和工業的應用將具有一定的運用價值。

    Neural style transfer (NST), a technique based on deep learning of convolution neural network (CNN) to create stylized pictures by stylizing ordinary pictures with the predetermined visual art style. In the past three years, NST has become a widely employed approach to produce various styles for the purpose of training in art education and industrial applications such as MoMA and Prisma. Whilst previously research is mainly focused on the production of abstract painting, the effect of NST is often visually impressive. However, the users argue that there are three issues should be carefully investigated during the generation of neural-stylized artwork which are the color scheme, the strength of style stroke, and the adjustment of contrast, which cannot meet the user needs. Based on the experiments of current NST-based methods, the author designed a post-processing software to validate the proposed method establish on image fusion, contrast enhancement, and blending technique which have been widely used in the processing of remote sensing images. The following are my research questions: 1) How to integrate BT, ICDDS and the blending technique into neural style transfer and provide more choices to users by generating adjusting parameters? 2) What is the value of selectivity offered by neural style transfer for artistic creation and art education? This thesis is a practice-based research that includes three phases: preliminary research, experiments, and evaluations. Preliminary research conducted with the iteration of projects, contextual review and reflective practice. Meanwhile, the author also conducted preliminary interview with design college students. Experiment involved testing of style transfer and automatic rendering machine. Evaluation including in-depth interview with experts to validate the proposed method for practical use in artistic creation and art education which show the value of trigger unconventional inspiration by using style transfer and automatic rendering machine in collect materials phase for artists and encourage students to develop personal unique style. In the light of this, since the method proposed in this research can provide multiple choices for the three issues in NST and no need to retrain, it will have certain application value in art education and industrial application.

    摘要 i ABSTRACT ii TABLE OF CONTENT iv LIST OF FIGURES vii LIST OF TABLE x I. INTRODUCTION 1 1.1 Research Questions and Methods 3 1.2 Thesis Structure 4 II. CONTEXTUAL REVIEW 5 2.1 Artificial Intelligence in Artistic Creation and Art Education Use 6 2.1.1 What is Artificial Intelligence 6 2.1.2 Machine Learning in Artistic Creation 7 2.1.2.1 Example 1: Chef Watson 9 2.1.2.2 Example 2: Magenta 10 2.1.3 Machine Learning in Art Education 11 2.1.3.1 Example 1: MoMA & machine learning 13 2.1.3.2 Example 2: Riyaz 14 2.2 Style transfer 16 2.2.1 Pastiche in art education 16 2.2.2 Transition of style transfer 16 2.2.3 The difference between the classic filters and NST 18 2.2.4 Deep Learning in Convolution Neural Network 20 2.2.5 Review of Classical Approaches in NST 21 2.2.5.1 Neural Style Transfer (NST) 21 2.2.5.2 Fast Style Transfer (FST) 24 2.2.5.3 Multi-Style Transfer (MST) 25 2.2.5.4 Universal Style Transfer (UST) 27 2.2.6 The Issues among Three Existing Approaches 28 2.3 Review of Image Processing Techniques for Remote Sensing Imagery 30 2.3.1 Image Fusion by IHS and BT 31 2.3.2 Color Enhance by ICDDS 32 III. RESEARCH METHOD 35 3.1 Research Flow 35 3.2 Contextual Review 37 3.3 Project 37 3.4 Data Collection 38 3.5 Analysis 39 IV. PROJECTS 40 4.1 Introduction 40 4.2 Initial Interview: Cognition and Acceptance of Neural Style Transfer 41 4.2.1 Evaluation 47 4.3 Project 1: Integrate BT and ICDDS into Proposed Method for NST-based Methods 49 4.3.1 Evaluation: Effect of Different Modules of the Post-Processing Framework 53 4.4 Project 2: Automatic Rendering Machine 60 4.4.1 Test of Automatic Rendering Machine 60 4.5 Comparison with UST 64 4.6 Summary 67 V. EVALUATION 68 5.1 Expert Interview: What is Neural Style Transfer to Artists and Art Educators 68 5.2 Discussion of the Proposed Method in Art Creation 71 5.3 Discussion of the Proposed Method in Art Education 75 5.4 Summary 78 VI. CONCLUSION AND FUTURE DIRECTION 79 6.1 Conclusion 79 6.2 Suggestion for Future Research 80 References 81 Appendix 88

    Alschuler, R. H., &Hattwick, L. A. (1943). Easel painting as an index of personality in preschool children. American Journal of Orthopsychiatry, 13(4), 616–625. https://doi.org/10.1111/j.1939-0025.1943.tb06024.x
    Attride-Stirling, J. (2001). Thematic networks: an analytic tool for qualitative research. Qualitative Research, 1(3), 385–405. https://doi.org/10.1177/146879410100100307
    Bethge, M., Ecker, A., Gatys, L., Kidziński, Ł., &Warchoł, M. (2015). deepart.io. Retrieved May9, 2018, from https://deepart.io/
    Blaise, A. y A. (2016). Art in the Age of Machine Intelligence. Retrieved May21, 2018, from https://medium.com/artists-and-machine-intelligence/what-is-ami-ccd936394a83
    Boden, M. A. (1998). Creativity and artificial intelligence. Artificial Intelligence, 103(1–2), 347–356. https://doi.org/10.1016/S0004-3702(98)00055-1
    Bogner, A., Littig, B., &Menz, W. (2005). Das Experteninterview : Theorie, Methode, Anwendung. VS Verlag für Sozialwissenschaften. Retrieved from https://books.google.com.tw/books/about/Das_Experteninterview.html?id=KqfMVC4rPukC&redir_esc=y
    Braae, N. (2015). Sonic Patterns and Compositional Strategies in Queen’s ‘Bohemian Rhapsody.’ Twentieth-Century Music, 12(02), 173–196. https://doi.org/10.1017/S147857221500002X
    Chhabra, G. (2018). AI future to give a lead to women? Retrieved July3, 2018, from http://www.tribuneindia.com/news/jobs-careers/pathfinder/ai-future-to-give-a-lead-to-women/553716.html
    COCO Consortium. (2015). COCO - Common Objects in Context. Retrieved May14, 2018, from http://cocodataset.org/#home
    Date, P., Ganesan, A., &Oates, T. (2017). Fashioning with Networks: Neural Style Transfer to Design Clothes. Retrieved from http://arxiv.org/abs/1707.09899
    Davis, E. (2017). Universal Style Transfer via Feature Transforms with TensorFlow & Keras. Retrieved from https://github.com/eridgd/WCT-TF
    Dexter, L. A. (2006). Elite and specialized interviewing. ECPR. Retrieved from https://books.google.com.tw/books?id=spGyXLNREukC&hl=zh-TW&source=gbs_navlinks_s
    Dumoulin, V., Shlens, J., &Kudlur, M. (2016). A Learned Representation For Artistic Style. Retrieved from http://arxiv.org/abs/1610.07629
    Durling, D. (2002). Discourses on research and the PhD in Design. Quality Assurance in Education, 10(2), 79–85. https://doi.org/10.1108/09684880210423564
    Eck, D., &Vranakis, S. (2018). NSynth Super. Retrieved from https://experiments.withgoogle.com/nsynth-super
    Engel, J., Resnick, C., Roberts, A., Dieleman, S., Simonyan, K., Norouzi, M., &Eck, D. (2017). NSynth: Neural Audio Synthesis. Retrieved from https://magenta.tensorflow.org/nsynth
    Eno, B. (1996). A year with swollen appendices. Faber and Faber. Retrieved from https://books.google.com.tw/books/about/A_Year_with_Swollen_Appendices.html?id=ahV9QgAACAAJ&redir_esc=y
    Fessenden, M. (2015, May). Computers Are Learning About Art Faster than Art Historians | Smart News | Smithsonian. Smithsonian. Retrieved from https://www.smithsonianmag.com/smart-news/computers-are-getting-better-identifying-artists-art-historians-are-180955241/
    Foster, H. (1985). Postmodern culture. Pluto Press. Retrieved from https://books.google.com.tw/books/about/Postmodern_Culture.html?id=pX_azoDGfpAC&redir_esc=y
    Gardiner, J. (2011). Exploring the emerging design territory of construction 3D printing - project led architectural research. Retrieved from https://researchbank.rmit.edu.au/view/rmit:160277
    Gatys, L. A., Bethge, M., Hertzmann, A., &Shechtman, E. (2016). Preserving Color in Neural Artistic Style Transfer. Retrieved from http://arxiv.org/abs/1606.05897
    Gatys, L. A., Ecker, A. S., &Bethge, M. (2015). A Neural Algorithm of Artistic Style. Retrieved from http://arxiv.org/abs/1508.06576
    Gatys, L. A., Ecker, A. S., &Bethge, M. (2016). Image Style Transfer Using Convolutional Neural Networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2414–2423). IEEE. https://doi.org/10.1109/CVPR.2016.265
    Gatys, L. A., Ecker, A. S., Bethge, M., Hertzmann, A., &Shechtman, E. (2016). Controlling Perceptual Factors in Neural Style Transfer. Retrieved from http://arxiv.org/abs/1611.07865
    Google Brain Team. (2016). Magenta. Retrieved June27, 2018, from https://magenta.tensorflow.org/
    Greene, R., Cushman, S., Cavanagh, C., Ramazani, J., Rouzer, P. F., Feinsod, H., …Slessarev, A. (2012). The Princeton encyclopedia of poetry and poetics. Princeton University Press. Retrieved from https://books.google.com.tw/books/about/The_Princeton_Encyclopedia_of_Poetry_and.html?id=MJVlZjIe5o8C&redir_esc=y
    Guest, G., MacQueen, K., &Namey, E. (2012). Applied Thematic Analysis. Sage Publications. https://doi.org/10.4135/9781483384436
    Henry, R., &Thomas, D. (2017). AutoDraw. Retrieved from https://www.autodraw.com/
    International Business Machines Corporation. (2013). IBM Chef Watson. Retrieved from https://www.ibmchefwatson.com/tupler
    Ioffe, S., &Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Retrieved from http://arxiv.org/abs/1502.03167
    Jaques, N., Gu, S., Turner, R. E., &Eck, D. (2016). Generating Music by Fine-Tuning Recurrent Neural Networks with Reinforcement Learning. In 2016 Conference on Neural Information Processing Systems. Retrieved from https://static.googleusercontent.com/media/research.google.com/zhTW/pubs/archive/45871.pdf
    Jing, Y., Liu, Y., Yang, Y., Feng, Z., Yu, Y., Tao, D., &Song, M. (2018). Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields. Retrieved from http://arxiv.org/abs/1802.07101
    Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., &Song, M. (2017). Neural Style Transfer: A Review. Retrieved from http://arxiv.org/abs/1705.04058
    Johnson, J., Alahi, A., &Fei-Fei, L. (2016). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. Retrieved from http://arxiv.org/abs/1603.08155
    Jongejan, J., Rowley, H., Kawashima, T., Kim, J., Thomson, R., &Fox-Gieg, N. (2016). Quick, Draw! Google A.I. Experiments. Retrieved from https://quickdraw.withgoogle.com/#
    Joshi, B., Stewart, K., &Shapiro, D. (2017). Bringing Impressionism to Life with Neural Style Transfer in Come Swim. Retrieved from http://arxiv.org/abs/1701.04928
    Karpathy, A. (2016). CS231n Convolutional Neural Networks for Visual Recognition. Retrieved May14, 2018, from http://cs231n.github.io/convolutional-networks/
    Koduri, G. K., Gulati, S., &Serra, X. (2018). Riyaz. Retrieved from https://riyazapp.com/
    Kolb, D. A. (1984). Experiential learning : experience as the source of learning and development. Prentice-Hall. Retrieved from https://books.google.com.tw/books/about/Experiential_Learning.html?id=ufnuAAAAMAAJ&redir_esc=y
    Ledley, R. S., Buas, M., &Golab, T. J. (n.d.). Fundamentals of true-color image processing. In [1990] Proceedings. 10th International Conference on Pattern Recognition (Vol. i, pp. 791–795). IEEE Comput. Soc. Press. https://doi.org/10.1109/ICPR.1990.118218
    Lee, S.-H. (2017). a fast neural style transfer implement with Keras 2. Retrieved from https://github.com/misgod/fast-neural-style-keras
    Lewis-Beck, M. S., Bryman, A., &Liao, T. F. (2004). The Sage encyclopedia of social science research methods. Sage. Retrieved from https://books.google.com.tw/books/about/The_SAGE_Encyclopedia_of_Social_Science.html?id=xUy-NNnSQIIC&redir_esc=y
    Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., &Yang, M.-H. (2017). Universal Style Transfer via Feature Transforms. Retrieved from http://arxiv.org/abs/1705.08086
    Lim, H. (2008). 80 Best Photo Effects Photoshop Tutorials. Retrieved May14, 2018, from https://www.hongkiat.com/blog/41-nicest-photoshop-photo-effects-photoshop-tutorials/
    Liu, J. G., &Moore, J. M. (1996). Direct decorrelation stretch technique for RGB colour composition. International Journal of Remote Sensing, 17(5), 1005–1018. https://doi.org/10.1080/01431169608949060
    Luckerson, V. (2014, July). Netflix, AT&T Enter Paid Peering Agreement to Improve Streaming | Time. Time. Retrieved from http://time.com/3059431/netflix-att-peering/
    Marr, B. (2016, September). The Top 10 AI And Machine Learning Use Cases Everyone Should Know About. Forbes. Retrieved from https://www.forbes.com/sites/bernardmarr/2016/09/30/what-are-the-top-10-use-cases-for-machine-learning-and-ai/#4bd407d94c90
    Martin, B., &Hanington, B. M. (2012). Universal methods of design : 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers. Retrieved from https://books.google.com.tw/books/about/Universal_Methods_of_Design.html?id=uZ8uzWAcdxEC&redir_esc=y
    McCormack, J. (2009). Niche Constructions : Jon McCormack. Retrieved May22, 2018, from http://jonmccormack.info/artworks/niche-constructions/
    McFarland, M. (2016). Google’s psychedelic “paint brush” raises the oldest question in art. Retrieved from
    https://www.washingtonpost.com/news/innovations/wp/2016/03/10/googles-psychedelic-paint-brush-raises-the-oldest-question-in-art/?noredirect=on&utm_term=.a8d1163bcbc9
    Mehmet, S., &Neriman, A. (2016). Developments in Educational Sciences - Child and Drawing. (R.Efe, I.Koleva, E.Atasoy, &İ.Cürebal, Eds.). St. Kliment Ohridski University.
    MIT Technology Review. (2015, May). The Machine Vision Algorithm Beating Art Historians at Their Own Game - MIT Technology Review. MIT Technology Review. Retrieved from https://www.technologyreview.com/s/537366/the-machine-vision-algorithm-beating-art-historians-at-their-own-game/
    Moiseenkov, A. (2016). Prisma. Retrieved from https://prisma-ai.com/
    MoMA, &Google Arts & Culture Lab. (2017). Identifying art through machine learning. Retrieved from https://experiments.withgoogle.com/moma
    Montfort, N., Pérez, R., Harrell, D. F., &Campana, A. (2013). Slant: A Blackboard System to Generate Plot, Figuration, and Narrative Discourse Aspects of Stories | Imagination, Computation, and Expression Laboratory. In the Fourth International Conference on Computational Creativity. Australia. Retrieved from http://groups.csail.mit.edu/icelab/content/slant-blackboard-system-generate-plot-figuration-and-narrative-discourse-aspects-stories
    Mordvintsev, A. (2015). Deep Dream Generator. Retrieved from https://deepdreamgenerator.com/#tools
    Narayanan, H. (2017). Convolutional neural networks for artistic style transfer — Harish Narayanan. Retrieved May14, 2018, from https://harishnarayanan.org/writing/artistic-style-transfer/
    Risner, R., &Ward II, T. E. (2007). Concrete Experiences and Practical Exercises: Interventions to Create a Context for a Synergistic Learning Environment. Retrieved from https://higherlogicdownload.s3.amazonaws.com/EVAL/e7137a06-d1d5-4ab6-8bd0-d1818afdb6ab/UploadedImages/Uploaded Documents/Concrete Experiences & Practical Exercises v3a.pdf
    Saleh, B., &Elgammal, A. (2015). Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature. Retrieved from http://arxiv.org/abs/1505.00855
    Scott, H. (2014). Putting the “critical” into critical studies in art education. Retrieved from https://e-space.mmu.ac.uk/580123/
    Simonyan, K., &Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. Retrieved from http://arxiv.org/abs/1409.1556
    Smith, H., &Dean, R. T. (2009). Practice-led research, research-led practice in the creative arts. Retrieved from https://books.google.com.tw/books?id=mBCrBgAAQBAJ&dq=Practice-led+Research,+Research-led+Practice+in+Creative+Arts&lr=&hl=zh-TW&source=gbs_navlinks_s
    Stigler, J. W., &Hiebert, J. (1999). The teaching gap : best ideas from the world’s teachers for improving education in the classroom. Free Press. Retrieved from https://books.google.com.tw/books/about/The_Teaching_Gap.html?id=LMfLxeHXzpAC&redir_esc=y
    Tu, T.-M., Hsu, C.-L., Tu, P.-Y., &Lee, C.-H. (2012). An Adjustable Pan-Sharpening Approach for IKONOS/QuickBird/GeoEye-1/WorldView-2 Imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(1), 125–134. https://doi.org/10.1109/JSTARS.2011.2181827
    Tu, T.-M., Huang, P. S., Hung, C.-L., &Chang, C.-P. (2004). A Fast Intensity–Hue–Saturation Fusion Technique With Spectral Adjustment for IKONOS Imagery. IEEE Geoscience and Remote Sensing Letters, 1(4), 309–312. https://doi.org/10.1109/LGRS.2004.834804
    Tyka, M. (2016). Mike Tyka- deepdream. Retrieved from http://www.miketyka.com/?s=deepdream
    Ulyanov, D., Vedaldi, A., &Lempitsky, V. (2016). Instance Normalization: The Missing Ingredient for Fast Stylization. Retrieved from http://arxiv.org/abs/1607.08022
    VonAhn, L., &Hacker, S. (2011). Duolingo. Retrieved from https://www.duolingo.com/

    下載圖示 校內:立即公開
    校外:立即公開
    QR CODE