研究生: |
杜庭瑜 Du, Ting-Yu |
---|---|
論文名稱: |
藉由實作探索一種後處理的調整方法用以創建更多種神經風格化的影像 A practice-based exploration of post-processing adjustment method for creating variety in neural stylized images |
指導教授: |
卓彥廷
Cho, Yen-Ting |
學位類別: |
碩士 Master |
系所名稱: |
規劃與設計學院 - 創意產業設計研究所 Institute of Creative Industries Design |
論文出版年: | 2018 |
畢業學年度: | 106 |
語文別: | 英文 |
論文頁數: | 109 |
中文關鍵詞: | 神經風格轉換 、深度學習 、影像融合 、亮度-色相-飽和度 、比值變換 、混合技術 |
外文關鍵詞: | Neural style transfer (NST), deep learning, image fusion, intensity-hue-saturation (IHS), Brovey transform (BT), the blending technique |
相關次數: | 點閱:156 下載:17 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
神經風格轉換是一種基於深度學習的卷積神經網路的應用,它可將一般的生活照片藉由選擇的藝術風格圖片來創建出具有此藝術風格的生活圖片。近三年來,神經風格轉換已廣泛地使用在藝術教育和工業應用上面,例如MoMA和Prisma等網站。雖然目前的研究主要集中在抽象繪畫的製作,但其轉換風格的效果往往在視覺上令人驚艷。然而,在神經風格化圖片的生成過程中仍有三個本質上的問題是使用者希望能改進的:即配色的方式,風格筆觸的強弱和對比度的調整。為了讓使用者能夠調整神經風格轉換後的圖片,作者在現有神經風格轉換方法的實驗基礎上,整合了遙測影像處理中的影像融合、對比度增強與混合技術等三種方法,設計了一個快速的後處理方法來滿足使用者的需求。以下為本論文的研究問題:1) 如何整合BT,ICDDS和混合技術等技術來應用到神經風格的轉換並藉由調整參數以提供多重選擇給使用者參考? 2) 對於藝術創作者與藝術教育者,神經風格轉移的價值為何? 本論文為實作型的研究,研究步驟則分為三個階段:初步研究,實驗與評估。初步研究是藉由專案、文獻探討和實務反思來進行;同時針對設計學院的學生進行初步訪談。實驗則包括設計的方法對神經風格轉換與自動渲染機的測試。最後的評估則是以專家的深入訪談來探討如何在藝術創作和藝術教育上的價值;結果顯示對藝術家在收集創作材料時的確可藉由作者提出的方法來啟發意想不到的靈感,同時也有助於學生發展其個人的獨特畫風。鑒於此,由於本研究所提出的方法對神經風格轉換三個本質上的問題可提供多樣的選擇且無需再訓練,在藝術教育和工業的應用將具有一定的運用價值。
Neural style transfer (NST), a technique based on deep learning of convolution neural network (CNN) to create stylized pictures by stylizing ordinary pictures with the predetermined visual art style. In the past three years, NST has become a widely employed approach to produce various styles for the purpose of training in art education and industrial applications such as MoMA and Prisma. Whilst previously research is mainly focused on the production of abstract painting, the effect of NST is often visually impressive. However, the users argue that there are three issues should be carefully investigated during the generation of neural-stylized artwork which are the color scheme, the strength of style stroke, and the adjustment of contrast, which cannot meet the user needs. Based on the experiments of current NST-based methods, the author designed a post-processing software to validate the proposed method establish on image fusion, contrast enhancement, and blending technique which have been widely used in the processing of remote sensing images. The following are my research questions: 1) How to integrate BT, ICDDS and the blending technique into neural style transfer and provide more choices to users by generating adjusting parameters? 2) What is the value of selectivity offered by neural style transfer for artistic creation and art education? This thesis is a practice-based research that includes three phases: preliminary research, experiments, and evaluations. Preliminary research conducted with the iteration of projects, contextual review and reflective practice. Meanwhile, the author also conducted preliminary interview with design college students. Experiment involved testing of style transfer and automatic rendering machine. Evaluation including in-depth interview with experts to validate the proposed method for practical use in artistic creation and art education which show the value of trigger unconventional inspiration by using style transfer and automatic rendering machine in collect materials phase for artists and encourage students to develop personal unique style. In the light of this, since the method proposed in this research can provide multiple choices for the three issues in NST and no need to retrain, it will have certain application value in art education and industrial application.
Alschuler, R. H., &Hattwick, L. A. (1943). Easel painting as an index of personality in preschool children. American Journal of Orthopsychiatry, 13(4), 616–625. https://doi.org/10.1111/j.1939-0025.1943.tb06024.x
Attride-Stirling, J. (2001). Thematic networks: an analytic tool for qualitative research. Qualitative Research, 1(3), 385–405. https://doi.org/10.1177/146879410100100307
Bethge, M., Ecker, A., Gatys, L., Kidziński, Ł., &Warchoł, M. (2015). deepart.io. Retrieved May9, 2018, from https://deepart.io/
Blaise, A. y A. (2016). Art in the Age of Machine Intelligence. Retrieved May21, 2018, from https://medium.com/artists-and-machine-intelligence/what-is-ami-ccd936394a83
Boden, M. A. (1998). Creativity and artificial intelligence. Artificial Intelligence, 103(1–2), 347–356. https://doi.org/10.1016/S0004-3702(98)00055-1
Bogner, A., Littig, B., &Menz, W. (2005). Das Experteninterview : Theorie, Methode, Anwendung. VS Verlag für Sozialwissenschaften. Retrieved from https://books.google.com.tw/books/about/Das_Experteninterview.html?id=KqfMVC4rPukC&redir_esc=y
Braae, N. (2015). Sonic Patterns and Compositional Strategies in Queen’s ‘Bohemian Rhapsody.’ Twentieth-Century Music, 12(02), 173–196. https://doi.org/10.1017/S147857221500002X
Chhabra, G. (2018). AI future to give a lead to women? Retrieved July3, 2018, from http://www.tribuneindia.com/news/jobs-careers/pathfinder/ai-future-to-give-a-lead-to-women/553716.html
COCO Consortium. (2015). COCO - Common Objects in Context. Retrieved May14, 2018, from http://cocodataset.org/#home
Date, P., Ganesan, A., &Oates, T. (2017). Fashioning with Networks: Neural Style Transfer to Design Clothes. Retrieved from http://arxiv.org/abs/1707.09899
Davis, E. (2017). Universal Style Transfer via Feature Transforms with TensorFlow & Keras. Retrieved from https://github.com/eridgd/WCT-TF
Dexter, L. A. (2006). Elite and specialized interviewing. ECPR. Retrieved from https://books.google.com.tw/books?id=spGyXLNREukC&hl=zh-TW&source=gbs_navlinks_s
Dumoulin, V., Shlens, J., &Kudlur, M. (2016). A Learned Representation For Artistic Style. Retrieved from http://arxiv.org/abs/1610.07629
Durling, D. (2002). Discourses on research and the PhD in Design. Quality Assurance in Education, 10(2), 79–85. https://doi.org/10.1108/09684880210423564
Eck, D., &Vranakis, S. (2018). NSynth Super. Retrieved from https://experiments.withgoogle.com/nsynth-super
Engel, J., Resnick, C., Roberts, A., Dieleman, S., Simonyan, K., Norouzi, M., &Eck, D. (2017). NSynth: Neural Audio Synthesis. Retrieved from https://magenta.tensorflow.org/nsynth
Eno, B. (1996). A year with swollen appendices. Faber and Faber. Retrieved from https://books.google.com.tw/books/about/A_Year_with_Swollen_Appendices.html?id=ahV9QgAACAAJ&redir_esc=y
Fessenden, M. (2015, May). Computers Are Learning About Art Faster than Art Historians | Smart News | Smithsonian. Smithsonian. Retrieved from https://www.smithsonianmag.com/smart-news/computers-are-getting-better-identifying-artists-art-historians-are-180955241/
Foster, H. (1985). Postmodern culture. Pluto Press. Retrieved from https://books.google.com.tw/books/about/Postmodern_Culture.html?id=pX_azoDGfpAC&redir_esc=y
Gardiner, J. (2011). Exploring the emerging design territory of construction 3D printing - project led architectural research. Retrieved from https://researchbank.rmit.edu.au/view/rmit:160277
Gatys, L. A., Bethge, M., Hertzmann, A., &Shechtman, E. (2016). Preserving Color in Neural Artistic Style Transfer. Retrieved from http://arxiv.org/abs/1606.05897
Gatys, L. A., Ecker, A. S., &Bethge, M. (2015). A Neural Algorithm of Artistic Style. Retrieved from http://arxiv.org/abs/1508.06576
Gatys, L. A., Ecker, A. S., &Bethge, M. (2016). Image Style Transfer Using Convolutional Neural Networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2414–2423). IEEE. https://doi.org/10.1109/CVPR.2016.265
Gatys, L. A., Ecker, A. S., Bethge, M., Hertzmann, A., &Shechtman, E. (2016). Controlling Perceptual Factors in Neural Style Transfer. Retrieved from http://arxiv.org/abs/1611.07865
Google Brain Team. (2016). Magenta. Retrieved June27, 2018, from https://magenta.tensorflow.org/
Greene, R., Cushman, S., Cavanagh, C., Ramazani, J., Rouzer, P. F., Feinsod, H., …Slessarev, A. (2012). The Princeton encyclopedia of poetry and poetics. Princeton University Press. Retrieved from https://books.google.com.tw/books/about/The_Princeton_Encyclopedia_of_Poetry_and.html?id=MJVlZjIe5o8C&redir_esc=y
Guest, G., MacQueen, K., &Namey, E. (2012). Applied Thematic Analysis. Sage Publications. https://doi.org/10.4135/9781483384436
Henry, R., &Thomas, D. (2017). AutoDraw. Retrieved from https://www.autodraw.com/
International Business Machines Corporation. (2013). IBM Chef Watson. Retrieved from https://www.ibmchefwatson.com/tupler
Ioffe, S., &Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Retrieved from http://arxiv.org/abs/1502.03167
Jaques, N., Gu, S., Turner, R. E., &Eck, D. (2016). Generating Music by Fine-Tuning Recurrent Neural Networks with Reinforcement Learning. In 2016 Conference on Neural Information Processing Systems. Retrieved from https://static.googleusercontent.com/media/research.google.com/zhTW/pubs/archive/45871.pdf
Jing, Y., Liu, Y., Yang, Y., Feng, Z., Yu, Y., Tao, D., &Song, M. (2018). Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields. Retrieved from http://arxiv.org/abs/1802.07101
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., &Song, M. (2017). Neural Style Transfer: A Review. Retrieved from http://arxiv.org/abs/1705.04058
Johnson, J., Alahi, A., &Fei-Fei, L. (2016). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. Retrieved from http://arxiv.org/abs/1603.08155
Jongejan, J., Rowley, H., Kawashima, T., Kim, J., Thomson, R., &Fox-Gieg, N. (2016). Quick, Draw! Google A.I. Experiments. Retrieved from https://quickdraw.withgoogle.com/#
Joshi, B., Stewart, K., &Shapiro, D. (2017). Bringing Impressionism to Life with Neural Style Transfer in Come Swim. Retrieved from http://arxiv.org/abs/1701.04928
Karpathy, A. (2016). CS231n Convolutional Neural Networks for Visual Recognition. Retrieved May14, 2018, from http://cs231n.github.io/convolutional-networks/
Koduri, G. K., Gulati, S., &Serra, X. (2018). Riyaz. Retrieved from https://riyazapp.com/
Kolb, D. A. (1984). Experiential learning : experience as the source of learning and development. Prentice-Hall. Retrieved from https://books.google.com.tw/books/about/Experiential_Learning.html?id=ufnuAAAAMAAJ&redir_esc=y
Ledley, R. S., Buas, M., &Golab, T. J. (n.d.). Fundamentals of true-color image processing. In [1990] Proceedings. 10th International Conference on Pattern Recognition (Vol. i, pp. 791–795). IEEE Comput. Soc. Press. https://doi.org/10.1109/ICPR.1990.118218
Lee, S.-H. (2017). a fast neural style transfer implement with Keras 2. Retrieved from https://github.com/misgod/fast-neural-style-keras
Lewis-Beck, M. S., Bryman, A., &Liao, T. F. (2004). The Sage encyclopedia of social science research methods. Sage. Retrieved from https://books.google.com.tw/books/about/The_SAGE_Encyclopedia_of_Social_Science.html?id=xUy-NNnSQIIC&redir_esc=y
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., &Yang, M.-H. (2017). Universal Style Transfer via Feature Transforms. Retrieved from http://arxiv.org/abs/1705.08086
Lim, H. (2008). 80 Best Photo Effects Photoshop Tutorials. Retrieved May14, 2018, from https://www.hongkiat.com/blog/41-nicest-photoshop-photo-effects-photoshop-tutorials/
Liu, J. G., &Moore, J. M. (1996). Direct decorrelation stretch technique for RGB colour composition. International Journal of Remote Sensing, 17(5), 1005–1018. https://doi.org/10.1080/01431169608949060
Luckerson, V. (2014, July). Netflix, AT&T Enter Paid Peering Agreement to Improve Streaming | Time. Time. Retrieved from http://time.com/3059431/netflix-att-peering/
Marr, B. (2016, September). The Top 10 AI And Machine Learning Use Cases Everyone Should Know About. Forbes. Retrieved from https://www.forbes.com/sites/bernardmarr/2016/09/30/what-are-the-top-10-use-cases-for-machine-learning-and-ai/#4bd407d94c90
Martin, B., &Hanington, B. M. (2012). Universal methods of design : 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers. Retrieved from https://books.google.com.tw/books/about/Universal_Methods_of_Design.html?id=uZ8uzWAcdxEC&redir_esc=y
McCormack, J. (2009). Niche Constructions : Jon McCormack. Retrieved May22, 2018, from http://jonmccormack.info/artworks/niche-constructions/
McFarland, M. (2016). Google’s psychedelic “paint brush” raises the oldest question in art. Retrieved from
https://www.washingtonpost.com/news/innovations/wp/2016/03/10/googles-psychedelic-paint-brush-raises-the-oldest-question-in-art/?noredirect=on&utm_term=.a8d1163bcbc9
Mehmet, S., &Neriman, A. (2016). Developments in Educational Sciences - Child and Drawing. (R.Efe, I.Koleva, E.Atasoy, &İ.Cürebal, Eds.). St. Kliment Ohridski University.
MIT Technology Review. (2015, May). The Machine Vision Algorithm Beating Art Historians at Their Own Game - MIT Technology Review. MIT Technology Review. Retrieved from https://www.technologyreview.com/s/537366/the-machine-vision-algorithm-beating-art-historians-at-their-own-game/
Moiseenkov, A. (2016). Prisma. Retrieved from https://prisma-ai.com/
MoMA, &Google Arts & Culture Lab. (2017). Identifying art through machine learning. Retrieved from https://experiments.withgoogle.com/moma
Montfort, N., Pérez, R., Harrell, D. F., &Campana, A. (2013). Slant: A Blackboard System to Generate Plot, Figuration, and Narrative Discourse Aspects of Stories | Imagination, Computation, and Expression Laboratory. In the Fourth International Conference on Computational Creativity. Australia. Retrieved from http://groups.csail.mit.edu/icelab/content/slant-blackboard-system-generate-plot-figuration-and-narrative-discourse-aspects-stories
Mordvintsev, A. (2015). Deep Dream Generator. Retrieved from https://deepdreamgenerator.com/#tools
Narayanan, H. (2017). Convolutional neural networks for artistic style transfer — Harish Narayanan. Retrieved May14, 2018, from https://harishnarayanan.org/writing/artistic-style-transfer/
Risner, R., &Ward II, T. E. (2007). Concrete Experiences and Practical Exercises: Interventions to Create a Context for a Synergistic Learning Environment. Retrieved from https://higherlogicdownload.s3.amazonaws.com/EVAL/e7137a06-d1d5-4ab6-8bd0-d1818afdb6ab/UploadedImages/Uploaded Documents/Concrete Experiences & Practical Exercises v3a.pdf
Saleh, B., &Elgammal, A. (2015). Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature. Retrieved from http://arxiv.org/abs/1505.00855
Scott, H. (2014). Putting the “critical” into critical studies in art education. Retrieved from https://e-space.mmu.ac.uk/580123/
Simonyan, K., &Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. Retrieved from http://arxiv.org/abs/1409.1556
Smith, H., &Dean, R. T. (2009). Practice-led research, research-led practice in the creative arts. Retrieved from https://books.google.com.tw/books?id=mBCrBgAAQBAJ&dq=Practice-led+Research,+Research-led+Practice+in+Creative+Arts&lr=&hl=zh-TW&source=gbs_navlinks_s
Stigler, J. W., &Hiebert, J. (1999). The teaching gap : best ideas from the world’s teachers for improving education in the classroom. Free Press. Retrieved from https://books.google.com.tw/books/about/The_Teaching_Gap.html?id=LMfLxeHXzpAC&redir_esc=y
Tu, T.-M., Hsu, C.-L., Tu, P.-Y., &Lee, C.-H. (2012). An Adjustable Pan-Sharpening Approach for IKONOS/QuickBird/GeoEye-1/WorldView-2 Imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(1), 125–134. https://doi.org/10.1109/JSTARS.2011.2181827
Tu, T.-M., Huang, P. S., Hung, C.-L., &Chang, C.-P. (2004). A Fast Intensity–Hue–Saturation Fusion Technique With Spectral Adjustment for IKONOS Imagery. IEEE Geoscience and Remote Sensing Letters, 1(4), 309–312. https://doi.org/10.1109/LGRS.2004.834804
Tyka, M. (2016). Mike Tyka- deepdream. Retrieved from http://www.miketyka.com/?s=deepdream
Ulyanov, D., Vedaldi, A., &Lempitsky, V. (2016). Instance Normalization: The Missing Ingredient for Fast Stylization. Retrieved from http://arxiv.org/abs/1607.08022
VonAhn, L., &Hacker, S. (2011). Duolingo. Retrieved from https://www.duolingo.com/