| 研究生: |
黃建學 Huang, Jian-Xue |
|---|---|
| 論文名稱: |
基於生成對抗網路之手寫風格字體生成分析與比較 Analysis and comparison of handwriting style font generation based on generative adversarial network |
| 指導教授: |
陳牧言
Chen, Mu-Yen |
| 學位類別: |
碩士 Master |
| 系所名稱: |
工學院 - 工程科學系 Department of Engineering Science |
| 論文出版年: | 2023 |
| 畢業學年度: | 111 |
| 語文別: | 中文 |
| 論文頁數: | 46 |
| 中文關鍵詞: | 中文字體生成 、風格遷移 、生成對抗網路 、深度學習 |
| 外文關鍵詞: | Chinese font geeration, Style transfer, Generative adversarial networks, Deep Learning |
| 相關次數: | 點閱:177 下載:34 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
近年來,由於科技的進步,人們交流都使用通訊軟體,如 Line、Facebook 等等,導致生活中寫字的機會越來越少,學習書法也不再像以前一樣受歡迎。 然而,練習寫字是很重要的,一個人寫的字往往會給他人既定印象,寫字的過 程對專注力、認知能力、情緒放鬆等方面亦有正面影響,若因為時代的發展導 致人們的字越來越醜,是一個非常遺憾的事情。現今雖然仍有熱衷寫字者會在 社群分享自己的字帖供他人欣賞、交流,但是他人若想模仿該字體卻非常困難, 因為能臨摹的僅有該作者分享的字,而且數量也相當有限。這些人要創造自己 的個人字體難度非常大,這是因為中文字有接近三萬個,要寫出這些字需要花 費相當多的時間、心力。因此,本研究欲透過模型輔助創造個人風格字體,減 少創造字體時所需要的字體數量,降低一般人創造字體時所花費的時間、心力, 提高人們對學習書法的動機,讓多人享受到練習書法時對認知行為能力的益處。
本研究將字體生成視為風格遷移問題,把一種字體的風格轉換為目標字體 的風格,因此本研究比較了不同的風格遷移模型,分別為 CycleGAN、Zi2zi、 DG-Font。由於大部分過往的研究都是使用了大量的訓練集來做成效評估,因此 本研究透過將逐步增加訓練集的方式找到生成字體所需的最低資料集數量,並 透過手寫辨識的準確率來保證生成字體的品質,藉此讓使用者可以最大程度的 降低負擔。另外,本研究也實驗了透過不同的評估指標比較不同原始字體對生 成效果的影響,結果發現不同原始字體間的效果差異顯著。一般的使用者無法 正確地分辨何種字體適合作為原始字體,也沒有一個標準,這便會導致使用模 型時可能會因為原始字體的選擇導致效果較差。因此,本研究進行了同書體相 似度比較的實驗,以找出原始字體選擇的標準,結果發現原始字體與目標字體 相似度越高生成效果便會越好。
Due to the development of science and technology, people now have fewer and fewer opportunities to write, and most of the time they communicate by typing, so fewer and fewer people are learning calligraphy. However, learning calligraphy has positive effects on cognition, concentration, and emotional relaxation. It would be a pity if people no longer learn calligraphy. Therefore, this study hopes to use font generation to improve the motivation to learn calligraphy, so that more people can enjoy the benefits of calligraphy in terms of cognitive behavior. This study regards font generation as a style transfer problem and compares three common style transfer models in font generation problems, namely CycleGAN, Zi2zi and DG-Font. We find the minimum number of datasets needed to generate fonts by gradually increasing the training set and ensure the quality of generated fonts through handwriting recognition. In addition, the impact of different original fonts is compared through different evaluation indicators, and the results are also significantly different. Finally, we also experimented and compared the similarity of the same font and found that the higher the similarity between the original font and the target font, the better the generation effect.
[1] C.-C. Hsiao, C.-C. Lin, C.-G. Cheng, Y.-H. Chang, H.-C. Lin, H.-C. Wu, and C.-A. Cheng, "Self-Reported Beneficial Effects of Chinese Calligraphy Handwriting Training for Individuals with Mild Cognitive Impairment: An Exploratory Study," International Journal of Environmental Research and Public Health, 2023, vol. 20, no. 2, p. 1031.
[2] H. S. Kao, S. P. Lam, and T. T. Kao, "Chinese calligraphy handwriting (CCH): a case of rehabilitative awakening of a coma patient after stroke," Neuropsychiatric Disease and Treatment, 2018, pp. 407-417.
[3] Z. Bin, T. Jing-Hua, L. Chun-Kai, and L. Jing-An, "Positive effect of practicing Chinese calligraphic handwriting on emotion and emotional regulation strategies in children," Journal of Psychological Science, 2013, vol. 36, no. 1, p. 98.
[4] W. Chen, Y. He, Y. Gao, C. Zhang, C. Chen, S. Bi, P. Yang, Y. Wang, and W. Wang, "Long-term experience of Chinese calligraphic handwriting is associated with better executive functions and stronger resting-state functional connectivity in related brain regions," Plos One, 2017, vol. 12, no. 1, p. e0170660.
[5] H. S. Kao, M. Xu, and T. T. Kao, "Calligraphy, psychology and the Confucian literati personality," Psychology and Developing Societies, 2021, vol. 33, no. 1, pp. 54-72.
[6] C. Wen, Y. Pan, J. Chang, Y. Zhang, S. Chen, Y. Wang, M. Han, and Q. Tian, "Handwritten Chinese font generation with collaborative stroke refinement," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 3882-3891.
[7] J. Chang, Y. Gu, Y. Zhang, Y.-F. Wang, and C. Innovation, "Chinese Handwriting Imitation with Hierarchical Generative Adversarial Network," in BMVC, 2018, p. 290.
[8] P. Lyu, X. Bai, C. Yao, Z. Zhu, T. Huang, and W. Liu, "Auto-encoder guided GAN for Chinese calligraphy synthesis," in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), 2017, vol. 1: IEEE, pp. 1095-1100.
[9] B. Chang, Q. Zhang, S. Pan, and L. Meng, "Generating handwritten chinese characters using cyclegan," in 2018 IEEE winter conference on applications of computer vision (WACV), 2018: IEEE, pp. 199-207.
[10] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial networks," Communications of the ACM, 2020, vol. 63, no. 11, pp. 139-144.
[11] Y. Yu, F. Zhan, S. Lu, J. Pan, F. Ma, X. Xie, and C. Miao, "Wavefill: A wavelet-based generation network for image inpainting," in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 14114-14123.
[12] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, and Z. Wang, "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681-4690.
[13] H. Liu, Z. Wan, W. Huang, Y. Song, X. Han, and J. Liao, "Pd-gan: Probabilistic diverse gan for image inpainting," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9371-9381.
[14] J. Li, W. Monroe, T. Shi, S. Jean, A. Ritter, and D. Jurafsky, "Adversarial learning for neural dialogue generation," ,2017, arXiv preprint arXiv:1701.06547.
[15] L. A. Gatys, A. S. Ecker, and M. Bethge, "Image style transfer using convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2414-2423.
[16] X. Huang and S. Belongie, "Arbitrary style transfer in real-time with adaptive instance normalization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1501-1510.
[17] J. Yoo, Y. Uh, S. Chun, B. Kang, and J.-W. Ha, "Photorealistic style transfer via wavelet transforms," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9036-9045.
[18] L. A. Gatys, A. S. Ecker, and M. Bethge, "A neural algorithm of artistic style," , 2015 arXiv preprint arXiv:1508.06576.
[19] H. Zhang and K. Dana, "Multi-style generative network for real-time transfer," in Proceedings of the European Conference on Computer Vision (ECCV) Workshops, 2018, pp. 0-0.
[20] J. Johnson, A. Alahi, and L. Fei-Fei, "Perceptual losses for real-time style transfer and super-resolution," in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, 2016: Springer, pp. 694-711.
[21] Z. Chen, C. Wang, B. Yuan, and D. Tao, "Puppeteergan: Arbitrary portrait animation with semantic-aware appearance transformation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13518-13527.
[22] C. Wang, C. Xu, and D. Tao, "Self-supervised pose adaptation for cross-domain image animation," IEEE Transactions on Artificial Intelligence, 2020, vol. 1, no. 1, pp. 34-46.
[23] X. Chen, C. Xu, X. Yang, and D. Tao, "Long-term video prediction via criticization and retrospection," IEEE Transactions on Image Processing, 2020, vol. 29, pp. 7090-7103.
[24] J. Dong, X. Li, C. Xu, X. Yang, G. Yang, X. Wang, and M. Wang, "Dual encoding for video retrieval by text," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, vol. 44, no. 8, pp. 4065-4080.
[25] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125-1134.
[26] M. Mirza and S. Osindero, "Conditional generative adversarial nets" ,2014 ,arXiv preprint arXiv:1411.1784.
[27] K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan, "Unsupervised pixel-level domain adaptation with generative adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3722-3731.
[28] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223-2232.
[29] Z. Lian and J. Xiao, "Automatic shape morphing for chinese characters," in SIGGRAPH Asia 2012 Technical Briefs, 2012, pp. 1-4.
[30] S. Yang, J. Liu, Z. Lian, and Z. Guo, "Awesome typography: Statistics-based text effects transfer," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7464-7473.
[31] J.-W. Lin, C.-Y. Hong, R.-I. Chang, Y.-C. Wang, S.-Y. Lin, and J.-M. Ho, "Complete font generation of Chinese characters in personal handwriting style," in 2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC), 2015: IEEE, pp. 1-5.
[32] D. Sun, T. Ren, C. Li, H. Su, and J. Zhu, "Learning to write stylized chinese characters by reading a handful of examples," arXiv preprint arXiv: 1712.06424,2017.
[33] S. Xu, T. Jin, H. Jiang, and F. C. Lau, "Automatic generation of personal chinese handwriting by capturing the characteristics of personal handwriting," in Twenty-First IAAI Conference, 2009: Citeseer.
[34] A. Zong and Y. Zhu, "Strokebank: Automating personalized chinese handwriting generation," in Proceedings of the AAAI conference on artificial intelligence, 2014, vol. 28, no. 2, pp. 3024-3029.
[35] W. Pan, Z. Lian, R. Sun, Y. Tang, and J. Xiao, "Flexifont: a flexible system to generate personal font libraries," in Proceedings of the 2014 ACM symposium on Document engineering, 2014, pp. 17-20.
[36] Y. Tian, "Master Chinese calligraphy with conditional adversarial networks," ed, 2017.
[37] Y. Xie, X. Chen, L. Sun, and Y. Lu, "Dg-font: Deformable generative networks for unsupervised font generation," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 5130-5140.
[38] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 2015: Springer, pp. 234-241.
[39] J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, "Deformable convolutional networks," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 764-773.
[40] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, 2004 , vol. 13, no. 4, pp. 600-612.
[41] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586-595.
[42] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, "Gans trained by a two time-scale update rule converge to a local nash equilibrium," Advances in neural information processing systems, 2017 , vol. 30.
[43] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818-2826.
[45] 故宮博物院 書畫典藏資料檢索系統 檢自 https://painting.npm.gov.tw/
[46] 思源黑體 檢自 https://briian.com/25754/
[47] 雅風手寫體 檢自 https://www.mianfeiziti.com/thread-69582.htm
[48] 蘭亭字體 檢自 http://www.fontvip.com/FangZheng-Fonts/43729.html
[49] 金梅字體 檢自 https://freefontsfile.com/2012-02-14-01-50-08.html
[50] 芫荽字體 檢自 https://www.kocpc.com.tw/archives/427062
[51] 源樣黑體 檢自https://github.com/ButTaiwan/genyog-font