人類一直致力於探索自然界的奧秘，並將其應用於科學、技術和工程領域。自然界中的生物系統擁有卓越的結構和功能，在環境挑戰下不斷適應和演化。生物材料展現出驚人的生物力學性能，啟發了仿生材料的發展，仿生材料模仿自然界中的結構和功能，具有高強度、輕量化、自修復和環境適應性等優勢，被廣泛應用於航空航天和建築等領域。珍珠母是一種具有高韌性和強度的生物衍生材料，它獨特的結構已成為仿生材料研究的重要焦點。人工智慧(AI)越來越多應用於工程領域，包括基於圖像的深度學習方法和自動化生產製造過程。強化學習(RL)作為AI的一個子領域，模擬了人類學習的過程，可以應用於材料結構設計和優化，然而傳統的試驗和錯誤方法耗時費力，而強化學習可以通過與環境的互動，持續實驗和調整設計，優化材料結構。
本研究提出了以強化學習設計框架來有效設計高韌性的珍珠質結構。通過強化學習與有限元素法結合的設計方法替換位於裂縫尖端周圍的局部結構，以此設計空間中的軟質材料和硬質材料進行結構最佳化，從初始珍珠層結構開始，通過在局部結構上佈置硬質和軟質材料來逐漸改進局部結構的配置，使整體結構能實現更高的韌性。接續進行有限元模擬和實驗測試時，強化學習最佳化的設計表現出對於預裂縫有極出色的抗裂能力，驗證了此設計流程能成功設計出更優秀的材料結構。而該設計框架在未來也可以應用於其他需要靈活性和強度的仿生結構的設計，如醫療植入物或軍用裝甲中使用的仿生結構，該方法可以大大減少設計的時間和成本，實現更高的設計精度和效率，然而每種仿生結構都有其獨特的設計要求，需要相應地修改強化學習和有限元方法模型以達到預期的結果。

Nacre is known for its uniquely high toughness and lightweight capabilities. Its unique structure is composed of soft nacre proteins and stiff calcium carbonates, allowing it to deflect cracks that expand in straight lines to increase energy dissipation. In this study, a design approach is proposed that combines reinforcement learning with finite element method for optimizing the fracture toughness of two-dimensional nacre-like structure.
The optimization objective of the reinforcement learning model is the fracture toughness of the nacre-like structure. The design space consists of a 14x14 local structure located around the crack tip. The model learns how to maximize the structure's toughness by altering the arrangement of soft and hard materials within the local structure during the design process.
In the design results obtained through reinforcement learning optimization, the fracture toughness of the structure is significantly improved compared to its initial state. This improvement greatly reduces the stress concentration at the crack tip. The presence of soft materials on both sides and directly above the pre-existing crack helps to disperse the stress at the tip. Finally, the design results were further validated through simulations and experiments using Abaqus, which consistently showed that the optimized structure exhibited superior crack resistance in both the simulated and experimental results.
This AI-based design method could effectively strengthen the failure toughness of the entire structure and can be easily applied to the design of other structures or materials. Furthermore, structures and materials can be designed for specific requirements through operations during optimization. The design method reported in this study can potentially be extended to the design of more complex structures or material systems.

[1] A. G. Mikos, P. C. Johnson, J. P. Fisher, J. A. Jansen, and D. F. Williams, “The Biomaterials Conundrum in Tissue Engineering,” https://home.liebertpub.com/tea, vol. 20, no. 7–8, pp. 1129–1131, Feb. 2014, doi: 10.1089/TEN.TEA.2013.0769.
[2] R. O. Ritchie, “The conflicts between strength and toughness,” Nature Materials 2011 10:11, vol. 10, no. 11, pp. 817–822, Oct. 2011, doi: 10.1038/nmat3115.
[3] H. L. Gao et al., “Mass production of bulk artificial nacre with excellent mechanical properties,” Nature Communications 2017 8:1, vol. 8, no. 1, pp. 1–8, Aug. 2017, doi: 10.1038/s41467-017-00392-z.
[4] S. H. Lee and S. Wang, “Biodegradable polymers/bamboo fiber biocomposite with bio-based coupling agent,” Compos Part A Appl Sci Manuf, vol. 37, no. 1, pp. 80–91, Jan. 2006, doi: 10.1016/J.COMPOSITESA.2005.04.015.
[5] M. E. Launey, M. J. Buehler, and R. O. Ritchie, “On the Mechanistic Origins of Toughness in Bone,” https://doi.org/10.1146/annurev-matsci-070909-104427, vol. 40, pp. 25–53, Jul. 2010, doi: 10.1146/ANNUREV-MATSCI-070909-104427.
[6] M. J. Buehler, “Molecular nanomechanics of nascent bone: fibrillar toughening by mineralization,” Nanotechnology, vol. 18, no. 29, p. 295102, Jun. 2007, doi: 10.1088/0957-4484/18/29/295102.
[7] U. G. K. Wegst, H. Bai, E. Saiz, A. P. Tomsia, and R. O. Ritchie, “Bioinspired structural materials,” Nature Materials 2014 14:1, vol. 14, no. 1, pp. 23–36, Oct. 2014, doi: 10.1038/nmat4089.
[8] T. H. Huang, C. S. Chen, and S. W. Chang, “Microcrack patterns control the mechanical strength in the biocomposites,” Mater Des, vol. 140, pp. 505–515, Feb. 2018, doi: 10.1016/J.MATDES.2017.12.015.
[9] F. C. Campbell, “Introduction to Composite Materials,” Structural Composite Materials, pp. 1–29, Nov. 2010, doi: 10.31399/ASM.TB.SCM.T52870001.
[10] R. Z. Wang, Z. Suo, A. G. Evans, N. Yao, and I. A. Aksay, “Deformation mechanisms in nacre,” J Mater Res, vol. 16, no. 9, pp. 2485–2493, 2001, doi: 10.1557/JMR.2001.0340.
[11] F. Barthelat and H. D. Espinosa, “An experimental investigation of deformation and fracture of nacre-mother of pearl,” Exp Mech, vol. 47, no. 3, pp. 311–324, Jun. 2007, doi: 10.1007/S11340-007-9040-1/FIGURES/18.
[12] J. Sun and B. Bhushan, “Hierarchical structure and mechanical properties of nacre: a review,” RSC Adv, vol. 2, no. 20, pp. 7617–7632, Aug. 2012, doi: 10.1039/C2RA20218B.
[13] H. Zhao, Z. Yang, and L. Guo, “Nacre-inspired composites with different macroscopic dimensions: strategies for improved mechanical performance and applications,” NPG Asia Materials 2018 10:4, vol. 10, no. 4, pp. 1–22, Apr. 2018, doi: 10.1038/s41427-018-0009-6.
[14] H. Bai et al., “Bioinspired Hydroxyapatite/Poly(methyl methacrylate) Composite with a Nacre-Mimetic Architecture by a Bidirectional Freezing Method,” Advanced Materials, vol. 28, no. 1, pp. 50–56, Jan. 2016, doi: 10.1002/ADMA.201504313.
[15] L. S. Dimas, G. H. Bratzel, I. Eylon, and M. J. Buehler, “Tough Composites Inspired by Mineralized Natural Materials: Computation, 3D printing, and Testing,” Adv Funct Mater, vol. 23, no. 36, pp. 4629–4638, Sep. 2013, doi: 10.1002/ADFM.201300215.
[16] A. Finnemore et al., “Biomimetic layer-by-layer assembly of artificial nacre,” Nature Communications 2012 3:1, vol. 3, no. 1, pp. 1–6, Jul. 2012, doi: 10.1038/ncomms1970.
[17] S. Wan et al., “Nacre-inspired integrated strong and tough reduced graphene oxide–poly(acrylic acid) nanocomposites,” Nanoscale, vol. 8, no. 10, pp. 5649–5656, Mar. 2016, doi: 10.1039/C6NR00562D.
[18] S. Jiang et al., “Nacre-Inspired Strong and Multifunctional Soy Protein-Based Nanocomposite Materials for Easy Heat-Dissipative Mobile Phone Shell,” Nano Lett, vol. 21, no. 7, pp. 3254–3261, Apr. 2021, doi: 10.1021/ACS.NANOLETT.1C00542/ASSET/IMAGES/LARGE/NL1C00542_0005.JPEG.
[19] D. Silver et al., “Mastering the game of Go with deep neural networks and tree search,” Nature 2016 529:7587, vol. 529, no. 7587, pp. 484–489, Jan. 2016, doi: 10.1038/nature16961.
[20] D. Silver et al., “Mastering the game of Go without human knowledge,” Nature 2017 550:7676, vol. 550, no. 7676, pp. 354–359, Oct. 2017, doi: 10.1038/nature24270.
[21] Y. K. Dwivedi et al., “‘So what if ChatGPT wrote it?’ Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy,” Int J Inf Manage, vol. 71, p. 102642, Aug. 2023, doi: 10.1016/J.IJINFOMGT.2023.102642.
[22] M. Van Otterlo and M. Wiering, “Reinforcement learning and markov decision processes,” Adaptation, Learning, and Optimization, vol. 12, pp. 3–42, 2012, doi: 10.1007/978-3-642-27645-3_1/COVER.
[23] C. H. Yu, Z. Qin, and M. J. Buehler, “Artificial intelligence design algorithm for nanocomposites optimized for shear crack resistance,” Nano Futures, vol. 3, no. 3, p. 035001, Aug. 2019, doi: 10.1088/2399-1984/AB36F0.
[24] C. H. Yu et al., “Hierarchical Multiresolution Design of Bioinspired Structural Composites Using Progressive Reinforcement Learning,” Adv Theory Simul, vol. 5, no. 11, p. 2200459, Nov. 2022, doi: 10.1002/ADTS.202200459.
[25] E. Even-Dar, S. Kakade, and Y. Mansour, “Experts in a Markov Decision Process,” Adv Neural Inf Process Syst, vol. 17, Jan. 2004, Accessed: May 05, 2023. [Online]. Available: https://repository.upenn.edu/statistics_papers/457
[26] A. W. Beggs, “On the convergence of reinforcement learning,” J Econ Theory, vol. 122, no. 1, pp. 1–36, May 2005, doi: 10.1016/J.JET.2004.03.008.
[27] S. Padakandla, “A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments,” ACM Computing Surveys (CSUR), vol. 54, no. 6, Jul. 2021, doi: 10.1145/3459991.
[28] S. Tavakoli and A. S. Klar, “Advanced Hydrogels as Wound Dressings,” Biomolecules 2020, Vol. 10, Page 1169, vol. 10, no. 8, p. 1169, Aug. 2020, doi: 10.3390/BIOM10081169.
[29] Y. Liang, J. He, and B. Guo, “Functional Hydrogels as Wound Dressing to Enhance Wound Healing,” ACS Nano, vol. 15, no. 8, pp. 12687–12722, Aug. 2021, doi: 10.1021/ACSNANO.1C04206/ASSET/IMAGES/MEDIUM/NN1C04206_0016.GIF.
[30] Y. H. Joung, “Development of Implantable Medical Devices: From an Engineering Perspective,” Int Neurourol J, vol. 17, no. 3, p. 98, Sep. 2013, doi: 10.5213/INJ.2013.17.3.98.
[31] Z. Shen, D. Hu, G. Yang, and X. Han, “Ballistic reliability study on SiC/UHMWPE composite armor against armor-piercing bullet,” Compos Struct, vol. 213, pp. 209–219, Apr. 2019, doi: 10.1016/J.COMPSTRUCT.2019.01.078.
[32] M. K. Islam, P. J. Hazell, J. P. Escobedo, and H. Wang, “Biomimetic armour design strategies for additive manufacturing: A review,” Mater Des, vol. 205, p. 109730, Jul. 2021, doi: 10.1016/J.MATDES.2021.109730.