製造數位孿生(Manufacturing Digital Twin, MDT)結合人工智慧與三維環境，仍面臨製造語意與場景分離、建置仰賴人工及互動偏向被動查詢等問題。本研究整合 OpenUSD、AI Agent與Human-in-the-Loop (HITL)，提出MDT自動化建置與主動式人本互動框架。首先，建立USD-based MDT MetaModel，定義Asset、Sensor、Behavior與Topology等語意及其與OpenUSD場景元素的映射。其次，由AI Agent將自然語言需求與工程資訊轉換為具型別約束的MDT-BuildSpec，再由受控工具生成、驗證及有限修復MDT模型實例。最後，整合即時資料、場景脈絡與來源文件，形成異常觸發、場景定位、處置選項、人員決策及結果記錄的 HITL 協作閉環。 OP2 加工站的案例中，模型實例於 45 題製造語意問答取得0.933的 Exact Match、0.911 的Judge Accuracy及零棄答率； AI Repair Agent於135次測試成功修復109次，成功率80.7%。自動化建置約需2分30秒，較手動與快速樣板流程分別縮短97.2%及94.5%。本研究建立以USD-based MDT MetaModel為語意基礎，串聯MDT模型實例的自動化建置、驗證與修復，以及異常情境下的場景定位、具來源依據之決策支援與HITL協作，形成涵蓋建置與運行階段的整合框架。在學術上，本研究提供製造語意與OpenUSD場景整合，以及AI Agent參與MDT建置與人機協作的方法架構；在產業應用方面，本研究可縮短MDT模型實例的建置時間、提升模型結構與資料關聯的一致性，並為可追溯的異常處理與人員決策提供實作基礎。

Manufacturing digital twins (MDTs) increasingly combine artificial intelligence with three-dimensional environments, yet existing approaches often separate manufacturing semantics from 3D scenes, rely on manual construction, and support mainly passive queries. This study integrates OpenUSD, AI agents, and human-in-the-loop (HITL) principles to develop a framework for automated MDT construction and proactive human-centered interaction. First, a USD-based MDT MetaModel defines Asset, Sensor, Behavior, and Topology semantics and maps them to OpenUSD scene elements. Second, AI agents transform natural-language requirements and engineering information into a typed MDT-BuildSpec, while controlled tools generate, validate, and perform constrained repairs on MDT model instances. Finally, the framework integrates real-time data, scene context, and source documents to support anomaly triggering, scene localization, source-grounded response options, human decisions, and outcome recording within an HITL loop.
In the OP2 machining station case, the MDT model instance achieved an Exact Match score of 0.933, a Judge Accuracy of 0.911, and a zero abstention rate across 45 manufacturing-semantic questions. The AI Repair Agent successfully repaired 109 of 135 test cases, achieving an 80.7% success rate. Automated construction required approximately 2 minutes and 30 seconds, reducing construction time by 97.2% relative to manual construction and 94.5% relative to the rapid-template process. These results demonstrate an integrated workflow spanning semantic modeling, model-instance construction and validation, anomaly localization, and human-authorized operational interaction. Academically, the proposed framework connects OpenUSD-based manufacturing semantics with AI-agent-assisted MDT construction and human–machine collaboration. Practically, it improves construction efficiency and model consistency while providing a foundation for traceable anomaly handling and human decision-making.

[1] Q. Qi and F. Tao, "Digital twin and big data towards smart manufacturing and Industry 4.0: 360 degree comparison," IEEE Access, vol. 6, pp. 3585–3593, 2018.
[2] F. Tao, H. Zhang, A. Liu, and A. Y. C. Nee, "Digital twin in industry: State-of-the-art," IEEE Transactions on Industrial Informatics, vol. 15, no. 4, pp. 2405–2415, Apr. 2019.
[3] G. Hildebrandt et al., “Data Integration for Digital Twins in Industrial Automation: A Systematic Literature Review,” IEEE Access, 2024.
[4] M. Breque, L. De Nul, and A. Petridis, Industry 5.0: Towards a Sustainable, Human-Centric and Resilient European Industry. Luxembourg: Publications Office of the European Union, 2021, doi: 10.2777/308407.
[5] D. B. Kim, M. S. Bajestani, J. Y. Lee, S.-J. Shin, and S. D. Noh, “Introduction of human-in-the-loop in smart manufacturing (H-SM),” International Journal of Precision Engineering and Manufacturing–Smart Technology, vol. 2, no. 2, pp. 209–214, Jul. 2024, doi: 10.57062/ijpem-st.2024.00115.
[6] S. Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” 2023.
[7] P. Lewis et al., "Retrieval-augmented generation for knowledge-intensive NLP tasks," in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2020.
[8] NVIDIA, “What is OpenUSD?” Learn OpenUSD. [Online]. Available: https://docs.nvidia.com/learn-openusd/latest/what-openusd/index.html. Accessed: Jul. 29, 2026.
[9] NVIDIA, “OpenUSD for developers.” [Online]. Available: https://developer.nvidia.com/openusd. Accessed: Jul. 29, 2026.
[10] BMW Group, “BMW Group and NVIDIA take virtual factory planning to the next level,” BMW Group PressClub Global, Apr. 13, 2021. [Online]. Available: https://www.press.bmwgroup.com/global/article/detail/T0329569EN/. Accessed: Jul. 29, 2026.
[11] M.-H. Hung et al., “A novel implementation framework of Digital Twins for intelligent manufacturing based on Container Technology and Cloud Manufacturing Services,” IEEE Transactions on Automation Science and Engineering, vol. 19, no. 3, pp. 1614–1630, Jul. 2022. doi:10.1109/tase.2022.3143832
[12] M.-H. Hung, M.-J. Shi, Y.-C. Lin, Y.-X. Chang, C.-C. Chen, Y.-C. Kao, and F.-T. Cheng, “A novel digital twin efficient development scheme for manufacturing,” in Proc. 2024 IEEE International Conference on Automation Science and Engineering (CASE), Bari, Italy, Aug. 28–Sep. 1, 2024.
[13] 黃于庭, 整合生成式AI與 AutomationML於製造數位孿生的自動化構建與人機互動, 2025年7月.
[14] T. Ji, J. Polzer, and X. Xu, “Developing cognitive digital twin for smart manufacturing: Methodology and case study,” in Proc. 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE), Bari, Italy, pp. 1656–1661, Aug. 2024, doi: 10.1109/CASE59546.2024.10711555.
[15] ISO 23247-1:2021, Automation Systems and Integration—Digital Twin Framework for Manufacturing—Part 1: Overview and General Principles. Geneva, Switzerland: International Organization for Standardization, 2021.
[16] ISO 23247-2:2021, Automation Systems and Integration—Digital Twin Framework for Manufacturing—Part 2: Reference Architecture. Geneva, Switzerland: International Organization for Standardization, 2021.
[17] Industrial Digital Twin Association, Specification of the Asset Administration Shell—Part 1: Metamodel, IDTA 01001-3-0-2, 2025. [Online]. Available: https://industrialdigitaltwin.org/wp-content/uploads/2025/03/IDTA-01001-3-0-2_SpecificationAssetAdministrationShell_Part1_Metamodel.pdf. Accessed: Jul. 29, 2026.
[18] OpenAI, “Function calling,” OpenAI API Documentation. [Online]. Available: https://developers.openai.com/api/docs/guides/function-calling. Accessed: Jul. 29, 2026.
[19] Model Context Protocol, “Model Context Protocol specification,” version 2025-11-25. [Online]. Available: https://modelcontextprotocol.io/specification/2025-11-25. Accessed: Jul. 29, 2026.
[20] Y. Xia, Z. Xiao, N. Jazdi, and M. Weyrich, “Generation of Asset Administration Shell with large language model agents: Toward semantic interoperability in digital twins in the context of Industry 4.0,” IEEE Access, vol. 12, pp. 84863–84877, 2024, doi: 10.1109/ACCESS.2024.3415470.
[21] A. Chandrasekhar, J. Chan, F. Ogoke, O. Ajenifujah, and A. Barati Farimani, “AMGPT: A large language model for contextual querying in additive manufacturing,” Additive Manufacturing Letters, vol. 11, Art. no. 100232, 2024, doi: 10.1016/j.addlet.2024.100232.
[22] Y. Pulse, Y.-B. Kang, A. Banerjee, A. Forkan, and P. P. Jayaraman, “FactoryLLM: A safe and open-source AI playground for evaluating LLMs in smart factories,” arXiv preprint arXiv:2606.14119, 2026, doi: 10.48550/arXiv.2606.14119.
[23] J. Leng, W. Sha, B. Wang, P. Zheng, C. Zhuang, Q. Liu, T. Wuest, D. Mourtzis, and L. Wang, “Industry 5.0: Prospect and retrospect,” Journal of Manufacturing Systems, vol. 65, pp. 279–295, Oct. 2022, doi: 10.1016/j.jmsy.2022.09.017.
[24] IEC 62714-1:2018, Engineering Data Exchange Format for Use in Industrial Automation Systems Engineering—Automation Markup Language—Part 1: Architecture and General Requirements. Geneva, Switzerland: International Electrotechnical Commission, 2018.
[25] ISO 7870-2:2023, Control Charts—Part 2: Shewhart Control Charts. Geneva, Switzerland: International Organization for Standardization, 2023.
[26] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ Questions for Machine Comprehension of Text,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA, pp. 2383–2392, 2016, doi: 10.18653/v1/D16-1264.
[27] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” Advances in Neural Information Processing Systems, vol. 36, pp. 46595–46623, 2023.
[28] P. Rajpurkar, R. Jia, and P. Liang, “Know What You Don’t Know: Unanswerable Questions for SQuAD,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, Australia, pp. 784–789, 2018, doi: 10.18653/v1/P18-2124.