臺灣組裝工廠在理解組裝說明書中面臨許多挑戰，包括需人力解讀、理解不一致及難以進行自動化組裝流程驗證的問題。因此，需要一個自動建模組裝說明書的系統，負責將指令轉換為機器可讀格式來解決上述問題。然而，現有的有關自動建模教學文本的研究並不足以應對組裝說明書的複雜性，除了無法充分表達組裝說明書中的資訊，還有解析指令不一致的問題。
因此，本研究提出了一種組裝說明書的自動建模系統，該系統包括兩種新方法：Graph Construction和State Diagram Construction。Graph Construction將每個指令轉換為一個圖，該圖不僅包含動作和物件，還充分表示了動作和物件的交互資訊。State Diagram Construction生成組裝說明書的狀態圖，該圖表示指令的動作和物件的變化對應的狀態，並包含指令之間的依賴關係。其中圖的動作-物件交互資訊以及狀態圖中的指令間的依賴關係可以用於驗證組裝流程。Graph Construction 在本文創建的資料集上實現了高準確度，而State Diagram Construction同樣在該資料集表明產生的狀態圖與組裝說明書所涵蓋的資訊高度相似。這些結果表明該系統作為組裝說明書建模工具的可用性和適用性。

Assembly factories in Taiwan face numerous challenges in understanding assembly instructions, including manual interpretation, inconsistent comprehension, and difficulties in automating the verification of assembly processes. However, the existing studies on automatically modeling instructional texts are insufficient to deal with the complexity of assembly instructions.
To address these challenges, this paper proposes an automatic modeling system for assembly instructions. This system includes two novel methods: Graph Construction and State Diagram Construction. Graph Construction transforms each instruction into a graph fully expresses the action-object interaction information. State Diagram Construction generates a state diagram of the assembly manual, which represents the states of the action and objects in each instruction, including the dependencies between the instructions. The action-object interaction information in the graph and the dependencies in the state diagram can be used to verify the assembly process.
Graph Construction achieves high accuracy on the dataset created in this paper, and State Diagram Construction also shows that the resulting state diagram contains information that is highly similar to the assembly manual when evaluated on this dataset. These results demonstrate the availability and suitability of the system as a tool for modeling assembly manuals.

[1] M. Bollini, S. Tellex, T. Thompson, N. Roy, and D. Rus, “Interpreting and Executing Recipes with a Cooking Robot”, in Experimental Robotics, 2013, pp. 481-495, doi: 10.1007/978-3-319-00065-7_33.
[2] M. Tenorth, D. Nyga and M. Beetz, “Understanding and Executing Instructions for Everyday Manipulation Tasks from the World Wide Web,” 2010 IEEE International Conference on Robotics and Automation, 2010, pp. 1486-1491, doi: 10.1109/ROBOT.2010.5509955.
[3] M. Stenmark and P. Nugues, “Natural language programming of industrial robots,” IEEE International Symposium on Robotics, 2013, pp. 1-5, doi: 10.1109/ISR.2013.6695630.
[4] C. Matusek, E. Herbst, L. Zettlemoyer and D. Fox, “Learning to Parse Natural Language Commands to a Robot Control System,” Experimental Robotics, 2013, pp. 403-415, doi: 88. 10.1007/978-3-319-00065-7_28.
[5] Z. Zhang, P. Webster, V. Uren, A. Varga and F. Ciravegna, “Automatically Extracting Procedural Knowledge from Instructional Texts using Natural Language Processing,” Proceedings of Eighth International Conference on language Resources and Evaluation, pp. 520-527, Istanbul, Turkey, European Language Resources Association, 2012.
[6] Y. Yamakata, S. Imahori, H. Maeta and S. Mori, “A Method for Extracting Major Workflow Composed of Ingredients, Tools, and Actions from Cooking Procedural Text,” IEEE International Conference on Multimedia & Expo Workshops, 2016, pp. 1-6, doi: 10.1109/ICMEW.2016.7574705.
[7] K. Walter, M. Minor, R. Bergmann, “Workflow Extraction from Cooking Recipes,” Proceedings of the International Conference on Case-Based Reasoning, pp. 207-216, 2011.
[8] M. Minor, R. Bergmann, S. Görg and K. Walter, “Adaptation of Cooking Instructions Following the Workflow Paradigm,” Proceedings of International Conference on Case-Based Reasoning, pp.199-208, 2010.
[9] P. Qi, Y. Zhang, Y. Zhang, J. Bolton and C. D. Manning, “Stanza: A Python Natural Language Processing Toolkit for Many Human Languages,” Association for Computational Linguistics (ACL) System Demonstrations, 2020.
[10] A. Taylor, M. Marcus and B. Santorini, “The Penn Treebank: An Overview,” 2003.
[11] L. Roger and A. Galen, “Tregex and Tsurgeon: Tools for Querying and Manipulating Tree Data Structures,” Proceedings of the Fifth International Conference on Language Resources and Evaluation, 2006.
[12] P. D. Albericio Martin, “The Interaction of Syntax and Semantics in the Lexicological Analysis of Combine verbs,” 2015.
[13] J. Zhang, P. Tsai, M. Tsai, “Semantic2Graph: Graph-based Multi-modal Feature Fusion for Action Segmentation in Videos,” 2022.
[14] J. Zhang, P. Tsai, M. Tsai, “Graph-Based Embedding Improvement Feature Distribution in Videos,” in IEEE International Conference on Consumer Electronics, 2023.
[15] J. Zhang, P. Tsai, M. Tsai, “DEDGraph: Delay Embedding of Dynamic Graph for Temporal Action Segmentation in Videos,” in the sixth IEEE International Symposium on Compter, Consumer and Control, 2023.
[16] J. R. Landis, G. G. Koch. “The Measurement of Observer Agreement for Categorical Data.” Biometrics, vol. 33, no. 1, 1977, pp. 159–74. 2023.
[17] NeuralCoref: https://github.com/huggingface/neuralcoref
[18] ChatGPT-3.5: https://chat.openai.com