近年來，機器學習與神經網路技術已廣泛應用於電腦輔助製造（Computer-Aided Manufacturing, CAM）領域。然而，傳統監督式學習與非監督式學習方法在面對製造產業中高維度且複雜的狀態空間時，常面臨著最佳解資料蒐集成本極高且難以標註等問題。為面對上述挑戰，本研究提出一種結合卷積神經網路（Convolutional Neural Network, CNN）與強化學習之刀具尺寸選擇方法，以模擬軟體擷取工件之三軸截面特徵影像，經圖片前處理過程製作餘料剖面合成圖，提供強化學習代理人(Agent)狀態輸入，再藉由 CNN 優異的特徵擷取能力，提取難以參數化之工件幾何特徵，作為動作價值函數（Action-Value Function）之函數近似器。代理人透過深度 Q 網路（Deep Q-Network, DQN）根據工件特徵影像估計各種銑刀尺寸對應之動作價值，並依據動作選擇策略決定各道加工工序所使用之刀具尺寸。於訓練過程中，代理人持續與加工模擬環境互動，並以加工時間效率及加工完成度作為獎勵函數之評估依據，同時搭配經驗回放機制重複利用已探索之加工刀具組合資料，逐步逼近實際加工所獲得之獎勵，此過程有助於提升模型收斂穩定性，亦能降低過擬合（Overfitting）風險，進一步增強模型之泛化能力。相較於傳統監督式學習需事先建立 Ground Truth 最佳解標籤，本研究所提出之方法可直接透過與環境互動學習刀具選擇策略，降低對大量標註資料之依賴。最終，本研究模型於測試集工件之加工工序類型判斷準確率為 81.25 %。儘管刀具尺寸完全命中最佳解之準確率為 56.25 %，但於加工完成度達 99% 之工件中，預測加工總時間與最佳解之平均相對誤差僅有 6.26 %。結果證實，本研究所提出之方法能有效學習刀具尺寸選擇策略，兼顧加工完成度與加工效率，並展現強化學習於銑削製程規劃之可行性，以及面對大量組合空間與複雜製造環境時之應用潛力。

In Computer-Aided Manufacturing (CAM), traditional machine learning faces high costs in collecting and labeling optimal solution data within high-dimensional state spaces. To address this, this study proposes a tool size selection method combining Convolutional Neural Networks (CNN) and reinforcement learning. This method extracts 3-axis slice sectional views via simulation software to generate a sectional composite image of a workpiece representing the remaining material, which serves as the agent's state input. CNN extracts complex geometric features to approximate the action-value function. Through a Deep Q-Network (DQN), the agent evaluates action values of various milling tool sizes to determine tools for each machining operation. During training, machining time efficiency and final completion rate serve as reward criteria. An experience replay mechanism enhances convergence stability and mitigates overfitting. Compared to supervised learning, this method learns policies directly through environmental interactions, significantly reducing reliance on optimal labels. Experimental results show the model achieves an 81.25% accuracy in classifying machining operation types. Although the accuracy of perfectly matching the optimal tool size is 56.25%, for workpieces reaching a 99% completion rate, the average relative error of predicted total machining time compared to the optimal solution is merely 6.26%. Results verify this method effectively balances machining completion and efficiency, demonstrating reinforcement learning's high potential in milling process planning.

[1] J. Wu, R. Lei, and Y. Peng, "Manufacturing Feature Recognition Method Based on Subgraph Decomposition," in 2022 International Conference on Frontiers of Artificial Intelligence and Machine Learning (FAIML), 2022: IEEE, pp. 120–126.
[2] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, "Pointnet: Deep learning on point sets for 3d classification and segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660.
[3] H. Zhang, S. Zhang, Y. Zhang, J. Liang, and Z. Wang, "Machining feature recognition based on a novel multi-task deep learning network," Robotics and Computer-Integrated Manufacturing, vol. 77, p. 102369, 2022.
[4] X. Yan, K. Yamazaki, and J. Liu, "Recognition of machining features and feature topologies from NC programs," Computer-aided design, vol. 32, no. 10, pp. 605–616, 2000.
[5] Z. Zhang, P. Jaiswal, and R. Rai, "Featurenet: Machining feature recognition based on 3d convolution neural network," Computer-Aided Design, vol. 101, pp. 12–22, 2018.
[6] 蔡仁翔, "使用 YOLO11n-seg 進行 2.5 D 銑削加工特徵辨識," 2025.
[7] J. Isotalo and K. Jansson, "A CAPP System for Automatic Tool Selection in Drilling Operations," in A Cost Effective Use of Computer Aided Technologies and Integration Methods in Small and Medium Sized Companies: Elsevier, 1993, pp. 37–42.
[8] X. L. Phung, H. S. Truong, and N. T. Bui, "Expert system based on integrated fuzzy AHP for automatic cutting tool selection," Applied Sciences, vol. 9, no. 20, p. 4308, 2019.
[9] M. Salehi and R. Tavakkoli-Moghaddam, "Application of genetic algorithm to computer-aided process planning in preliminary and detailed planning," Engineering Applications of Artificial Intelligence, vol. 22, no. 8, pp. 1179–1187, 2009.
[10] L. T. Tunc, "Smart tool path generation for 5-axis ball-end milling of sculptured surfaces using process models," Robotics and Computer-Integrated Manufacturing, vol. 56, pp. 212–221, 2019.
[11] O. Ozkirimli, L. T. Tunc, and E. Budak, "Generalized model for dynamics and stability of multi-axis milling with complex tool geometries," Journal of Materials Processing Technology, vol. 238, pp. 446–458, 2016.
[12] M. Yavartanoo, S. Hong, R. Neshatavar, and K. M. Lee, "CNC-Net: self-supervised learning for CNC machining operations," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 9816–9825.
[13] C. Chung, C.-W. Yang, and H.-M. Chang, "Development of convolutional neural network based autonomous milling process planning system for 2.5 D parts," The International Journal of Advanced Manufacturing Technology, vol. 141, no. 7, pp. 4489–4504, 2025.
[14] J.-Z. Li, "Development of Automatic Tool-Path Planning System for Clamped Parts Using NX," Master's thesis Master's thesis, Department of Mechanical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, 2019.
[15] 黃柏松, "透過緊密連接卷積神經網路開發自動化銑削製程規劃系統," 2023.
[16] 楊騏瑋, "應用不同卷積神經網路架構於自動銑削製程規劃之研究," 2024.
[17] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction (no. 1). MIT press Cambridge, 1998.
[18] R. S. Sutton, "Learning to predict by the methods of temporal differences," Machine learning, vol. 3, no. 1, pp. 9–44, 1988.
[19] C. J. Watkins and P. Dayan, "Q-learning," Machine learning, vol. 8, no. 3, pp. 279–292, 1992.
[20] Y. Wan, W. Xu, and T.-Y. Zuo, "Tool path optimization for complex cavity milling based on reinforcement learning approach," IEEE Access, vol. 11, pp. 66793–66807, 2023.
[21] D. Kalandyk, B. Kwiatkowski, and D. Mazur, "CNC machine control using deep reinforcement learning," Bulletin of the Polish Academy of Sciences Technical Sciences, pp. e148940–e148940, 2024.
[22] W. Wu, Z. Huang, J. Zeng, and K. Fan, "A fast decision-making method for process planning with dynamic machining resources via deep reinforcement learning," Journal of manufacturing systems, vol. 58, pp. 392–411, 2021.
[23] Y. Sugisawa, K. Takasugi, and N. Asakawa, "Machining sequence learning via inverse reinforcement learning," Precision Engineering, vol. 73, pp. 477–487, 2022.
[24] T. Dannen, L. Becker, C. Lürken, P. Niemietz, and T. Bergs, "Methodology for automated and computer-aided work plan generation using voxel-based accessibility analysis through reinforcement learning in one-off manufacturing," Procedia CIRP, vol. 130, pp. 288–293, 2024.
[25] H. Rui, Y. Shilong, Z. Fan, and H. Bo, "An effective NC machining process planning method for complex parts by integrating CT-AOG and reinforcement learning," Computers & Industrial Engineering, p. 111986, 2026.
[26] 震虎精密科技股份有限公司. "震虎綜合型錄 General Catalogue." https://www.speedtiger.com.tw/document/ (accessed 20 July, 2026).
[27] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
[28] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, vol. 25, 2012.
[29] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning (no. 2). MIT press Cambridge, 2016.
[30] D. H. Hubel and T. N. Wiesel, "Receptive fields of single neurones in the cat's striate cortex," The Journal of physiology, vol. 148, no. 3, p. 574, 1959.
[31] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
[32] V. Dumoulin and F. Visin, "A guide to convolution arithmetic for deep learning," arXiv preprint arXiv:1603.07285, 2016.
[33] Y.-L. Boureau, J. Ponce, and Y. LeCun, "A theoretical analysis of feature pooling in visual recognition," in Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 111–118.
[34] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929–1958, 2014.
[35] J. Tompson, R. Goroshin, A. Jain, Y. LeCun, and C. Bregler, "Efficient object localization using convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 648–656.
[36] X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks. InProceedings of the fourteenth international conference on artificial intelligence and statistics 2011 Jun 14 (pp. 315-323)," in JMLR Workshop and Conference Proceedings.
[37] P. J. Huber, "Robust estimation of a location parameter," in Breakthroughs in statistics: Methodology and distribution: Springer, 1992, pp. 492–518.
[38] R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440–1448.
[39] M. L. Puterman, Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
[40] R. Bellman, "Dynamic programming," science, vol. 153, no. 3731, pp. 34–37, 1966.
[41] R. S. Sutton and A. G. Barto, "Reinforcement learning: An introduction 2nd ed," MIT press Cambridge, vol. 1, no. 2, p. 25, 2018.
[42] V. Mnih et al., "Human-level control through deep reinforcement learning," nature, vol. 518, no. 7540, pp. 529–533, 2015.
[43] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, "Deep reinforcement learning: A brief survey," IEEE signal processing magazine, vol. 34, no. 6, pp. 26–38, 2017.