高階醫療骨材植入物的生產需同時滿足少量多樣與高精度的生產標準。作為醫療用塑膠料件的超高分子量聚乙烯（Ultra-High Molecular Weight Polyethylene, UHMWPE），其電腦數值控制（Computer Numerical Control, CNC）加工為確保品質的關鍵製程，且生產現場常可能面臨不可預期之突發性機台故障，導致既有排程失效造成產能劇烈波動。面對此類挑戰，傳統以靜態規則為主之派工方式，難以在故障頻繁發生的動態環境下，有效兼顧交期承諾與產能利用。
為解決上述問題，本研究提出混合式動態排程優化方法（Hybrid Dynamic Scheduling Optimization Approach, HDSOA），針對突發性機台故障，所引發之平行機台事件驅動重排程問題(考量序列相依設定時間)，整合強化學習(Reinforcement Learning, RL)之Q-learning的即時決策與禁忌搜尋(Tabu Search, TS)之鄰域深層優化，建構強化學習前導之禁忌搜尋演算法（Reinforcement Learning-Initialized Tabu Search, RL-First），透過先學習引導、後優化精修之演算法（Tabu Search-Guided Reinforcement Learning, TS-First），透過先搜尋、後學習機制，以TS的搜尋成果引導RL快速習得高效率的排程法則的兩種混合模式。本研究設計採二階段目標規劃之方法：首要目標為最小化總延遲時間；在確保交期水準的基礎上，進一步降低換線次數與成本，藉此提升現場執行之穩定性。
本研究以國內骨材廠為例，進行實際生產資料的效能評估，並特別針對突發性故障，設計六種不同的故障強度情境。結果顯示 HDSOA在各情境下均能降低總延遲；其中在情境六中，RL-First可將總延遲由 Baseline的791,390分鐘降至286,895分鐘（改善63.7%），並將延遲工單由141張降至109張；同時在約346.9秒內產生可行重排程方案，證實其具備應對突發變故之即時可行性。
實證結果指出，即使在嚴重故障情境下，透過策略性跨機台轉單進行負載平衡，雖可能微幅增加換線成本，但能有效抑制延遲擴散，並提升交期服務水準。本研究為高階醫材製造場域提供了一套可量化，且具備實務應用價值的智慧化重排程決策方法，可有效因應突發性機台故障之挑戰。

In high-end medical device manufacturing, the production of artificial joint implants must simultaneously satisfy requirements for high precision and high-mix, low-volume (HMLV) production. Taking the critical CNC machining of Ultra-High Molecular Weight Polyethylene (UHMWPE) as an example, processing times and process stability are subject to uncertainty due to material properties. More critically, unexpected machine failures can easily render existing schedules invalid. Traditional dispatching methods based on static rules (such as Earliest Due Date) struggle to balance delivery times and capacity utilization in such dynamic environments.
This study proposes a Hybrid Dynamic Scheduling Optimization Approach (HDSOA) targeting event-driven rescheduling in a parallel machine environment with sequence-dependent setup times. The approach integrates the real-time decision-making capabilities of Q-learning with the deep neighborhood optimization of Tabu Search, constructing two hybrid modes: RL-First (Reinforcement Learning-Initialized Tabu Search) and TS-First (Tabu Search-Guided Reinforcement Learning). The objective function adopts a two-stage goal programming design: prioritizing the minimization of total tardiness as the primary goal, and secondarily minimizing changeover counts and costs to enhance operational stability, provided that delivery performance is improved.
This study validates the approach using actual production data from a domestic orthopedic implant manufacturer (17 CNC machines, 687 work orders) and designs six machine failure intensity scenarios. The results indicate that HDSOA reduces total tardiness across all scenarios. Specifically, in Scenario 6 (high-intensity failure), the RL-First mode reduced total tardiness from a Baseline of 791,390 minutes to 286,895 minutes (an improvement of 63.7%) and reduced the number of delayed orders from 141 to 109. Simultaneously, it generated feasible rescheduling solutions within approximately 346.9 seconds, demonstrating the feasibility of real-time response.
The empirical results of this study point out that under severe machine failures, strategic cross-machine order reallocation for load balancing—despite potentially increasing some setup costs—can effectively curb the spread of delays and improve delivery service levels. This provides a quantifiable and robust intelligent rescheduling decision method for the high-end medical device manufacturing field.

1.臺北榮民總醫院關節重建科（2022）。什麼是骨材置換術？一次彙整骨材費用、材質、術後復健等常見問題！ 臺北榮民總醫院。https://www.jrvghtpe.com.tw/article_d.php?lang=tw&tb=2&cid=102&id=1509
2.唐祖湘（2021）。製造業導入新世代製造模式。 工業技術與資訊月刊，354(8)，28-29。
3.黃允成、陳奕安&丁儀安(2019)。應用人工智慧於多機多途程最適生產排程之研究。技術學刊，34(2)，95-107。
4.黃允成&江旻倩(2022)。多產線多段式交期多產品雙屬性生產排程之研究-以某科技工廠為例。技術學刊，37(2)，111-128。
5.健康醫療網（2022）。350萬人膝關節退化痛　七旬伯機器手臂換膝「當天就能站」。健康醫療網。https://www.healthnews.com.tw/article/56185
6.鍾文仁(2024)。智慧製造在塑膠射出模具暨成型場域的挑戰與未來。 機械工業雜誌，(494)，48-57。
7.楊錦洲(2020)。智能製造之闡述。 品質月刊， 56(8)， 6-11。
8.優分析產業數據中心（2024）。聯合骨科的未來，是否重演過去成長趨緩的循環？ https://uanalyze.com.tw/articles/408784407
9.Allahverdi, A., Ng, C. T., Cheng, T. E., & Kovalyov, M. Y. (2008). A survey of scheduling problems with setup times or costs. European Journal of Operational Research, 187(3), 985-1032. https://doi.org/10.1016/j.ejor.2006.06.060
10.Armentano, V. C. A., & Yamashita, D. S. (2000). Tabu search for scheduling on identical parallel machines to minimize mean tardiness. Journal of Intelligent Manufacturing, 11(5), 453-460.https://doi.org/10.1023/A:1008918229511
11.Bahroun, Z., Shamayleh, A., As’ ad, R., & Zakaria, R. (2024). Integrated proactive-reactive tool for dynamic scheduling of parallel machine operations. International Journal of Engineering Business Management, 16, 18479790241301164.https://doi.org/10.1177/18479790241301164
12.Blum, C., & Roli, A. (2003). Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Computing Surveys, 35(3), 268–308.https://doi.org/10.1145/937503.937505
13.Branke, J., Nguyen, S., Pickardt, C. W., & Zhang, M. (2015). Automated design of production scheduling heuristics: A review. IEEE Transactions on Evolutionary Computation, 20(1), 110-124.10.1109/TEVC.2015.2429314
14.Brucker, P., & Schlie, R. (1991). Job-shop scheduling with multi-purpose machines. Computing, 45(4), 369-375https://doi.org/10.1007/BF02238804
15.Chien, C. F., & Lan, Y. B. (2021). Agent-based approach integrating deep reinforcement learning and hybrid genetic algorithm for dynamic scheduling for Industry 3.5 smart production. Computers & Industrial Engineering, 162, 107782.https://doi.org/10.1016/j.cie.2021.107782
16.Coherent Market Insights. (2024). Knee Replacement Market Analysis, 2025–2032. Coherent Market Insights. https://www.coherentmarketinsights.com/market-insight/knee-replacement-market-4328
17.Data Bridge Market Research. (2024). Global artificial knee joint market – industry trends and forecast to 2032. Data Bridge Market Research. https://www.databridgemarketresearch.com/reports/global-artificial-knee-joint-market
18.Ehrgott, M. (2005). Multicriteria optimization. Springer Berlin Heidelberg.https://doi.org/10.1007/3-540-27659-9_9
19.Elyasi, M., Altan, B., Ekici, A., Özener, O. Ö., Yanıkoğlu, İ., & Dolgui, A. (2024). Production planning with flexible manufacturing systems under demand uncertainty. International Journal of Production Research, 62(1-2), 157-170.https://doi.org/10.1080/00207543.2023.2288722
20.Fortune Business Insights. (2024). Knee replacement market size, share & growth report, 2025–2032. Fortune Business Insights. https://www.fortunebusinessinsights.com/industry-reports/knee-replacement-implants-market-101244
21.Fu, Y., Wang, Y., Gao, K., & Huang, M. (2024). Review on ensemble meta-heuristics and reinforcement learning for manufacturing scheduling problems. Computers and Electrical Engineering, 120, 109780.https://doi.org/10.1016/j.compeleceng.2024.109780
22.Glover, F. (1989). Tabu search—Part I. ORSA Journal on Computing, 1(3), 190–206.https://doi.org/10.1287/ijoc.1.3.190
23.Glover, F. (1990). Tabu search—part II. ORSA Journal on Computing, 2(1), 4-32.https://doi.org/10.1287/ijoc.2.1.4
24.Jartnillaphand, P., Mardaneh, E., & Bui, H. T. (2025). A tabu search algorithm for Unspecified Parallel Machine scheduling with shift consideration. Computers & Operations Research, 107151.https://doi.org/10.1016/j.cor.2025.107151
25.Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.https://doi.org/10.1613/jair.301
26.Lai, X., Yue, D., Hao, J. K., & Glover, F. (2018). Solution-based tabu search for the maximum min-sum dispersion problem. Information Sciences, 441, 79-94.https://doi.org/10.1016/j.ins.2018.02.006
27.Lee, D., Park, I. B., & Kim, K. (2024). An incremental learning approach to dynamic parallel machine scheduling with sequence-dependent setups and machine eligibility restrictions. Applied Soft Computing, 164, 112002.https://doi.org/10.1016/j.asoc.2024.112002
28.Logendran, R., McDonell, B., & Smucker, B. (2007). Scheduling unrelated parallel machines with sequence-dependent setups. Computers & Operations Research, 34(11), 3420-3438.https://doi.org/10.1016/j.cor.2006.02.006
29.Medical Device Network. (2024). Knee reconstruction market in Taiwan: market size trends.https://www.medicaldevice-network.com/data-insights/knee-reconstruction-taiwan-market-size/
30.Niroumandrad, N., Lahrichi, N., & Lodi, A. (2024). Learning tabu search algorithms: A scheduling application. Computers & Operations Research, 170, 106751.https://doi.org/10.1016/j.cor.2024.106751
31.Nwasuka, N. C., & Nwaiwu, U. (2024). Computer-based production planning, scheduling and control: A review. Journal of Engineering Research, 12(1), 275-280.https://doi.org/10.1016/j.jer.2023.09.027
32.Ouelhadj, D., & Petrovic, S. (2009). A survey of dynamic scheduling in manufacturing systems. Journal of Scheduling, 12(4), 417-431.https://doi.org/10.1007/s10951-008-0090-8
33.Pinedo, M. L. (2012). Scheduling: Theory, algorithms, and systems (4th ed.). Springer.https://doi.org/10.1007/978-1-4614-2361-4
34.Piska, M., & Urbancova, K. (2022). Advanced machining of joint implant UHMWPE inserts. Machines, 10(11), 1008.https://doi.org/10.3390/machines10111008
35.Ren, F., & Liu, H. (2024). Dynamic scheduling for flexible job shop based on MachineRank algorithm and reinforcement learning. Scientific Reports, 14(1), 29741.https://doi.org/10.1038/s41598-024-79593-8
36.Shahrabi, J., Adibi, M. A., & Mahootchi, M. (2017). A reinforcement learning approach to parameter estimation in dynamic job shop scheduling. Computers & Industrial Engineering, 110, 75-82.https://doi.org/10.1016/j.cie.2017.05.026
37.Statista. (2024). Orthopedic Devices – Taiwan https://www.statista.com/outlook/hmo/medical-technology/medical-devices/orthopedic-devices/taiwan
38.Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). The MIT Press.
39.Tessereau, C., O’Dea, R., Coombes, S., & Bast, T. (2021). Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation. Brain and Neuroscience Advances, 5, 2398212820975634.https://doi.org/10.1177/2398212820975634
40.T’kindt, V., & Billaut, J. C. (2006). Multicriteria scheduling: Theory, models and algorithms (2nd ed.). Springer.
41.Vieira, G. E., Herrmann, J. W., & Lin, E. (2003). Rescheduling manufacturing systems: a framework of strategies, policies, and methods. Journal of Scheduling, 6(1), 39-62.https://doi.org/10.1023/A:1022235519958
42.Wai, W. K., Ting, L., Pang, L. W., Budihardjo, H., Liang, G. C., Liya, D., ... & T-Howe, C. P. (2016). Decision support system for production scheduling (DSSPS). Procedia Computer Science, 96, 315-323.https://doi.org/10.1016/j.procs.2016.08.144
43.Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279-292.https://doi.org/10.1007/BF00992698
44.Watson, J. P., Barbulescu, L., Whitley, L. D., & Howe, A. E. (2002). Contrasting structured and random permutation flow-shop scheduling problems: search-space topology and algorithm performance. INFORMS Journal on Computing, 14(2), 98-123.https://doi.org/10.1287/ijoc.14.2.98.120
45.Xu, H., Huang, L., Tao, J., Zhang, C., & Zheng, J. (2025). A Deep Reinforcement Learning-Based Evolutionary Algorithm for Distributed Heterogeneous Green Hybrid Flowshop Scheduling. Processes, 13(3), 728.https://doi.org/10.3390/pr13030728
46.Yan, J., Zhao, T., Zhang, T., Chu, H., Yang, C., & Zhang, Y. (2024). A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process. Mathematics, 13(1), 4.https://doi.org/10.3390/math13010004
47.Yan, Q., Wang, H., & Wu, F. (2022). Digital twin-enabled dynamic scheduling with preventive maintenance using a double-layer Q-learning algorithm. Computers & Operations Research, 144, 105823.https://doi.org/10.1016/j.cor.2022.105823
48.Zhang, C., Cao, Z., Song, W., Wu, Y., & Zhang, J. (2022). Deep reinforcement learning guided improvement heuristic for job shop scheduling. arXiv preprint arXiv:2211.10936.https://doi.org/10.48550/arXiv.2211.10936
49.Zhang, Y., Zhu, H., Tang, D., Zhou, T., & Gui, Y. (2022). Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems. Robotics and Computer-Integrated Manufacturing, 78, 102412.https://doi.org/10.1016/j.rcim.2022.102412
50.Zhang, Z., Zheng, L., & Weng, M. X. (2007). Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning. The International Journal of Advanced Manufacturing Technology, 34, 968-980.https://doi.org/10.1007/s00170-006-0662-8