石油化學工業，以下簡稱石化工業(petrochemical industry)，是一種連續的生產模式。在化工業中，有許多特殊的化學性質以及特殊的生產限制會顯著地影響生產排程的表現以及生產排程的可行性。在實務上，現場人員透過經驗法則以及簡單的派工法則進行排程，但排程績效通常會略差(例如:較長的完工時間或較多的過渡產品)，而且經驗法則只有少數人知道，無法有效地進行知識管理，因此，對於石化工業來說，如何開發一個有效率且能進行知識管理的生產排程是急迫且需要的。
本研究根據石化產業中的生產目標及需求，提出了一張工程經驗參考表來記錄石化工廠的生產特性與限制，以及一個基於該參考表發展的啟發式演算法以產生可行的化工廠生產排程。本研究考慮了多個目標以及限制，其中主要圍繞著三個關鍵生產議題 – 減少產能損失(減少過渡產品的產生)，增加生產良率(與換線換模次數有關)，滿足客戶交期需求。接著提出一項多目標基因演算法內嵌強化學習的技術來解決石化工業的排程問題，定義多目標基因演算法在強化學習中每一次的狀態、動作以及獎勵，藉此得出最佳策略，並運用最佳策略調整基因演算法中每一次迭代的交配率以及突變率，或是選擇不同的交配機制和突變機制，藉此來提升多目標基因演算法的表現。實證研究以台灣石化工業為例，探討其生產排程以驗證所提出的演算法。實證結果顯示所提出的演算法將有效地改善排成程績效，關鍵績效指標(KPI)皆有顯著提升。

Petrochemical industry refers to the industry that uses petroleum or natural gas as the main raw materials, chemical reaction, processing and production to produce various chemical products. In practice, on-site operator conduct scheduling through rule of thumb and the simple rules of dispatching, however the scheduling performance is usually worse (i.e. longer makespan or more transition products). In addition, only a few people know that the rule of thumb that is hard to manage knowledge effectively in the company. This study proposes the engineering experience sheets to characterize the production characteristics and the constraints in petrochemical industry and develop a heuristic algorithm based on the sheets for generating the feasible scheduling. This study focuses on three key production issues – reducing capacity loss (reducing the production of transitional products), enhancing production yield (defective products depending on the number of transitions), and meeting customer delivery requirements. Then proposing a multi-objective genetic algorithm embedded with reinforcement learning technique to solve the scheduling problem. The study defines the state, action, and reward of genetic operators and developed in reinforcement learning to fine tune the hyperparameters or to select the appropriate mechanism of genetic operators in multi-objective genetic algorithm. An empirical study of petrochemical industry was conducted to validate the proposed algorithms. The results show that the multi-objective genetic algorithm embedded with reinforcement learning can provide better performance than the current heuristic algorithm in the petrochemical production scheduling.

Abdi, H. (2007). The Kendall rank correlation coefficient. Encyclopedia of Measurement and Statistics. Sage, Thousand Oaks, CA, 508-510.
Abdullah, S., Shamayleh, A., & Ndiaye, M. (2016). Capacitated lot-sizing and scheduling with sequence-dependent setups in petrochemical plants. Paper presented at the Proceedings of the 2016 International Conference on Industrial Engineering and Operations Management, Kuala Lumpur, Malaysia.
Alipour, M. M., Razavi, S. N., Derakhshi, M. R. F., & Balafar, M. A. (2018). A hybrid algorithm using a genetic algorithm and multiagent reinforcement learning heuristic to solve the traveling salesman problem. Neural Computing and Applications, 30(9), 2935-2951.
Arbiza, M. J., Bonfill, A., Guillén, G., Mele, F. D., Espuna, A., & Puigjaner, L. (2008). Metaheuristic multiobjective optimisation approach for the scheduling of multiproduct batch chemical plants. Journal of Cleaner Production, 16(2), 233-244.
Bellman, R. (1957). Dynamic programming: Princeton univ. press. N・J, 95.
Blömer, F., & Günther, H.-O. (1998). Scheduling of a multi-product batch process in the chemical industry. Computers in industry, 36(3), 245-259.
Blomer, F., & Gunther, H.-O. (2000). LP-based heuristics for scheduling chemical batch processes. International Journal of Production Research, 38(5), 1029-1051.
Chen, Q., Huang, M., Xu, Q., Wang, H., & Wang, J. (2020). Reinforcement Learning-Based Genetic Algorithm in Optimizing Multidimensional Data Discretization Scheme. Mathematical Problems in Engineering, 2020.
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on evolutionary computation, 6(2), 182-197.
Dunford, N., Schwartz, J. T., Bade, W. G., & Bartle, R. G. (1967). Linear operators: General theory: Interscience Publishers.
Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Paper presented at the Kdd.
Graves, S. C. (1981). A review of production scheduling. Operations research, 29(4), 646-675.
Hill, A., Cornelissens, T., & Sörensen, K. (2016). Efficient multi-product multi-BOM batch scheduling for a petrochemical blending plant with a shared pipeline network. Computers & Chemical Engineering, 84, 493-506.
Holland, J. (1975). Adaptation in natural and artificial systems, univ. of mich. press. Ann Arbor.
Howard, R. A. (1960). Dynamic programming and markov processes.
Kondili, E., Pantelides, C. C., & Sargent, R. W. (1993). A general algorithm for short-term scheduling of batch operations—I. MILP formulation. Computers & Chemical Engineering, 17(2), 211-227.
Koza, J. R. (1997). Future work and practical applications of genetic programming. Handbook of Evolutionary Computation, H1.
Lei, D. (2008). A Pareto archive particle swarm optimization for multi-objective job shop scheduling. Computers & Industrial Engineering, 54(4), 960-971.
Liu, F., & Zeng, G. (2009). Study of genetic algorithm with reinforcement learning to solve the TSP. Expert Systems with Applications, 36(3), 6995-7001.
Mihaylov, M., Jurado, S., Avellana, N., Van Moffaert, K., de Abril, I. M., & Nowé, A. (2014). NRGcoin: Virtual currency for trading of renewable energy in smart grids. Paper presented at the 11th International conference on the European energy market (EEM14).
Murata, T., & Ishibuchi, H. (1994). Performance evaluation of genetic algorithms for flowshop scheduling problems. Paper presented at the Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.
Omar, M., Baharum, A., & Hasan, Y. A. (2006). A Job-shop Scheduling Problem (JSSP) Using Genetic Algorithm (GA). Paper presented at the Proceedings of the 2nd im TG T Regional Conference on Mathematics, Statistics and Applications Universiti Sains Malaysia.
Sadegheih, A. (2006). Scheduling problem using genetic algorithm, simulated annealing and the effects of parameter values on GA performance. Applied Mathematical Modelling, 30(2), 147-154.
Sehgal, A., La, H., Louis, S., & Nguyen, H. (2019). Deep reinforcement learning using genetic algorithm for parameter optimization. Paper presented at the 2019 Third IEEE International Conference on Robotic Computing (IRC).
Vamplew, P., Yearwood, J., Dazeley, R., & Berry, A. (2008). On the limitations of scalarisation for multi-objective reinforcement learning of pareto fronts. Paper presented at the Australasian Joint Conference on Artificial Intelligence.
Van Moffaert, K., Drugan, M. M., & Nowé, A. (2013a). Hypervolume-based multi-objective reinforcement learning. Paper presented at the International Conference on Evolutionary Multi-Criterion Optimization.
Van Moffaert, K., Drugan, M. M., & Nowé, A. (2013b). Scalarized multi-objective reinforcement learning: Novel design techniques. Paper presented at the 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).
Van Moffaert, K., & Nowé, A. (2014). Multi-objective reinforcement learning using sets of pareto dominating policies. The Journal of Machine Learning Research, 15(1), 3483-3512.
Van Otterlo, M. (2009). Markov decision processes: concepts and algorithms. Course on ‘Learning and Reasoning.
Wu, C.-M. (2019). An Empirical Study of Two-Phase Multi-Objective Genetic Algorithm for Lithography Scheduling in Semiconductor DRAM Fab. (Master). National Cheng Kung University, Tainan. Retrieved from https://hdl.handle.net/11296/p5tey6
Wu, M.-Y. (2016). Multi-Objective Stochastic Scheduling Optimization: A Study of Auto Parts Manufacturer in Taiwan. (Master). National Cheng Kung University, Tainan. Retrieved from https://hdl.handle.net/11296/4e23b3
Zar, J. H. (2005). Spearman rank correlation. Encyclopedia of Biostatistics, 7.
Zhang, Y., Feng, Y., & Rong, G. (2018). Data-driven rolling-horizon robust optimization for petrochemical scheduling using probability density contours. Computers & Chemical Engineering, 115, 342-360.
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & Da Fonseca, V. G. (2003). Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on evolutionary computation, 7(2), 117-132.