本論文旨在開發一套具備基於強化式學習之優化排程演算法以及3D列印失敗偵測演算法之分享式平台，以導入人工智慧達到更好的分享效果。本平台包含了3D列印機、3D列印聯網模組、雲端伺服器三部分，並利用3D列印聯網模組蒐集有關3D列印機的狀態並回傳給雲端伺服器，也可接收雲端伺服器之命令傳送給3D列印機。本論文所提出之基於強化式學習優化排程演算法透過各3D列印聯網模組所回傳之狀態來決定在共享式平台系統靈活、多變的架構下該如何分配任務，並利用演算法取代需人工時間監控3D列印是否失敗的3D列印失敗偵測演算法。本論文使用泊松過程(Poisson process)所產生1,000個任務以測試方法之有效性，共有6個工作種類分別是列印(1) 2小時、(2) 5小時、(3) 11.5小時、(4) 19.5小時、(5) 30小時、(6) 36小時，所測試的機台數量共有8種分別是(1) 5、(2) 10、(3) 15、(4) 20、(5) 25、(6) 30、(7) 40、(8) 50數量的機型。本論文共使用兩種類型之3D列印機進行收案(Prusa I3及Delta Bot)，我們所蒐集的列印影像資料共614張，其中，列印失敗的影像共358張，列印成功的影像共256張，這兩種機型所產生的列印張數各半。研究結果驗證基於強化式學習之優化排程演算法可在不同環境情況下有效學習並不斷優化學習指標，其最佳表現在不同機器數量皆可優於傳統之排程演算法，而對於3D列印失敗偵測演算法在無列印失敗辨識率為80.4%，對於列印失敗辨識率為84.4%，本方法可以即時偵測到3D列印失敗之狀態並減少大量線材的損耗及危險產生。在未來期望能將演算法放置3D列印共享平台上，提升3D列印使用者之便利性。

This thesis aims to develop a 3D printer sharing platform with a reinforcement learning-based scheduling algorithm and a 3D printing failure detection algorithm to embed artificial intelligence for better performance. The platform consists of 3D printers, 3D printing network modules, and a cloud server. The 3D printing network modules collect data/information from the 3D printers and transmit to the cloud server. The commands of the cloud server are sent to the 3D printers with the 3D printing network modules. The reinforcement learning-based scheduling algorithm has been developed to assign printing tasks based on the status of each the 3D printing network module under a flexible and reconfigurable platform architecture. In addition, the 3D printing failure detection algorithm has been proposed in this thesis to continuously monitor the printing tasks and automatically terminate the printing task if failures are detected. The Poisson process was used to generate 1,000 simulated printing tasks for validating the effectiveness of the proposed algorithms. A total of six task categories were applied: (1) 2 hours, (2) 5 hours, (3) 11.5 hours, (4) 19.5 hours, (5) 30 hours, and (6) 36 hours. In the experimental setting, two types of 3D printers: Prusa I3 and Delta Bot, were used, and various numbers of 3D printers that were simulated to connect to the sharing platform include (1) 50, (2) 40, (3) 30, (4) 25, (5) 20, (6) 15, (7) 10, and (8) 5 sets. For the failure detection, a total of 614 images including 358 images with printing failures and 256 images without printing failures (successful printing tasks) were collected. The results show that the reinforcement learning-based scheduling algorithm can effectively converge and outperform the traditional scheduling algorithm under different environmental conditions. The recognition rate for successful printing tasks of the 3D printing failure detection algorithm is 80.4%, while the recognition rate for failed printing tasks is 84.4%. The proposed algorithms is able to detect the failure of 3D printing tasks and thus reduce the loss of filament and the possibility of dangers caused by the failed printing tasks. In the future, it is expected that the algorithm can be placed on a 3D printer sharing platform to improve user convenience of 3D printing.

[1] http://wohlersassociates.com/2016report.htm.
[2] http://www.mse.ncku.edu.tw/index.php?option=module&lang=cht&task=pageinfo&id=639&index=4.
[3] https://all3dp.com/1/top-3d-printing-companies-3d-printer-manufacturers/
[4] D. W. Kim, D. G. Na, and F. F. Chen, “Unrelated parallel machine scheduling with setup times and a total weighted tardiness objective,” Robotics and Computer-Integrated Manufacturing, vol. 19, pp. 173-181, 2003.
[5] J. J. Kanet, and X. Li, “A weighted modified due date rule for sequencing to minimize weighted tardiness,” Journal of Scheduling, vol. 7, pp. 261-276, 2004.
[6] T. E. Morton, and R. V. Rachamadugu, “Myopic heuristics for the single machine weighted tardiness problem,” Technical Report CMU-RI-TR-83-9 Graduate School of Industrial Administration Carnegie-Mellon University , 1983.
[7] A. Volgenant, and E. Teerhuis, “Improved heuristics for the n-job single-machine weighted tardiness problem,” Computers & Operations Research, vol. 26, pp. 35-44, 1999.
[8] D.C. Carroll, “Heuristic sequencing of jobs with single and multiple components,” Ph.D. Thesis, Sloan School of Management, MIT, 1965.
[9] A. P. J. Vepsalainen, and T. E. Morton, “Priority rules for job shops with weighted tardiness costs,” Manage. Sci., vol. 33, pp. 1035-1047, 1987.
[10] V. C. Wang, and John M. Usher, “Application of reinforcement learning for agent-based production scheduling,” Engineering Applications of Artificial Intelligence, vol. 18, pp. 73-82, 2005.
[11] Z. Zhang, L. Zheng, N. Li, W. Wang, S. Zhong, and K. Hu, “Minimizing mean weighted tardiness in unrelated parallel machine scheduling with reinforcement learning,” Comput. Oper. Res., vol. 39, no. 7, pp. 1315-1324, 2012.
[12] http://www.image-net.org/.
[13] N. Dalal, and B. Triggs, “Histograms of oriented gradients for human detection,” Computer Vision and Pattern Recognition, vol. 1, pp. 886-893, 2005.
[14] Z. Sun, G. Bebis, and R. Miller, “On-road vehicle detection: A review,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 5, pp. 694-711, 2006.
[15] S. Sivaraman, and M. M. Trivedi, “A general active-learning framework for on-road vehicle recognition and tracking,” IEEE Trans. Intell. Transp. Syst., vol. 11, no. 2, pp. 267-276, 2010.
[16] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing Atari with Deep Reinforcement Learning,” arXiv:1312.5602v1, pp. 1-9, 2013.
[17] https://www.raspberrypi.org/.
[18] J. F. C. Kingman, Poisson Processes, 1993.
[19] C. Watkins, and P. Dayan, “Q-learning,” Machine Learning, vol. 8, pp. 279-292, 1992.
[20] S. Thrun, and A. Schwartz, “Issues in using function approximation for reinforcement learning,” in Proc. 1993 Connectionist Models Summer School, 1993, pp. 255-263.
[21] H. V. Hasselt, “Double Q-learning,” in Proc. 24th Adv. Neural Inf. Process. Syst. (NIPS), 2010, pp. 2613-2621.
[22] Z. Wang, N. de Freitas, and M. Lanctot, “Dueling network architectures for deep reinforcement learning,” CoRR, vol. abs/1511, pp. 1–15, 2015.
[23] Hessel, and Matteo, “Rainbow: Combining Improvements in Deep Reinforcement Learning,” Proceedings of the AAAI Conference on Artificial Intelligence, 2017.
[24] J. Peng and R. J. Williams, “Incremental multi-step Q-learning,” Machine Learning. Morgan Kaufmann, vol. 22, pp. 226-232, 1994,.
[25] T. Schaul, J. Quan, I. Antonoglou et al., “Prioritized experience replay,” in Proceedings of the IEEE International Conference on Learning Representations, 2016, pp. 1666 - 1676.
[26] Fortunato, and Meire, “Noisy networks for exploration,” In International Conference on Learning Representations (ICLR2018), 2018.
[27] Bellemare, G. Marc, D. Will, and M. Rémi, “A distributional perspective on reinforcement learning,” In ICML, Proceedings of Machine Learning Research, vol. 70, pp. 449-458, 2017.
[28] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Representations, 2015, pp. 1-14 .
[29] C. Cortes and V. Vapnik, “Support-Vector Network,” Machine Learning, vol. 20, pp. 273-297, 1995.
[30] https://www.raspberrypi.com.tw/tag/pi-zero-w/
[31] http://setosa.io/ev/principal-component-analysis/