能源預測在能源管理中扮演著至關重要的角色。準確的能源預測不僅是能源管理系統制定有效決策的基礎，也是進行電網規劃和廠務管理等關鍵模組的前提。未來能源需求的精確預測使得能源供應者能夠更有效地調配資源、管理供應風險、並制定應對策略，從而在保障能源供應的穩定性和可持續性的同時，優化經濟效益和環境影響。在能源預測的方法上，以循環神經網路中的長短期記憶(LSTM)表現最佳，但LSTM可能無法有效識別和利用時間序列中的所有關鍵時間點，因此加入注意力機制至LSTM，透過賦予模型對輸入資料中特定部分更多的「關注」能力，而其關鍵參數—注意力機制的時間長度會大大影響模型之預測表現，但大多數研究並無提及此關鍵時間長度之設定值為多少。因此，如何設計一個可以針對識別長時間序列中關鍵時間點進行預測，且具有最佳週期的第三代全自動虛擬量測模組即為本研究之重點。
本文以AVMLSTM為基礎提出了最佳週期性注意力之長短期記憶(Best Period Attention-Long Short-Term Memory, BPA-LSTM)，透過AVMLSTM理論基礎之週期計算器(Period Calculator, PC)來自動尋找注意力層關鍵參數—時間長度來建立AVMBPA-LSTM模型，以增強模型對時間序列資料中週期性變化和長期依賴關係的捕捉能力。由兩組案例分析實驗結果可以得知：1) AVMBPA-LSTM模型在能耗預測上相較於AVMLSTM模型表現出更高的預測精度和穩定性；2) 以PC自動尋找的時間長度建立之AVMBPA-LSTM模型之表現皆為最佳。

Energy consumption prediction plays a crucial role in energy management systems (EMSs). Accurate energy predictions are fundamental for EMSs to make effective decisions and serve as basis for key activities such as electrical grid planning and facility management.
Among various energy forecasting methods, Long Short-Term Memory (LSTM) networks within the recurrent neural networks show the best performance. However, LSTM may not identify and utilize all critical time steps in a time series effectively. Thus, integrating attention mechanisms into LSTM grants the model the ability to ‘focus’ more on specific parts of the input data, thereby enhancing its performance. The key parameter—the time steps of the attention mechanism—affects the predictive accuracy of the model significantly. Therefore, this research focuses on designing a third generation Automatic Virtual Metrology (AVMIII) module with optimal periodicity that identifies and predicts critical time steps from long time series data.
This paper proposes an optimal periodic attention-based Long Short-Term Memory model, AVMBPA-LSTM, based on AVMLSTM. By using the Period Calculator (PC) from AVMLSTM to automatically identify critical parameters—time steps for the attention layer—the model enhances its ability to capture periodic changes and long-term dependencies in time series data. The experimental results from the two cases indicate: 1) the AVMBPA-LSTM model demonstrates higher prediction accuracy and certainty in energy consumption forecasting compared to the AVMLSTM; and 2) the AVMBPA-LSTM built on time steps identified by the PC can achieve optimal performance.

[1]H. Tieng, T.-C. Ou, T.-H. Tsai, Y.-Y. Li, M.-H. Hung, and F.-T. Cheng, “I4.2-GiM: A Novel Green Intelligent Manufacturing Framework for Net Zero”, early access (December, 2023), IEEE Transactions on Automation Science and Engineering.
[2]F.-T. Cheng, C.-A. Kao, C.-F. Chen, and W.-H. Tsai, “Tutorial on Applying the VM Technology for TFT-LCD Manufacturing, IEEE Transactions on Semiconductor Manufacturing, vol. 28, no. 1, pp. 55-69, 2015. doi:10.1109/TSM.2014.2380433.
[3]Y. -M. Hsieh, T. -J. Wang, C. -Y. Lin, L. -H. Peng, F. -T. Cheng and S. -Y. Shang, "Convolutional Neural Networks for Automatic Virtual Metrology," in IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5720-5727, July 2021, doi: 10.1109/LRA.2021.3084882.
[4]G. Brauwers and F. Frasincar, "A General Survey on Attention Mechanisms in Deep Learning," in IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3279-3298, 1 April 2023, doi: 10.1109/TKDE.2021.3126456.
[5]Z. Niu, G. Zhong, and H. Yu, "A review on the attention mechanism of deep learning," Neurocomputing, vol. 452, pp. 48-62, 2021.
[6]X. Wen and W. Li, "Time Series Prediction Based on LSTM-Attention-LSTM Model," in IEEE Access, vol. 11, pp. 48322-48331, 2023, doi: 0.1109/ACCESS.2023.3276628.
[7]T. Yang, B. Li and Q. Xun, "LSTM-Attention-Embedding Model-Based Day-Ahead Prediction of Photovoltaic Power Output Using Bayesian Optimization," in IEEE Access, vol. 7, pp. 171471-171484, 2019, doi: 10.1109/ACCESS.2019.2954290.
[8]H. Zhou, Y. Zhang, L. Yang, Q. Liu, K. Yan and Y. Du, "Short-Term Photovoltaic Power Forecasting Based on Long Short Term Memory Neural Network and Attention Mechanism," in IEEE Access, vol. 7, pp.7806378074, 2019, doi:10.1109/ACCESS.2019.2923006.
[9]F.-T. Cheng, H.-C. Huang, and C.-A. Kao, “Developing an automatic virtual metrology system”, IEEE Trans. on Automation Science and Engineering, vol. 9, no. 1, pp. 181-188, Jan. 2012.
[10]S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 15 Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
[11]S.-H. Fan, "Application of LSTM-based AVM in Energy Forecasting," M.S. thesis, National Cheng Kung Univ., Tainan, Taiwan, 2023. [Online]. Available: https://hdl.handle.net/11296/k6n5g2.