為確保製程穩定以及高良率的產出，工廠均須進行離線品質抽檢。然而此作法僅抽取少數樣本進行量測，並且具有時間延遲的缺點，無法達到即時監控與線上全檢的目標，而採用全自動虛擬量測(Automatic Virtual Metrology, AVM)即能解決以上問題。
但隨著科技的進步，製程亦變得愈加複雜，資料數量也隨之增長。現今的AVM中的預測模組為倒傳遞神經網路(Back Propagation Neural Networks, BPNN)，其為機器學習範疇中的演算法。因此，即使製程資料的數據量增加，其可提升的表現亦有所極限；而深度學習的演算法則不同，其預測的精準度將隨資料量的增加而提升。
為了再精進預測精度，本研究提出一個新的AVM架構，其中所使用的預測演算法是基於深度學習中之卷積神經網路(Convolutional Neural Networks, CNN)。透過半導體產業大量資料進行多次實驗，本研究驗證了所提出之CNN架構可萃取資料特徵並以此提升了AVM的精確程度，打破以BPNN為基之AVM精度極限。
與此同時，本研究亦發展了一新的信心指標模組，用以評估CNN-based AVM預測值的可靠程度。其原理是比較CNN-based AVM所採用的之兩種演算法的預測值之相似程度，並以此換算為信心指標。透過該指標與上述的預測模組相輔相成，能更加實現產品即時監控與可靠之線上全檢的目標。

To ensure stable manufacturing and high-yield of production, factories conduct quality inspection by adopting sampling inspection, which fails to achieve real-time and online total inspection because of the sampling strategy and metrology delay. Automatic Virtual Metrology (AVM) is by far the best solution to tackle with the above-mentioned problem, as it can convert sampling inspection with metrology delay into on-line and real-time total inspection.
However, with the advancement of technology, the processes become more sophisticated, and the requirement for the accuracy of virtual metrology becomes higher as well. The current AVM prediction algorithm is a traditional machine learning method called Back-Propagation Neural Networks (BPNN). However, even if the amount of data in this method increases, the performance has its limits of improvement. To improve the prediction accuracy, this work proposes the deep learning method, Convolutional Neural Networks (CNN), for the AVM server. The accuracy of CNN improves as the amount of data grows. In other words, if there are sufficient data, the current accuracy limit of machine learning can be eliminated. Experimental results reveal that CNN can automatically extract highly informative features from the data and improve the original AVM accuracy.
Meanwhile, this research develops a new method to evaluate the reliability of the CNN-based AVM predicted value. The principle is to compare the similarity of the two predictive algorithms utilized in the CNN-based AVM and convert it into a reliance indicator. With this indicator and the CNN forecasting module, the goal of real-time product monitoring and online reliable total inspection can be achieved.

[1] F.-T. Cheng, H.-C. Huang, and C.-A. Kao, “Developing an automatic virtual metrology system”, IEEE Trans. on Automation Science and Engineering, vol. 9, no. 1, pp. 181-188, Jan. 2012.
[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks”, Proc. Neural Inf. Process. Syst., 2012, pp. 1097–1105.
[3] C. Ding and D. Tao, “Trunk-Branch ensemble convolutional neural networks for video-based face recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 1002-1014, 1 April 2018.
[4] Y. S. Chernyshova, A. V. Sheshkus and V. V. Arlazarov, “Two-step CNN framework for text line recognition in camera-captured images”, IEEE Access, vol. 8, pp. 32587-32600, 2020.
[5] K. B. Lee, S. Cheon and C. O. Kim, “A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes”, IEEE Transactions on Semiconductor Manufacturing, vol. 30, no. 2, pp. 135-142, May 2017.
[6] G. Wen, Z. Gao, Q. Cai, Y. Wang, and S. Mei, “A novel method based on deep convolutional neural networks for wafer semiconductor surface defect inspection”, IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 12, pp. 9668-9680, Dec. 2020.
[7] M. Maggipinto, M. Terzi, C. Masiero, A. Beghi, and G. A. Susto, “A computer vision-Inspired deep learning architecture for virtual metrology modeling with 2-dimensional data”, IEEE Transactions on Semiconductor Manufacturing, vol. 31, no. 3, pp. 376-384, Aug. 2018.
[8] F. Itakura, “Minimum prediction residual principle applied to speech recognition”, IEEE Trans. Acoust. Speech Signal Process., vol. 23, no. 1, pp. 67–72, Feb. 1975.
[9] H. Sakoe and S. Chiba, “Dynamic programming algorithm optimization for spoken word recognition”, IEEE Trans. Acoust. Speech Signal Process., vol. 26, no. 1, pp. 43–49, Feb. 1978.
[10] J. F. Lichtenauer, E. A. Hendriks, and M. J. T. Reinders, “Sign language recognition by combining statistical DTW and independent classification”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 2040–2046, Nov. 2008.
[11] Y.-M. Hsieh, R. Lu, J.-W. Lu, F.-T. Cheng, and M. Adnan, “Automated Classification Scheme plus AVM for Wafer Sawing Processes”, IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4525-4532, July 2020.
[12] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition”, in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov 1998.
[13] Q. Li, X. Zhang, A. Rigat, and Y. Li, “Parameters Optimization of Back Propagation Neural Network Based on Memetic Algorithm Coupled with Genetic Algorithm”, 2015 IEEE 12th Intl Conf on Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom), 2015, pp. 1359-1364.
[14] F.-T. Cheng, Y.-T. Chen, Y.-C. Su, and D.-L. Zeng, “Evaluating Reliance Level of a Virtual Metrology System”, IEEE Transactions on Semiconductor Manufacturing, vol. 21, no. 1, pp. 92-103, February 2008.
[15] Y. -M. Hsieh, T. -J. Wang, C. -Y. Lin, L. -H. Peng, F. -T. Cheng and S. -Y. Shang, “Convolutional Neural Networks for Automatic Virtual Metrology, ” in IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5720-5727, July 2021, doi: 10.1109/LRA.2021.3084882.