隨著人工智慧 （Artificial Intelligence, AI）的日益蓬勃，AI應用也實際在各界付諸實現並大放異彩。而AI技術在工業製造的產業升級中也扮演著舉足輕重的地位，尤其是在異常檢測 （Anomaly Detection, AD）方面，異常檢測技術對於提高產品品質與操作安全至關重要。然而，許多現有方法雖然在準確性方面取得了顯著進展，但在資源消耗與處理時間方面仍存在缺點。例如：基於記憶體的異常檢測PatchCore與耦合超球體特徵適應法（Coupled-hypersphere-based Feature Adaptation, CFA），雖然性能卓越，但需要大量的計算資源與時間成本。近年來，模型壓縮的一項熱門技術──知識蒸餾（Knowledge Distillation, KD），是抽取複雜模型訓練出的輸出分佈與中間層特徵表示為另一簡單模型所用，已經在無監督異常檢測（Unsupervised Anomaly Detection, UAD）這具有挑戰性的任務上取得了有希望的結果。我們重新審視師生反向蒸餾（Reverse Distillation, RD）技術，並引入改良方法，即師生重訪反向蒸餾（Revisiting Reverse Distillation, RRD）技術。 RRD 保留了反向蒸餾的低延遲優勢，並通過多任務學習進一步提升其表現，在推論速度與資源使用上均優於現有的先進方法。在MVTec AD資料集中， RRD 在異常檢測任務中比 PatchCore 快六倍、比 CFA 快兩倍，而延遲時間與 RD 基本持平。 RRD 的核心創新包含特徵緊密性和異常訊號抑制兩個方面。首先，我們引入了自監督最佳傳輸方法來增強特徵緊密性，確保正常樣本之間的特徵表示更加緊湊，其次，我們通過使用單形噪聲模擬虛擬異常樣本，並最小化重建損失來抑制異常訊號。這些改進使得 RRD 在多個資料集上的表現均優於現有方法，並展示其在不同領域的廣泛適用性。我們在多個公共資料集包括MVTec AD、BTAD（BeanTech Anomaly Detection）、KSDD2（KolektorSDD2）與東台精機公司提供的客戶資料集TTCI（TongTai Cast Iron）上進行廣泛的實驗，特別的是，客戶資料集的像素級異常瑕疵位置由本實驗室親自標註。結果表明， RRD 在異常檢測與定位方面均達到最新的性能，舉例像是在MVTec AD資料集上， RRD 在 Pixel AUPRO 指標上相較於 RD 提升了約 1.32 %，平均達到了 94.17%，而在 TTCI 資料集上， RRD 的 Pixel AUPRO 指標相較於 RD 也提升了約 1.2%，平均達到了 85.5%，超越了其他方法。此外， RRD 方法在實時性方面至少比最新的對手快兩倍，使其成為實際上異常檢測應用中極具潛力的方法。

With the rapid growth of Artificial Intelligence (AI), its applications have been widely implemented and have made a significant impact across various sectors. In particular, AI technology plays a critical role in the industrial manufacturing sector, especially in Anomaly Detection (AD), where it is essential for improving product quality and operational safety. However, while many existing methods have achieved significant progress in terms of accuracy, they still suffer from drawbacks in terms of resource consumption and processing time. For instance, memory-based anomaly detection methods such as PatchCore and Coupled-hypersphere-based Feature Adaptation (CFA), although highly effective, require substantial computational resources and time. In recent years, a popular model compression technique known as Knowledge Distillation (KD) has shown promising results in the challenging task of Unsupervised Anomaly Detection (UAD) by extracting the output distribution and intermediate feature representations of a complex model for use in a simpler model. We revisited the Reverse Distillation (RD) technique and introduced an improved method, Revisiting Reverse Distillation (RRD). RRD retains the low-latency advantage of RD while further enhancing its performance through multi-task learning, achieving superior inference speed and resource efficiency compared to existing state-of-the-art methods. In the MVTec AD dataset, RRD outperformed PatchCore by six times and CFA by two times in anomaly detection tasks, with a latency comparable to that of RD. The core innovations of RRD include two aspects: feature compactness and anomaly signal suppression. Firstly, we introduced a self-supervised optimal transport method to enhance feature compactness, ensuring more compact feature representations among normal samples. Secondly, we simulated virtual anomaly samples using simplex noise and minimized reconstruction loss to suppress anomaly signals. These improvements enabled RRD to outperform existing methods across multiple datasets and demonstrate its broad applicability in various domains. We conducted extensive experiments on multiple public datasets, including MVTec AD, BTAD (BeanTech Anomaly Detection), KSDD2 (KolektorSDD2), and a customer dataset provided by TongTai Cast Iron (TTCI), where the pixel-level anomaly defect locations were manually annotated by our laboratory. The results indicate that RRD achieved state-of-the-art performance in both anomaly detection and localization. For example, in the MVTec AD dataset, RRD improved the Pixel AUPRO metric by approximately 1.32% compared to RD, reaching an average of 94.17%. In the TTCI dataset, RRD also improved the Pixel AUPRO metric by approximately 1.2%, achieving an average of 85.5%, surpassing other methods. Additionally, the RRD method demonstrated at least twice the real-time efficiency of the latest competitors, making it a highly promising approach for practical anomaly detection applications.

[1] A. Bochkovskiy, C.Y. Wang, and H.Y.M. Liao. "Yolov4: Optimal speed and accuracy of object detection," in arXiv:2004.10934, 2020.
[2] Y. Hu, Y. Chen, X. Li, and J. Feng. "Dynamic feature fusion for semantic edge detection," in arXiv:1902.09104, 2019.
[3] D. Acuna, A. Kar, and S. Fidler. "Devil is in the edges: Learning semantic boundaries from noisy annotations," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11075-11083, 2019.
[4] R. Wu, M. Feng, W. Guan, D. Wang, H. Lu, and E. Ding. "A mutual learning method for salient object detection with intertwined multi-supervision," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 8150-8159, 2019.
[5] Y. Pang, X. Zhao, L. Zhang, and H. Lu. "Multi-scale interactive network for salient object detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9413-9422, 2020.
[6] C.Y. Wang, A. Bochkovskiy, and H.Y.M. Liao. "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7464-7475, 2022.
[7] G. Wang, S. Han, E. Ding, and D. Huang. "Student-Teacher Feature Pyramid Matching for Anomaly Detection," in the British Machine Vision Conference, arXiv:2103.04257, 2021.
[8] J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. Yuille, and T. Kong. "iBOT: Image BERT Pre-Training with Online Tokenizer," in The International Conference on Learning Representations, arXiv:2111.07832, 2022.
[9] H. Deng and X. Li. "Anomaly Detection via Reverse Distillation from One-Class Embedding," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9737-9746, 2022.
[10] H. Bao, L. Dong, S. Piao, and F. Wei. "BEiT: BERT Pre-Training of Image Transformers," in The International Conference on Learning Representations, arXiv:2106.08254, 2022.
[11] Z. Peng, L. Dong, H. Bao, Q. Ye, and F. Wei. "BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers," in The International Conference on Learning Representations, in arXiv:2208.06366, 2023.
[12] X. Wen, W. Zhao, Z. Yu, J. Zhao and K. Song. "A Novel Anomaly Detection Method for Strip Steel Based on Multi-Scale Knowledge Distillation and Feature Information Banks Network," Coatings (2079-6412) 13.7, 2023.
[13] T.D. Tien, A.T. Nguyen, N.H. Tran, T.D. Huy, S.T.M. Duong, C.D.T. Nguyen, and S.Q.H. Truong. "Revisiting Reverse Distillation for Anomaly Detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24511-24520, 2023.
[14] W. Wang, H. Bao, L. Dong, J. Bjorck, Z. Peng, Q. Liu, K. Aggarwal, O.K. Mohammed, S. Singhal, S. Som, and F. Wei. "Image as a Foreign Language: BEiT (V3) Pretraining for Vision and Vision-Language Tasks," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19175-19186, 2023.
[15] T.T. Nguyen and A.T. Luu. "Improving Neural Cross-Lingual Abstractive Summarization via Employing Optimal Transport Distance for Knowledge Distillation," in Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11103-11111, 2022.
[16] P. Bergmann, M. Fauser, D. Sattlegger and C. Steger. "MVTec AD--A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9592-9600, 2019.
[17] K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox and P. Gehler. "Towards Total Recall in Industrial Anomaly Detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318-14328, 2022.
[18] S. Lee, S. Lee and B.C. Song. "CFA: Coupled-Hypersphere-Based Feature Adaptation for Target-Oriented Anomaly Localization," IEEE Access, 10, 78446-78454, 2022.
[19] P. Chen, S. Liu, H. Zhao and J. Jia "Distilling Knowledge via Knowledge Review," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5008-5017, 2021.
[20] M. Salehi, N. Sadjadi, S. Baselizadeh, M.H. Rohban and H.R. Rabiee "Multiresolution Knowledge Distillation for Anomaly Detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14902-14912, 2021.
[21] A. Abdusalomov and T.K. Whangbo "Detection and Removal of Moving Object Shadows Using Geometry and Color Information for Indoor Video Streams," Applied Sciences, 9(23), 5165, 2019.
[22] S. Shah and J. Tembhurne "Object Detection Using Convolutional Neural Networks and Transformer-Based Models: A Review," Journal of Electrical Systems and Information Technology, 10(1), 54, 2023.
[23] P. Bergmann, M. Fauser, D. Sattlegger and C. Steger "Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4183-4192, 2020.
[24] J. Wyatt, A. Leach, S.M. Schmon and C.G. Willcocks "AnoDDPM: Anomaly Detection with Denoising Diffusion Probabilistic Models Using Simplex Noise," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 650-656, 2022.
[25] J. Deng, W. Dong, R Socher, L.J. Li, K Li and L. Fei-Fei "Imagenet: A Large-Scale Hierarchical Image Database," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.