隨著卷積神經網絡（CNNs）的逐漸成熟，許多高性能模型已被開發出來。然而，這些模型往往需要大量的計算資源，使得它們難以直接應用於計算能力有限的嵌入式設備上。為了解決這一問題，許多研究轉向探索輕量化模型的策略，包括量化、剪枝和知識蒸餾。這些策略在減小網絡大小方面取得了一定的成功，但往往涉及到模型結構的複雜調整，並需要相當的時間來驗證輕量化模型的性能。

輕量化卷積結構作為輕量化CNN模型的主要選擇之一，目前缺乏一種基於目標設備可用的計算資源，通過這些輕量化卷積結構快速重新組合CNN模型的方法。

本研究提出了一種創新的方法「膨脹法」，它利用輕量化卷積結構針對目標設備可用計算資源有效地重新創建CNN模型，在最大化使用設備計算資源的同時盡可能地保持原始模型的準確度。這項研究的結果不僅證明了這種方法可以迅速地使CNN模型輕量化，而且還證明了重新創建的模型的性能接近於原始模型的性能。

With the gradual maturation of Convolutional Neural Networks (CNNs), many high-performance models have been developed. However, these models often require substantial computational resources, making them difficult to apply directly on embedded devices with limited computing power. To address this issue, many studies have shifted towards exploring strategies for lightweight models, including quantization, pruning, and knowledge distillation. These strategies have been somewhat successful in reducing the size of the networks, but often involve complex adjustments to the model's structure and require considerable time to validate the performance of the lightweight models.

Lightweight convolutional structures, as one of the primary choices for lightweight CNN models, currently lack a method for quickly reconfiguring CNN models through these lightweight convolutional structures based on the computational resources available to the target device.

This study proposes an innovative method called "Expansion Technique" which leverages lightweight convolutional structures to effectively recreate CNN models tailored to the computational resources available on target devices, while striving to maintain the accuracy of the original model as much as possible and maximizing the use of the device's computational resources. The results of this research not only demonstrate that this method can quickly lighten CNN models but also show that the performance of the recreated models closely approximates that of the original models.

[1] L. Alzubaidi, J. Zhang, A. J. Humaidi, et al. Review of deep learning: concepts, cnn architectures, challenges, applications, future directions. Journal of Big Data, 8(53), 2021.
[2] Sathiyabhama Balasubramaniam, Yuvarajan Velmurugan, Dhayanithi Jaganathan, and Seshathiri Dhanasekaran. A modified lenet cnn for breast cancer diagnosis in ultrasound images. Diagnostics, 13(17), 2023.
[3] Simon Becker, Yao Zhang, and Alpha A. Lee. Geometry of energy landscapes and the optimizability of deep neural networks. Physical Review Letters, 124(10), March 2020.
[4] Sam Buchanan, Dar Gilboa, and John Wright. Deep networks and the multiple manifold problem, 2021.
[5] Miaomiao Cao, Victor O. K. Li, and Vincent W. S. Chan. A cnn-lstm model for traffic speed prediction. In 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), pages 1–5, 2020.
[6] Roger Creus Castanyer, Silverio Martínez-Fernández, and Xavier Franch. Integration of convolutional neural networks in mobile applications, 2021.
[7] François Chollet. Xception: Deep learning with depthwise separable convolutions. CoRR, abs/1610.02357, 2016.
[8] Chuntao Ding, Zhichao Lu, Felix Juefei-Xu, Vishnu Naresh Boddeti, Yidong Li, and Jiannong Cao. Towards transmission-friendly and robust cnn models over cloud and device. IEEE Transactions on Mobile Computing, 22(10):6176–6189, October 2023.
[9] Huu-Thanh Duong, Viet-Tuan Le, and Vinh Truong Hoang. Deep learning-based anomaly detection in video surveillance: A survey. Sensors, 23(11), 2023.
[10] Fabian Eitel, Marc-André Schulz, Moritz Seiler, Henrik Walter, and Kerstin Ritter. Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research. Experimental Neurology, 339:113608, May 2021.
[11] Victor V. Erokhin and Elena V. Eliseeva. Influence of neural network architecture on its performance. SOFT MEASUREMENTS AND COMPUTING, 2022.
[12] Anastasios Fanariotis, Theofanis Orphanoudakis, Konstantinos Kotrotsios, Vassilis Fotopoulos, George Keramidas, and Panagiotis Karkazis. Power efficient machine learning models deployment on edge iot devices. Sensors, 23(3), 2023.
[13] H. Gang, S. Guanglei, W. Xiaofeng, et al. Ccnnet: a novel lightweight convolutional neural network and its application in traditional chinese medicine recognition. Journal of Big Data, 10:114, 2023. 33
[14] Eleonora Grassucci, Aston Zhang, and Danilo Comminiello. Phnns: Lightweight neural networks via parameterized hypercomplex convolutions. IEEE Transactions on Neural Networks and Learning Systems, pages 1–13, 2022.
[15] Yuanyuan Guo, Yifan Xia, Jing Wang, Hui Yu, and Rung-Ching Chen. Real-time facial affective computing on mobile devices. Sensors, 20(3), 2020.
[16] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn, 2018.
[17] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015.
[18] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR, abs/1704.04861, 2017.
[19] Ming Jiang and Li Zhang. An interactive evolution strategy based deep convolutional generative adversarial network for 2d video game level procedural content generation. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–6, 2021.
[20] Ali Farouk Khalifa, Hesham N. Elmahdy, and Eman Badr. Real-time human detection model for edge devices, 2021.
[21] Amin Khatami, Asef Nazari, Amin Beheshti, Thanh Thi Nguyen, Saeid Nahavandi, and Jerzy Zieba. Convolutional neural network for medical image classification using wavelet features. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2020.
[22] Kyungho Kim, Sung-Joon Jang, Jonghee Park, Eunchong Lee, and Sang-Seol Lee. Lightweight and energy-efficient deep learning accelerator for real-time object detection on edge devices. Sensors, 23(3), 2023.
[23] Yoon Kim. Convolutional neural networks for sentence classification, 2014.
[24] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets, 2017.
[25] Zewen Li, Fan Liu, Wenjie Yang, Shouheng Peng, and Jun Zhou. A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 33(12):6999–7019, 2022.
[26] Tailin Liang, John Glossner, Lei Wang, Shaobo Shi, and Xiaotong Zhang. Pruning and quantization for deep neural network acceleration: A survey. Neurocomputing, 461:370–403, 2021.
[27] Kun-Yu Lin, Pei-Yi Liu, Po-Kai Wang, Chih-Lin Hu, and Ying Cai. Predicting road traffic risks with cnn-and-lstm learning over spatio-temporal and multi-feature traffic data. In 2023 IEEE International Conference on Software Services Engineering (SSE), pages 305–311, 2023.
[28] Gangzhao Lu, Weizhe Zhang, and Zheng Wang. Optimizing depthwise separable convolution operations on gpus. IEEE Transactions on Parallel and Distributed Systems, 33(1):70–87, 2022. 34
[29] Aoife McDonagh, Joseph Lemley, Ryan Cassidy, and Peter Corcoran. Synthesizing game audio using deep neural networks. In 2018 IEEE Games, Entertainment, Media Conference (GEM), pages 1–9, 2018.
[30] Saad Naeem, Noreen Jamil, Habib Khan, and Shah Nazir. Complexity of deep convolutional neural networks in mobile computing. Complexity, 2020, 09 2020.
[31] Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization, 2021.
[32] Puvvada Nagesh, L. Akhil, K. Rishi, and T. Sai Bhargav. Traffic signs recognition using cnn and keras. In 2023 International Conference on Innovative Data Communication Technologies and Application (ICIDCA), pages 221–224, 2023.
[33] Quynh N. Nguyen and Matthias Hein. The loss surface and expressivity of deep convolutional neural networks. ArXiv, abs/1710.10928, 2017.
[34] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans, 2017.
[35] Keiron O’Shea and Ryan Nash. An introduction to convolutional neural networks, 2015.
[36] Taiwo R. Oyedare, Vijay Shah, Daniel Jakubisin, and Jeffrey Reed. Keep it simple: Cnn model complexity studies for interference classification tasks, 03 2023.
[37] Shashank Parmar and Rahul. Chess piece classification via convolutional neural networks. In 2023 International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS), pages 36–42, 2023.
[38] Reyhan Patria, Sean Favian, Anggoro Caturdewa, and Derwin Suhartono. Cheat detection on online chess games using convolutional and dense neural network. In 2021 4th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), pages 389–395, 2021.
[39] Xuan Qi, Zegang Sun, Xue Mei, and Ryad Chellali. A lightweight binarized convolutional neural network model for small memory and low-cost mobile devices. Mobile Information Systems, 2023, 04 2023.
[40] Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. On the expressive power of deep neural networks, 2017.
[41] M. J. Rasch, C. Mackin, M. Le Gallo, et al. Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators. Nature Communications, 14(5282), 2023.
[42] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection, 2016.
[43] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors. Nature, 323:533–536, 1986.
[44] Farhana Sultana, Abu Sufian, and Paramartha Dutta. Advancements in image classification using convolutional neural network. In 2018 Fourth International Conference 35 on Research in Computational Intelligence and Communication Networks (ICRCICN). IEEE, November 2018.
[45] Mohammad Mustafa Taye. Theoretical understanding of convolutional neural network: Concepts, architectures, applications, future directions. Computation, 11(3), 2023.
[46] W Voon, YC Hum, YK Tee, WS Yap, MIM Salim, TS Tan, H Mokayed, and KW Lai. Performance analysis of seven convolutional neural networks (cnns) with transfer learning for invasive ductal carcinoma (idc) grading in breast histopathological images. Sci Rep, 12(1):19200, 2022.
[47] Vincent Wilmet, Sauraj Verma, Tabea Redl, Håkon Sandaker, and Zhenning Li. A comparison of supervised and unsupervised deep learning methods for anomaly detection in images, 07 2021.
[48] Lingling Xu, Haoran Xie, Si-Zhao Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameterefficient fine-tuning methods for pretrained language models: A critical review and assessment, 2023.
[49] A. Yana. Application of the deep pretrained language model processing method in social network sentiment analysis. Mathematical Problems in Engineering, 2022.
[50] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-toimage translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2242–2251, 2017.