簡易檢索 / 詳目顯示

回結果列表
研究生： 李俊德
Li, Jun-De
論文名稱： 針對資源匱乏邊緣運算裝置之深度學習推論框架設計與實作
Design and Implementation of a Deep Learning Inference Framework for Resource-Scarce Edge Devices
指導教授： 張大緯
Chang, Da-Wei
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 資訊工程學系
Department of Computer Science and Information Engineering
論文出版年： 2020
畢業學年度： 109
語文別： 英文
論文頁數： 40
中文關鍵詞： 嵌入式系統 、邊緣運算 、記憶體使用量 、深度學習 、機器學習 、人工智慧
外文關鍵詞： Embedded System, Edge Computing, Memory Footprint, Deep Learning, Machine Learning, Artificial Intelligence
相關次數： 點閱：300  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 近年來，機器學習和深度學習已經成為一種受歡迎的技術。人們相信結合機器學習與邊緣運算可以帶來許多好處。這些好處包括更好的隱私性以及較低的網路延遲和耗能。但是，為了結合機器學習與邊緣運算將會出現許多新的挑戰。例如，有限的記憶體空間以及佈署的困難。在這篇論文中，我們基於 TFLite for microcontrollers 提出一個名為 μInference 的新框架來解決上述問題。這是一個讓開發者可以在邊緣運算裝置或是其他嵌入式裝置上開發機器學習應用程式的框架。跟現有的 TFLite for microcontrollers 相比之下，μInference 降低了整體的記憶體使用量。除此之外，μInference 可以更輕鬆的完成應用程式的佈署。

    Machine learning and deep learning have become a growing trend in recent years, and people believe that machine learning can be combined with edge computing for many benefits, including privacy, lower network latency, and lower energy consumption. However, there are many issues that must be addressed when combining machine learning with edge computing, such as limited memory and difficulties related to deployment. In this paper, we propose a new framework called μInference to solve the above problems. μInference is based on TFLite for microcontrollers. It is a framework designed to run machine learning applications on microcontrollers or other edge devices. μInference improved the memory footprint of existing TFLite for microcontrollers and can be deployed more easily.

    摘要 i ABSTRACT ii 致謝 iii TABLE OF CONTENTS iv LIST OF TABLES vii LIST OF FIGURES viii Chapter I – INTRODUCTION 1 Chapter II – BACKGROUND 4 A. Deep Learning 4 B. Edge Computing 4 C. The Challenges of Edge AI 5 Chapter III – RELATED WORK 7 A. Artificial Intelligence with Edge Computing 7 B. Tensorflow Lite for Microcontroller 8 1) Overview 8 2) Model File of Tensorflow Lite for Microcontroller 10 3) Interpreter 12 4) Allocator 12 5) Operator Resolver 13 6) Memory Planner 14 7) Performing Inference 14 Chapter IV – DESIGN AND IMPLEMENTATION 15 A. Overview 15 1) The Choice of Target Hardware 16 2) FreeRTOS 16 3) FatFS 16 4) LwIP 17 B. Deep Learning Application 17 1) PrepareInference() 18 2) GetModelInput() and GetModelOutput() 18 3) DoInference() 18 4) GetICBState() 18 C. Preparation Task 18 1) Program Execution Flow 19 2) Operator Library 20 3) Inference Control Block (ICB) 21 4) Model Format 23 5) Memory Allocation for Tensors 24 D. Network Packet Receiver 26 E. Inference Task 26 1) Program Execution Flow 26 2) Dynamically Loading Read-only Tensor Data 27 Chapter V – EVALUATION 29 A. Experimental Environment and Workloads 29 B. Memory Footprint 30 C. Performance 32 CHAPTER VI – CONCLUSION AND FUTURE WORK 36 REFERENCES 37

    [1] S. Branco, A. G. Ferreira, and J. Cabral, “Machine learning in resource-scarce embedded systems, FPGAs, and end-devices: A survey,” Electronics, vol. 8, no. 11, p. 1289, Nov. 2019.
    [2] J. Kaur, and S. R. N. Reddy, “Operating systems for low-end smart devices: a survey and a proposed solution framework,” Int. J. Inf. Tecnol., vol. 10, no. 1, pp. 49-58, Mar. 2018, DOI: 10.1007/s41870-017-0044-5.
    [3] A. Sehgal, V. Perelman, S. Kuryla, and J. Schonwalder, ‘‘Management of resource constrained devices in the Internet of Things,’’ IEEE Commun. Mag., vol. 50, no. 12, pp. 144-149, Dec. 2012.
    [4] Google. 2018. TFLite for Microcontrollers Benchmarks. [Online]. Available: https://github.com/tensorflow/tensorflow/treter/tensorflow/lite/micro/benchmarks
    [5] L. Lai, N. Suda, and V. Chandra, “CMSIS-NN: Efficient neural network kernels for arm cortex-M CPUs,” arXiv: 1801.06601 [cs], Jan. 2018.
    [6] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in Proc. Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 4510–4520.
    [7] S. Pouyanfar, S. Sadiq, Y. Yan, H. Tian, Y. Tao, M. P. Reyes, M.-L. Shyu, S.-C. Chen, and S. S. Iyengar, “A survey on deep learning: Algorithms, techniques, and applications,” ACM Comput. Surv., vol. 51, no. 5, pp. 92:1–92:36, Sep. 2018. [Online]. Available: http://doi.acm.org/10.1145/3234150
    [8] H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1153–1159, May 2016.
    [9] J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Netw., vol. 61, pp. 85–117, 2015.
    [10] Y. Al-Dhuraibi, F. Paraiso, N. Djarallah, and P. Merle, “Elasticity in Cloud Computing: State of the Art and Research Challenges,” IEEE Transactions on Services Computing, vol. 11, no. 2, pp. 430–447, March 2018.
    [11] D. Fernandes, A. G. Ferreira, R. Abrishambaf, et al. “Survey and taxonomy of transmissions power control mechanisms for wireless body area networks,” IEEE Commun. Surv Tut., vol. 20, no.2 pp. 1292-1328, Dec. 2017.
    [12] J. Chen and X. Ran, “Deep Learning With Edge Computing: A Review,” Proc. of the IEEE, vol. 107, no. 8, pp. 1655–1674, 2019.
    [13] S. Dey, J. Mondal, and A. Mukherjee, “Offloaded execution of deep learning inference at edge: Challenges and insights,” in 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), March 2019, pp. 855–861.
    [14] Z. Zhao, K. M. Barijough, and A. Gerstlauer, “Deepthings: Distributed adaptive deep learning inference on resource-constrained iot edge clusters,” IEEE Trans. Computer-Aided Design Integr. Circuits Syst., vol. 37, no. 1, pp. 2348–2359, 2018.
    [15] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” 2019, arXiv:1905.10083. [Online]. Available: https://arxiv.org/abs/1905.10083
    [16] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. 25th Int. Conf. Neural Inf. Process. Syst., 2013, pp. 1097–1105.
    [17] M. Baccouche, F. Mamalet, C. Wolf, C. Garcia, and A. Baskurt, “Sequential deep learning for human action recognition,” in Human Behavior Understanding. New York, NY, USA: Springer-Verlag, 2011, pp. 29–39.
    [18] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Proc. 28th Int. Conf. Neural Inf. Process. Syst., 2013, pp. 3111–3119.
    [19] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 770–778.
    [20] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” in Proc. 28th Int. Conf. Neural Inf. Process. Syst., 2015, pp. 91–99.
    [21] K. Ashton, “That ‘Internet of Things’ thing,” IEEE RFiD J., vol. 22, pp. 97–114, 2009.
    [22] H. Sundmaeker et al., Vision and Challenges for Realising the Internet of Things, 3.3, Cluster Eur. Res. Projects Internet Things, Eur. Commision, Brussels, Belgium, 2010, pp. 34–36.
    [23] W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, Oct. 2016.
    [24] E. Cuervo et al., “MAUI: Making smartphones last longer with code offload,” in Proc. 8th Int. Conf. Mobile Syst. Appl. Services, San Francisco, CA, USA, 2010, pp. 49–62.
    [25] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog computing and its role in the Internet of things,” in Proc. 1st Edition MCC Workshop Mobile Cloud Comput., Helsinki, Finland, 2012, pp. 13–16.
    [26] N. Cheng et al., “Big data driven vehicular networks,” IEEE Netw., vol. 32, no. 6, pp. 160–167, Nov./Dec. 2018.
    [27] H. Li, K. Ota, and M. Dong, “Learning IoT in edge: Deep learning for the internet of things with edge computing,” IEEE Netw., vol. 32, no. 1, pp. 96–101, Jan. 2018.
    [28] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “XNOR-Net: Imagenet classification using binary convolutional neural networks,” in Proc. Eur. Conf. Comput. Vis., Oct. 2016, pp. 525–542.
    [29] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” Arxiv:1704.04861, 2017.
    [30] J. Wu, C. Leng, Y. Wang, Q. Hu, and J. Cheng, “Quantized Convolutional Neural Networks for Mobile Devices,” In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 4820–4828.
    [31] S. B. Calo, M. Touna, D. C. Verma and A. Cullen, "Edge computing architecture for applying AI to IoT," 2017 IEEE International Conference on Big Data (Big Data), Boston, MA, 2017, pp. 3012-3016, doi: 10.1109/BigData.2017.8258272.
    [32] J. Shao and J. Zhang, "BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems," 2020 IEEE International Conference on Communications Workshops (ICC Workshops), Dublin, Ireland, 2020, pp. 1-6, doi: 10.1109/ICCWorkshops49005.2020.9145068.
    [33] C. Gao, A. Rios-Navarro, X. Chen, T. Delbruck and S. Liu, "EdgeDRNN: Enabling Low-latency Recurrent Neural Network Edge Inference," 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genova, Italy, 2020, pp. 41-45, doi: 10.1109/AICAS48895.2020.9074001.
    [34] K. Yang, Y. Shi, W. Yu and Z. Ding, "Energy-Efficient Processing and Robust Wireless Cooperative Transmission for Edge Inference," in IEEE Internet of Things Journal, vol. 7, no. 10, pp. 9456-9470, Oct. 2020, doi: 10.1109/JIOT.2020.2979523.

    下載圖示 校內：2025-11-18公開
    校外：2025-11-18公開
    QR CODE