深度學習技術現已逐漸被使用在物聯網（Internet of Things, IoT）的應用中。然而，由於深度學習需要大量的運算與記憶體資源，也因此相關的技術要在資源缺乏的IoT裝置上使用，成為很困難的一件事情。我們提出的軟體框架架構在以python語言為基底的深度學習網路軟體系統Chainer上，使用嵌入到框架的低階程式語言如C/C++編寫的計算層去建構卷積神經網路模型。
我們的架構可以藉由直接訓練以C語言編寫的計算層，產生出可以直接在IoT裝置上編譯的卷積神經網路模型，進而方便控制模型的設計和需要的記憶體使用量，從而避免訓練後優化對於在裝置上推斷時產生的不確定性。
我們觀察到簡化問題的必要性，並且在我們的架構上證實了簡化問題對於準確度的影響。
我們將框架應用到針對環境監測做影像和聲音的辨識以驗證可行性，另外也分享了實驗中得到的經驗以及分析架構本身的可攜性。
在實驗中我們的架構使用二元分析器在3.24KB記憶體的情況下達到了86%的準確度，我們認為我們的架構適合為記憶體在幾十KB內的低階裝置設計適用的卷積神經網路模型。

Deep learning methods have been gradually adopted in the Internet-of-Things (IoT) applications. Nevertheless, the heavy demands for the computation and memory resources make these methods are difficult to be deployed in the field as the resource-constrained IoT devices would be overwhelmed by the computations incurred by the inference operations of deployed deep learning models.
In this article, we develop a software framework, which leverages the Python-based neural network framework, Chainer, to form CNNs by mounting the C/C++ implementations of the regular and/or lightweight neural network layers. By directly training the C-based layers formed CNN and building machine code runnable on the device, the proposed framework enables free control over the model design and measures the memory usages of the built CNNs, avoids the occurrence of unexpected results cause by post-training optimizations during the on-device inference.
We have observed the necessity of problem simplify and verify the accuracy of the proposed framework. To prove feasibility, we emulate the proposed work on the most common applications, i.e., image
processing and acoustic monitoring in environmental monitoring. Besides, we share the lessons during the experiments and analysis of the applicability of our work. In particular, our framework achieves more than 86% accuracy while only consume 3.24KB memory for binary classification. We believe that our work is more suitable for developing CNNs for low-end devices with limited memory space, e.g,. with several tens of KBs memory.

[1] Bmxnet. https://github.com/hpi-xnor/BMXNet-v2, 2020. [Online; accessed 10-Feburary-2020].
[2] cnet. https://github.com/yang-le/cnet?fbclid=IwAR1Pi6-lEiBthjgC6hFIgypvShR9FYQJo15yVuMzvVgTkXm6H1DO471L7Lg,2020. [Online; accessed 10-Feburary-2020].
[3] Integrating caffe2 on ios/android. https://caffe2.ai/docs/mobile-integration.html, 2020. [Online; accessed 10-Feburary-2020].
[4] Kann. https://github.com/attractivechaos/kann, 2020. [Online; accessed 10-Feburary-2020].
[5] ncnn. https://github.com/Tencent/ncnn, 2020. [Online; accessed 10-Feburary-2020].
[6] Tensorflow model optimization toolkit —pruning api. https://blog.tensorflow.org/2019/05/tf-model-optimization-toolkit-pruning-API.html, 2020. [Online; accessed 10-Feburary-2020].
[7] Imen Abdennadher, Nesrine Khabou, Ismael Bouassida Rodriguez, and Mohamed Jmaiel. Designing energy efficient smart buildings in ubiquitous environments. In 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA), pages 122–127. IEEE, 2015.
[8] Ikkyu Aihara, Daichi Kominami, Yasuharu Hirano, and Masayuki Murata. Mathematical modelling and application of frog choruses as an autonomous distributed communication system. Royal Society open science, 6(1):181117, 2019.
[9] Hande Alemdar, Vincent Leroy, Adrien Prost-Boucle, and Frédéric Pétrot. Ternary neural networks for resource-efficient ai applications. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 2547–2554. IEEE, 2017.
[10] Rosa Ma Alsina-Pagès, Unai Hernandez-Jayo, Francesc Alías, and Ignacio Angulo. Design of a mobile low-cost sensor network using urban buses for real-time ubiquitous noise monitoring. Sensors, 17(1):57, 2017.
[11] AmebaIoT. Realtek iot/wi-fi mcu solutions. https://www.amebaiot.com/boards/, 2020. [Online; accessed 22-June-2020].
[12] Arduino. Mkr family. https://store.arduino.cc/usa/arduino/ arduino-mkr-family, 2020. [Online; accessed 22-June-2020].
[13] Eyuel D Ayele, Nirvana Meratnia, and Paul JM Havinga. Towards a new opportunistic iot network architecture for wildlife monitoring system. In 2018 9th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pages 1–5. IEEE, 2018.
[14] Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano. Benchmark analysis of representative deep neural network architectures. IEEE Access, 6:64270–64277, 2018.
[15] Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? arXiv preprint arXiv:2003.03033, 2020.
[16] Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332, 2018.
[17] Emre Cakir, Sharath Adavanne, Giambattista Parascandolo, Konstantinos Drossos, and Tuomas Virtanen. Convolutional recurrent neural networks for bird audio detection. In 2017 25th European Signal Processing Conference (EUSIPCO), pages 1744–1748. IEEE, 2017.
[18] Jungwook Choi, Swagath Venkataramani, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan, Zhuo Wang, and Pierce Chuang. Accurate and efficient 2-bit quantized neural networks. In Proceedings of the 2nd SysML Conference, 2019.
[19] Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016.
[20] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
[21] Misha Denil, Babak Shakibi, Laurent Dinh, Marc’Aurelio Ranzato, and Nando De Freitas. Predicting parameters in deep learning. In Advances in neural information processing systems, pages 2148–2156, 2013.
[22] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377, 2018.
[23] evilmucedin. tiny-dnn. https://github.com/tiny-dnn/tiny-dnn, 2020. [Online; accessed 10-Feburary-2020].
[24] Julian Faraone, Nicholas Fraser, Michaela Blott, and Philip HW Leong. Syq: Learning symmetric quantization for efficient deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4300–4309, 2018.
[25] Igor Fedorov, Ryan P Adams, Matthew Mattina, and Paul Whatmough. Sparse: Sparse architecture search for cnns on resource-constrained microcontrollers. In Advances in Neural Information Processing Systems, pages 4977–4989, 2019.
[26] Eduardo Fonseca, Manoj Plakal, Daniel PW Ellis, Frederic Font, Xavier Favory, and Xavier Serra. Learning sound event classifiers from web audio with noisy labels. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 21–25. IEEE, 2019.
[27] Graham Gobieski, Brandon Lucia, and Nathan Beckmann. Intelligence beyond the edge: Inference on intermittent embedded systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 199–213. ACM, 2019.
[28] Chirag Gupta, Arun Sai Suggala, Ankit Goyal, Harsha Vardhan Simhadri, Bhargavi Paranjape, Ashish Kumar, Saurabh Goyal, Raghavendra Udupa, Manik Varma, and Prateek Jain. Protonn: Compressed and accurate knn for resource-scarce devices. In International Conference on Machine Learning, pages 1331–1340, 2017.
[29] Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015.
[30] Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European Conference on Computer Vision (ECCV), pages 784–800, 2018.
[31] Agnes Incze, Henrietta-Bernadett Jancsó, Zoltán Szilágyi, Attila Farkas, and Csaba Sulyok. Bird sound recognition using a convolutional neural network. In 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY), pages 000295–000300. IEEE, 2018.
[32] TEXAS INSTRUMENTS. Msp430 ultra-low-power mcus–products. https://www.ti.com/microcontrollers/msp430-ultra-low-power-mcus/products.html, 2020. [Online; accessed 22-June-2020].
[33] Felix Juefei-Xu, Vishnu Naresh Boddeti, and Marios Savvides. Local binary convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 19–28, 2017.
[34] Jung-Yoon Kim, Chao-Hsien Chu, and Sang-Moon Shin. Issaq: An integrated sensing systems for real-time indoor air quality monitoring. IEEE Sensors Journal, 14(12):4230–4244, 2014.
[35] Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, and Dongjun Shin. Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530, 2015.
[36] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013.
[37] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: http://www.cs.toronto.edu/kriz/cifar.html, 55, 2014.
[38] Ashish Kumar, Saurabh Goyal, and Manik Varma. Resource-efficient machine learning in 2 kb ram for the internet of things. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1935–1944. JMLR. org, 2017.
[39] SILICON LABS. EFM32™ Gecko 32-bit Microcontroller. https://www.silabs.com/mcu/32-bit/efm32-gecko, 2019. [Online; accessed 19-December-2019].
[40] Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy. Fixed point quantization of deep convolutional networks. In International conference on machine learning, pages 2849–2858, 2016.
[41] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. In Proceedings of the European Conference on Computer Vision (ECCV), pages 19–34, 2018.
[42] Qing Lu, Weiwen Jiang, Xiaowei Xu, Yiyu Shi, and Jingtong Hu. On neural architecture search for resource-constrained hardware platforms. arXiv preprint arXiv:1911.00105, 2019.
[43] Oisin Mac Aodha, Rory Gibb, Kate E Barlow, Ella Browning, Michael Firman, Robin Freeman, Briana Harder, Libby Kinsey, Gary R Mead, Stuart E Newson, et al. Bat detective—deep learning tools for bat acoustic signal detection. PLoS computational biology, 14(3):e1005995, 2018.
[44] Bradley McDanel, Surat Teerapittayanon, and HT Kung. Embedded binarized neural networks. arXiv preprint arXiv:1709.02260, 2017.
[45] Bradley McDanel, Sai Qian Zhang, HT Kung, and Xin Dong. Full-stack optimization for accelerating cnns using powers-of-two weights with fpga validation. In Proceedings of the ACM International Conference on Supercomputing, pages 449–460, 2019.
[46] Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268, 2018.
[47] Karol J Piczak. Environmental sound classification with convolutional neural networks. In 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2015.
[48] Karol J Piczak. Esc: Dataset for environmental sound classification. In Proceedings of the 23rd ACM international conference on Multimedia, pages 1015–1018, 2015.
[49] pjreddie. Darknet. https://github.com/pjreddie/darknet, 2020. [Online; accessed 10-Feburary-2020].
[50] Peter Prince, Andrew Hill, Evelyn Piña Covarrubias, Patrick Doncaster, Jake L Snaddon, and Alex Rogers. Deploying acoustic detection algorithms on low-cost, opensource acoustic sensors for environmental monitoring. Sensors, 19(3):553, 2019.
[51] Sujith Ravi. Projectionnet: Learning efficient on-device deep networks using neural projections. arXiv preprint arXiv:1708.00630, 2017.
[52] Khatal Sachin Sahebrao. Gps/gsm based animal tracking and health monitoring system. International Journal Of Engineering, Education And Technology (ARDIJEET), 3(2):1–4, 2015.
[53] Dialog semiconductor. Animal Tracker / Location Tracker. https://www.dialog-semiconductor.com/application-solution-diagram/iot/connected-consumer/animal-tracker-location-tracker, 2019. [Online; accessed 19-December-2019].
[54] Taylor Simons and Dah-Jye Lee. A review of binarized neural networks. Electronics, 8(6):661, 2019.
[55] Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE, 105(12):2295–2329, 2017.
[56] Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2820–2828, 2019.
[57] TensorFlow. Get started with microcontrollers. https://www.tensorflow.org/lite/microcontrollers/get_started, 2020. [Online; accessed 6-Janurary-2020].
[58] TensorFlow. Post-training quantization. https://www.tensorflow.org/lite/performance/post_training_quantization, 2020. [Online; accessed 28-Janurary-2020].
[59] TensorFlow. Quantization aware training. https://www.tensorflow.org/model_optimization/guide/quantization/training, 2020. [Online; accessed 28-Janurary-2020].
[60] TensorFlow. Tensorflow lite for microcontrollers. https://www.tensorflow.org/lite/microcontrollers, 2020. [Online; accessed 6-May-2020].
[61] Jack Turner, José Cano, Valentin Radu, Elliot J Crowley, Michael O’Boyle, and Amos Storkey. Characterising across-stack optimisations for deep convolutional neural networks. In 2018 IEEE International Symposium on Workload Characterization (IISWC), pages 101–110. IEEE, 2018.
[62] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10734–10742, 2019.
[63] Yuzhong Wu and Tan Lee. Enhancing sound texture in cnn-based acoustic scene classification. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 815–819. IEEE, 2019.
[64] Tien-Ju Yang, Andrew Howard, Bo Chen, Xiao Zhang, Alec Go, Mark Sandler, Vivienne Sze, and Hartwig Adam. Netadapt: Platform-aware neural network adaptation for mobile applications. In Proceedings of the European Conference on Computer Vision (ECCV), pages 285–300, 2018.
[65] Y.SAISHO. Cicada songs. http://zikade.world.coocan.jp/cicadasongaac_e.html, 2020. [Online; accessed 9-June-2020].
[66] Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefanet: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016.
[67] Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578, 2016.