在局部二值化卷積神經網路中，把原本的卷積層拆解為兩個較小的卷積層子層，第一子層為稀疏三元權重的卷積層，第二子層為1x1權重的卷積層。透過這樣拆解，局部二值化卷積神經網路比起傳統卷積神經網路有更低的計算複雜度及記憶體使用量。
為了更進一步提升效能，局部二值化卷積神經網路的硬體電路已被提出。由於第一子層結構為三元權重的卷積層，僅需要加法運算即可，因此在之前提出的硬體電路採取在一個週期內處理多筆加法運算使得整體效能提升。然而，這仍然需要大量的記憶體存取，故還有很大的改善空間。
在這篇論文中，我們提出一個高效能局部二值化卷積神經網路硬體架構，利用局部二值化卷積神經網路非常稀疏的特點，節省大部分不必要的運算並減少記憶體使用量，同時設計硬體電路架構改善記憶體存取次數。實驗結果顯示，我們提出的硬體電路架構比先前提出的架構面積減少20%、加法運算量減少93%、乘法運算量減少47%及總共所需運算週期數減少86%。

In order to simplify convolutional neural network (CNN), local binary convolutional neural network (LBCNN) has been proposed.
In the local binary convolutional neural network (LBCNN), a convolutional layer is divided into two sublayers, Sublayer 1 and Sublayer 2. Sublayer 1 is a sparse ternary-weighted convolutional layer, and Sublayer 2 is a 1x1 convolutional layer. By using two sublayers, LBCNN has lower computational complexity and lower memory usage than CNN.
To further improve the performance, an accelerator circuit designed for LBCNN has been proposed. Because Sublayer 1 of LBCNN is ternary-weighted, it only requires addition operations to perform convolutional operations and it can perform several additions in a cycle to improve the overall performance. However, it still requires significant memory accesses, which still has room for improvement. In this work, we propose an accelerator architecture to take advantage of sparsity in LBCNN and design a layer accelerator to reduce computational complexity and memory accesses. Experimental results show that under 20% area reduction, addition operations are reduced by 93%, multiplication operations are reduced by 47%, and clock cycles are reduced by 86%.

[1] K. Ando, K. Ueyoshi, K. Orimo, H. Yonekawa, S. Sato, H. Nakahara, S. Takamaeda-Yamazaki, M. Ikebe, T. Asai, T. Kuroda, and M. Motomura. Brein memory: A single-chip binary/ternary reconfigurable in-memory deep neural network accelerator achieving 1.4 tops at 0.6 w. In IEEE Journal of Solid-State Circuits, page 983–994, 2018.
[2] M. Bojarski, D. D. Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel,M. Monfort, U. Muller, J. Zhang, et al. End to end learning for self-driving cars.arXivpreprint arXiv:1604.07316, 2016.
[3] Y. H. Chen, T. Krishna, J. S. Emer, and V. Sze. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks. In IEEE Journal of Solid-State Circuits, page 127–138, 2016.
[4] M. Courbariaux, Y. Bengio, and J. P. David. Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in Neural InformationProcessing Systems, page 3105–3113, 2015.
[5] Design Automation Standards Committee. Ieee standard for systemverilog uni-fied hardware design, specification, and verification language standard ieee 1800.http://www.edastds.org/sv/, 2017.
[6] Z. Hu, J. Tang, Z. Wang, K. Zhang, L. Zhang, and Q. Sun. Deep learning for image-based cancer detection and diagnosis- a survey.Pattern Recognition,83:134–149,2018.
[7] F. Juefei-Xu, V. N. Boddeti, and M. Savvides. Local binary convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog-nition, pages 19–28, 2017.
[8] A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. InAdvances in Neural Information Processing Systems(NIPS), page 1097–1105, 2012.
[9] F. Li, B. Zhang, and B. Liu. Ternary weight networks. In arXiv preprint arXiv:1605.04711., pages 525–542, 2016.
[10] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conferenceon ComputerVision, pages 525–542, 2016.
[11] M. Shimoda, S. Sato, and H. Nakahara. All binarized convolutional neural network and its implementation on an fpga. In International Conference on Field Programmable Technology (ICFPT), page 291–294, 2017.
[12] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[13] H. M. Wallach. Topic modeling: beyond bag-of-words. In Proceedings of the 23rd international conference on Machine learning, pages 977–984. ACM, 2006.
[14] H. Yonekawa and H. Nakahara. On-chip memory based binarized convolutional deep neural network applying batch normalization free technique on an fpga. In IEEE IPDPSW, page 98–105, 2017.
[15] A. Zhakatayev and J. Lee. Efficient FPGA implementation of local binary convolutional neural network. In Proceedings of the 24th Asia and South Pacific Design Automation Conference, pages 699–704, 2019.