成功大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	余驊宸 YU, Hua-Chen
論文名稱：	深度學習加速器之硬體與編譯器共同最佳化 Hardware-Compiler Co-Optimization for a Deep Learning Accelerator
指導教授：	陳中和 Chen, Chung-Ho
學位類別：	碩士 Master
系所名稱：	電機資訊學院 - 電機工程學系 Department of Electrical Engineering
論文出版年：	2026
畢業學年度：	114
語文別：	中文
論文頁數：	112
中文關鍵詞：	神經網路處理器、Microsoft Floating Point 、查找表運算架構、編譯器碼生成、Flash Attention 、轉置最佳化
外文關鍵詞：	Neural Processing Unit, Microsoft Floating Point, LUT-Based Compute Architecture, Compiler Code Generation, Flash Attention, Transpose Optimization
相關次數：	點閱：13 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

近年來，深度學習推論工作負載逐漸由純卷積架構擴展為同時包含矩陣乘法、歸約運算、資料重排與非線性函數的異質型態。卷積神經網路在視覺應用中仍扮演重要角色，而 Transformer 模型則因具備良好的表示能力與可擴展性，已廣泛應用於影像與序列處理任務。為支援此類異質運算需求，本實驗室先前已提出可同時支援 CNN 與 Transformer 工作負載之 Novella-NPU 統一架構。然而，在 Brown version 實際部署過程中，仍面臨數值格式支援、運算核心硬體複雜度、編譯器指令生成流程，以及轉置操作所帶來的額外搬移成本等挑戰。

本論文基於既有 Novella-NPU 平台，主軸聚焦於三項 Brown NPU 部署工作：（一）Andersen backend 至 Brown backend 之 compiler 遷移與指令生成流程建立；（二）Transformer／ViT 類工作負載之 lowering、資料搬移與端到端執行支援；（三）Flash Attention kernel generation 與排程最佳化。除此之外，本論文亦針對 MSFP 導向之 LUT-based compute core 與 memory-based transpose optimization 進行設計探索與可行性分析，作為後續硬體延伸方向。

在編譯器後端方面，本論文完成 Andersen 至 Brown backend 之系統性遷移，涵蓋 operand index 重構、資料格式由 INT8／scale 轉換至 BF16／MSFP、Unified Buffer 記憶體配置重映射，以及 bank-aware scheduling 以避免 SRAM bank conflict。針對 Transformer 類工作負載，本論文新增 transpose、unfolded load、class token load／store 等 macro-operation 生成流程，並完成與 Brown ISS、AlgoSim 及 RTL 驗證流程之整合，建立支援 Brown NPU 之完整 kernel code generation 路徑。針對 attention 執行，本論文進一步建立 Flash Attention custom op、online softmax decomposition、phase-interleaved bank allocation 與 cross-token prefetch 排程流程。在硬體設計探索方面，本論文以 BF16 activation 搭配 MSFP weight 為前提，評估 LUT-based mantissa product path 與 memory-based transpose 作為未來 Brown NPU 延伸方向之可行性，而非主張其已完成完整 post-synthesis implementation。

實驗結果顯示，Brown backend 所產生之 instruction stream 已於 ISS 與 AlgoSim 間達成 bit-match，確認 operand migration、SRAM placement 與 bank conflict detection 之正確性。本論文並完成 RTL end-to-end ViT 執行驗證，確認所建立之 compiler backend、macro-operation generation 與 Brown NPU 執行環境可共同支援 Transformer 類模型之端到端推論需求。上述工作使 Novella-NPU 平台成功延伸至 Brown NPU 執行環境，並為後續完整浮點推論部署、複合 kernel 排程與 CNN／Transformer 混合工作負載支援奠定基礎。

Recent deep learning inference workloads have evolved from relatively homogeneous convolution-dominated execution into heterogeneous workloads that combine convolution, matrix multiplication, element-wise operations, reduction operations, data-layout transformation, and nonlinear functions. Convolutional neural networks remain important in many vision applications, while Transformer-based models have been widely adopted due to their strong representation capability and scalability. To support such heterogeneous workloads, our laboratory previously developed Novella-NPU as a unified inference platform for both CNN and Transformer models. However, practical deployment on the Brown version of this platform still requires further support in numerical format handling, compiler backend implementation, instruction generation, data movement, and Transformer-oriented execution.

This thesis builds upon the existing Novella-NPU platform. The primary focus is on establishing and verifying the Brown NPU deployment stack across three primary deployment tasks. First, this thesis completes the migration from the Andersen backend to the Brown backend, including operand index reconstruction, data type migration from INT8/scale-based representation to BF16/MSFP representation, Unified Buffer remapping, bank-aware scheduling, and integration with Brown ISS, AlgoSim, and RTL verification flows. Second, this thesis establishes Transformer-oriented macro-operation generation support, including unfolded load, transpose, class token load/store, QKV input reuse, and CTU+PPU fused kernel generation. Third, this thesis adds Flash Attention kernel generation and scheduling, covering a custom operator definition, online softmax decomposition, phase-interleaved bank allocation, software-pipelined KV tile execution, and cross-token QK prefetch. In addition, this thesis explores two hardware-design directions as future extensions: an experimental LUT-based compute-core design for MSFP mantissa product generation, and a memory-based transpose optimization using diagonal bank mapping. Both explorations are positioned as feasibility analyses and proposed design directions rather than completed post-synthesis results.

Experimental verification shows that the kernel instruction streams generated by the Brown backend achieve bit-match between ISS and AlgoSim for supported deterministic kernels, confirming the correctness of operand migration, SRAM placement, and bank conflict detection. In addition, RTL end-to-end ViT execution verification is completed, demonstrating that the compiler backend, macro-operation generation flow, data movement support, and Brown NPU execution environment can jointly support Transformer-style inference. Overall, this thesis extends the existing Novella-NPU platform toward Brown NPU execution, improves its compiler and verification infrastructure, and provides design directions for low-cost MSFP-friendly compute cores and transpose-related data movement optimization.

摘要 i
英文延伸摘要 ii
誌謝 vii
目錄 viii
表格 xii
圖片 xiii
Chapter 1. 緒論 1
1. 研究動機 1
2. 研究挑戰 2
3. 論文貢獻 3
4. 論文架構 4
Chapter 2. 背景知識與相關研究 6
1. 深度學習模型與計算特性 6
1.1. 卷積神經網路 6
1.2. Vision Transformer 7
1.3. 異質深度學習工作負載 7
2. 運算子類型與硬體對應需求 8
2.1. MAC-dominated operators 8
2.2. 向量式、reduction 與非線性運算子 8
2.3. Layout-transform operators 9
3. 既有 Novella-NPU Brown 平台 9
3.1. 整體架構與資料路徑 9
3.2. Control flow 與執行順序 11
3.3. CTU：Convolution and Transformer Unit 11
3.4. PPU、MLU 與 Unified Buffer 12
3.5. Shared data handling 與 intermediate reuse 12
4. 數值格式與模型部署 13
4.1. INT8 Quantization 13
4.2. BF16 13
4.3. Microsoft Floating Point（MSFP） 13
5. 模型編譯器與程式生成流程 15
5.1. TVM 編譯流程 15
5.2. 前端轉換與後端 lowering 15
5.3. 硬體限制下的 code generation 16
6. Flash Attention、Transpose 與 Resource-Constrained Scheduling 16
6.1. Flash Attention 的運算流程 16
6.2. Transpose 與資料重排的角色 17
6.3. Dependency handling 與 resource-aware scheduling 17
7. 本章小結 18
Chapter 3. 編譯器後端實作 19
1. 概述 19
2. Andersen 至 Brown 後端遷移 20
2.1. 資料型別遷移 20
3. Kernel Generation Flow 22
4. Macro-Operation Lowering 24
4.1. BrownMacroOpLower 設計概述 24
4.2. 卷積 Macro-Operation 生成 25
5. Input Channel Tiling 策略與 Cost Model 26
5.1. ICH Tiling 的動機 26
5.2. Cost Model 設計 27
5.3. is_mem_bound 與 tile_size_limitation 28
5.4. 執行時的 Partial Sum Spill/Reload 29
6. SRAM 配置策略 30
6.1. Unified Buffer 架構 30
6.2. SRAM 使用量追蹤 31
7. Bank-Aware Scheduling 31
7.1. SRAM 位址映射模式 31
7.2. Bank 與位址雙重重疊檢查 32
8. ISS 整合與 Bit-Match 驗證 33
8.1. NpuVersion 參數化 33
8.2. Bit-Match 驗證流程 33
9. 小結 34
Chapter 4. ViT 工作負載之 Lowering 與 Kernel 生成 35
1. 概述 35
2. Unfold 機制與資料表示轉換 35
2.1. Unfold 的動機 35
2.2. 形狀重構與 Weight 調整 37
3. Unfold IFM 的載入策略 38
3.1. IFM Layout 選擇 38
3.2. 輸出元素起始位址 38
3.3. Channel-wise 位址計算 39
4. AlgoSim Unfold 支援 39
5. Dependency-Aware Instruction Scheduling 40
5.1. IFM Ping-Pong Bank 配置 40
5.2. MLU Z 維度再利用 41
5.3. DependencyTag 依賴標籤系統 41
5.4. 依賴感知指令提升 42
5.5. 傳遞隱含標籤移除 43
6. Multi-Head QKV 投影的 Weight 合併 44
7. 新增 Fusion Kernel 生成 45
7.1. Kernel 分類概述 45
7.2. CTU+PPU fused Kernel 的 Bank 配置通用原則 46
7.3. ConvBinLnMacroGen（Conv + BinaryEltwise + LayerNorm） 46
8. 小結 48
Chapter 5. Flash Attention Kernel 生成與排程 50
1. 概述 50
2. Custom Op 定義 50
2.1. NovellaFlashAttn Op 50
3. Legalization：MHSA 至 Flash Attention 的圖改寫 51
3.1. R2C Rewriter 中的 Flash Attention 路徑 51
4. Kernel 生成：FlashAttnMacroGen 52
4.1. 結構概述 52
4.2. Macro-Operation 序列 53
4.3. Online Softmax PPU 分解 54
4.4. KV Tile 軟體流水線概述 57
4.5. 跨 Token QK 預取 57
4.6. Stage 0：首個 KV Tile 的啟動排程 58
4.7. Steady State：KV 流水線的多層提前發出排程 59
4.8. 軟體流水線排程總結 61
5. Phase-Interleaved Bank 配置 62
5.1. Bank 配置設計 62
6. Depth Tiling 抑制與靜態 QK 提升 63
7. XNode、AlgoSim、ISS 與 RTL 驗證 64
7.1. XNode 機制 64
7.2. 數值驗證流程 64
8. 小結 66
Chapter 6. 運算核心與資料搬移最佳化之設計探索 68
1. 概述 68
2. LUT-based MSFP Compute Core 68
2.1. 設計動機 68
2.2. High/Low 4-bit Product LUT 69
2.3. Approximate Negative Mantissa Handling 70
2.4. Cross-Block Fixed-Point Accumulation 72
2.5. Exponent Constraint 與 Tensor Compression Opportunity 73
2.6. AlgoSim Modeling Scope 74
2.7. 設計限制 74
3. Transpose-related Data Movement Optimization 75
3.1. 動機與 PPU-bound Projection 75
3.2. Memory-based Transpose 與 Diagonal Slot Mapping 76
3.3. Brown Unified Buffer 整合 78
3.4. Estimated Performance Impact 79
4. 小結 79
Chapter 7. 實驗環境與結果分析
1. 概述 81
2. 實驗環境 81
2.1. 硬體平台規格 81
2.2. 編譯器與驗證工具鏈 82
3. Compiler Backend 正確性驗證 82
3.1. 驗證目標 82
4. Fusion Kernel Cycle Analysis 83
5. Compiler Scheduling 優化分析 84
6. LUT-based Compute Core Accuracy Evaluation 84
7. Transpose-related Analytical Evaluation 86
8. 與其他 ViT 加速器之比較 87
9. 本章小結 87
Chapter 8. 結論與未來工作 89
1. 結論 89
2. 未來工作 91
2.1. 完整 4-bit Weight Support 與 End-to-End Evaluation 92
2.2. LUT-based Compute Core RTL Implementation 92
2.3. Memory-based Transpose RTL 與 Compiler Integration 92
2.4. Flash Attention Performance Tuning 93
2.5. More General Compiler Scheduling Framework 93
2.6. 完整系統層級評估 93
3. 總結 94
References 95
                                    

[1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention Is All You Need,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
[2] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M.Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in International Conference on Learning Representations, 2021.
[3] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training Data-Efficient Image Transformers & Distillation through Attention,” in Proceedings of the 38th International Conference on Machine Learning, PMLR, vol. 139, pp. 10347–10357, 2021.
[4] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520, 2018.
[5] T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, M. Cowan, H. Shen, L. Wang, Y. Hu, L. Ceze, C. Guestrin, and A. Krishnamurthy, “TVM: An Automated End-to-End Optimizing Compiler for Deep Learning,” in Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation, pp. 578–594, 2018.
[6] T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Ré, “Flash Attention: Fast and Memory-Efficient Exact Attention with IO-Awareness,” in Advances in Neural Information Processing Systems, vol. 35, pp. 16344–16359, 2022.
[7] B. D. Rouhani et al., “Pushing the Limits of Narrow Precision Inferencing at Cloud Scale with Microsoft Floating Point,” in Advances in Neural Information Processing Systems, vol. 33, pp. 10271–10281, 2020.
[8] D. Kalamkar, D. Mudigere, N. Mellempudi, D. Das, K. Banerjee, S. Avancha, D. T. Vooturi, N. Jammalamadaka, J. Huang, H. Yuen, J. Yang, J. Park, A. Heinecke, E. Georganas, S. Srinivasan, A. Kundu, M. Smelyanskiy, B. Kaul, and P. Dubey, “A Study of BFLOAT16 for Deep Learning Training,” arXiv preprint arXiv:1905.12322, 2019.
[9] N. P. Jouppi et al., “In-Datacenter Performance Analysis of a Tensor Processing Unit,” in Proceedings of the 44th Annual International Symposium on Computer Architecture, pp. 1–12, 2017.
[10] Y.-H. Chen, J. Emer, and V. Sze, “Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks,” in Proceedings of the 43rd ACM/IEEE Annual International Symposium on Computer Architecture, pp. 367–379, 2016.
[11] Q. Shang, Y. Fan, W. Shen, S. Shen, and X. Zeng, “Single-Port SRAM-Based Transpose Memory With Diagonal Data Mapping for Large Size 2-D DCT/IDCT,” IEEE 95Transactions on Very Large Scale Integration (VLSI) Systems, vol. 22, no. 11, pp. 2423–2426, 2014.
[12] Y. Dai, “Design of a Kernel-agnostic Compute Core for Convolution and GEMM,” M.S. thesis, Dept. of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, 2024.
[13] H. You, Z. Sun, H. Shi, Z. Yu, Y. Zhao, Y. Zhang, C. Li, B. Li, and Y. Lin, “ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design,” in Proceedings of the IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2023.
[14] S. Nag, G. Datta, S. Kundu, N. Chandrachoodan, and P. A. Beerel, “ViTA: A Vision Transformer Inference Accelerator for Edge Applications,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), 2023.
[15] K. Marino, P. Zhang, and V. K. Prasanna, “ME-ViT: A Single-Load Memory-Efficient FPGA Accelerator for Vision Transformers,” in Proceedings of the IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC), 2023.
[16] C.-C. Wei, H.-C. Yu, C.-K. Su, C.-Y. Tsai, F.-Y. Song, M.-J. Tseng, and C.-H. Chen, “A Unified Neural Processing Unit for CNN and Transformer Workloads with a Microsoft Floating Point-Based Compute Core,” in Proceedings of the IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), 2025.

簡易檢索 / 詳目顯示

相關論文