簡易檢索 / 詳目顯示

回結果列表
研究生： 陳盈秀
Chen, Ying-Hsiu
論文名稱： 基於遞迴神經網絡長短記憶迴歸模型且應用於時間交錯型逐漸趨近式類比數位轉換器的校正技巧
Calibration Technique based on LSTM Regression Neural Network for Time-Interleaved SAR Analog-to-Digital Converter
指導教授： 張順志
Chang, Soon-Jyh
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 102
中文關鍵詞： 類比數位轉換器 、逐漸趨近式 、時間交錯式 、時序偏移 、校正 、長短記憶模型 、遞迴神經網絡 、迴歸分析模型
外文關鍵詞： analog-to-digital converter (ADC), successive-approximation register (SAR), time-interleaved (TI), timing-skew, calibration, Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), regression model
相關次數： 點閱：196  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 本論文提出一個遞迴神經網絡(RNN)長短記憶(LSTM)迴歸分析模型(Regression Model)應用於校正時間交錯型(TI)逐漸趨近式(SAR)類比數位轉換器(ADC)的技術。為了更高速、同時節省功耗地將類比輸入訊號轉換成數位訊號，由多通道SAR ADC組成的TI ADC架構日益受到重視，因其可以達到現今高速與低功耗要求下的完美平衡。不過此架構容易遭受多種不匹配錯誤量(Mismatch Error)的影響導致數位轉換器的效能大幅度下降。本篇論文為了解決多種有可能出現的不匹配錯誤，首先建構TI SAR ADC的電路行為模型，用以產生需要的神經網絡訓練資料庫。接著提出許多訓練技巧與演算法來提升訓練長短記憶迴歸模型在校正上的精確度。之後，將所訓練的模型應用於校正多通道TI SAR ADC。
    經由多個類比數位轉換器電路行為模型模擬、與取樣速度十六億次的真實類比數位轉換器電路量測的雙重測試，本論文所提出的長短記憶校正架構可以有效減低通道不匹配產生的錯誤、並大幅精進多通道TI SAR ADCs的效能。

    This thesis proposes a Long Short-Term Memory model (LSTM) Recurrent Neural Network (RNN) model calibration technique for time-interleaved (TI) successive-approximation register (SAR) analog-to-digital converter (ADC). In order to achieve a higher speed beyond the technological limits and a lower power consumption on translating analog inputs for digital data processing purposes, multi-channel architecture TI ADC has been designed. Nevertheless, the performance of TI ADC is significantly degraded by several channel mismatch errors. In the purposed LSTM calibration structure, we built a TI-ADC behavior model with different mismatch errors to generate the desired ADC calibration dataset for Neural Network training, also enhance the training speed and accuracy performance of LSTM regression model by multi-aspect of training skills and customized loss function. And then, the trained model is applied to calibrate the multi-channel TI SAR ADC.
    Numerical testing after experimental simulations and real chip measurements have been presented to demonstrate the effectiveness of the proposed structure, which shows the LSTM calibration structure could significantly improve the performance of the TI SAR ADCs.

    摘 要III Abstract IV 致 謝 VI List of Tables X List of Figures XI Chapter 1 Introduction 1 1.1 Motivation and Background 1 1.2 Time-interleaved SAR ADC 5 1.3 LSTM Calibration Structure 7 1.4 Thesis Organization 7 Chapter 2 Time-interleaved SAR ADC model 9 2.1 Channel operation 10 2.1.1 Single channel sampled in time and frequency domain 10 2.1.2 Multiple channels sampled in time and frequency domain 13 2.2 Mismatch errors in Multiple channel ADC 17 2.2.1 Gain Mismatch Error 18 2.2.2 Offset Mismatch Error 21 2.2.3 Timing skew Mismatch 25 2.3 Behavior model of TI ADC 28 Chapter 3 LSTM Recurrent Neural Network 31 3.1 Deep learning 32 3.1.1 Neural Network Architecture 33 3.1.2 Comparison of various Neural Networks 41 3.2 LSTM topology 44 3.2.1 Recurrent Neural Network 44 3.2.2 The limitation on RNN 46 3.2.3 The unit cell of LSTM 46 3.3 LSTM sequential model 51 3.3.1 LSTM sequential model 52 3.3.2 Parameter of LSTM layers 54 Chapter 4 TI SAR ADC calibration on LSTM model 57 4.1 LSTM TI SAR ADC calibration model 58 4.2 Behavior model to LSTM calibration model 61 4.2.1 Splitting and First shuffle from ADC behavior model 62 4.2.2 Normalization 63 4.2.3 Second Shuffle and input vector timestep 64 4.3 LSTM Model hyper-parameter definition 67 4.3.1 Timestep and hidden node for LSTM calibration 67 4.3.2 Training time and Batch size 70 4.4 LSTM Model on training 72 4.4.1 Loss target 72 4.4.2 Customized Loss function 74 4.4.3 Adjust method from Training and validation loss 77 4.4.4 LSTM training algorithm and target 78 4.4.5 Learning rate decay of LSTM model training 79 Chapter 5 Experiment and Result 82 5.1 Behavior model 82 5.2 Real chip data 88 Chapter 6 Conclusion and Future Works 92 Bibliography 96

    [1] B. Murmann, “ADC Performance Survey 1997-2018,” [Online]. Available: http://web.stanford.edu/~murmann/adcsurvey.html.
    [2] W. Black and D. Hodges, “Time interleaved converter arrays,” in IEEE Journal of Solid-State Circuits, vol. 15, pp. 1022-1029, Dec. 1980.
    [3] J. A. McNeill, “Split ADC calibration for all-digital correction of time-interleaved ADC errors,” in IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 56, no. 5, pp. 344–348, May 2009.
    [4] L. Luo, S. Chen, M. Zhou and T. Ye, “A 0.014mm2 10-bit 2GS/s time-interleaved SAR ADC with low-complexity background timing skew calibration,” 2017 Symposium on VLSI Circuits, Kyoto, Japan, pp. 278-279, 2017.
    [5] S. Kundu, “A 1.2 V 2.64 GS/s 8bit 39 mW skew-tolerant time-interleaved SAR ADC in 40 nm digital LP CMOS for 60 GHz WLAN,” in Proceedings of the IEEE 2014 Custom Integrated Circuits Conference, San Jose, CA, USA, pp. 1-4, 2014.
    [6] Yongtao Qiu, Jie Zhou, Youjiang Liu, Yafan Huangfu, “A Novel Calibration Method of Gain and Time-skew Mismatches for Time-interleaved ADCs Based on Neural Network,” 2019 IEEE MTT-S International Wireless Symposium (IWS), pp. 1-3, 2019.
    [7] Junya Matsuno and Takafumi Yamaji, “All-digital background calibration technique for time-interleaved ADC using pseudo aliasing signal,” in IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 60, no. 5, pp. 1113–1121, May 2013.
    [8] D. Kumar, “Calibration of Sampling Clock Skew in High-Speed, High-Resolution Time-Interleaved ADCs,” Ph. D. thesis, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2015.
    [9] A. M. Turing, “Computing Machinery and Intelligence,” in Mind, vol. 5, pp. 433–460, Oct. 1950.
    [10] Walter Pitts and Warren McCulloch, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” in Bulletin of Mathematical Biophysics, vol. 5, pp. 115-133, 1943.
    [11] George Cybenko, “Approximation by Superpositions of a Sigmoidal Function,” in Math. Control Signal Systems, vol. 2, pp. 303-314, 1989.
    [12] S. I. Gallant, “Perceptron-based learning algorithms,” in IEEE Transactions on Neural Networks, vol. 1, no. 2, pp. 179-191, June 1990.
    [13] Pierluigi Nuzzo, Fernando De Bernardinis, Pierangelo Terreni, and Geert Van der Plas, “Noise analysis of regenerative comparators for reconfigurable ADC architectures,” in IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 55, no. 6, pp. 1441-1454, July 2008.
    [14] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams, “Learning representations by back-propagating errors,” in Nature, vol. 323, pp. 533-536, 1986.
    [15] J. L. Elman, “Finding Structure in Time. Cognitive Science,” in Cognitive Science, vol.14(2), pp. 179–211, 1990.
    [16] M. I. Jordan, “Attractor dynamics and parallelism in a connectionist sequential machine,” in IEEE Computer Society Neural Networks Technology Series, pp. 112-127, 1990.
    [17] K. J. Lang, A. H. Waibel, and G. E. Hinton, “A Time-delay Neural Network Architect,” in Neural Networks, vol.3, Issue 1, pp. 23-43, 1990.
    [18] H. Jaeger, “The Echo State Approach to Analyzing and Training Recurrent Neural Networks,” Technical Report GMD Report 148, German National Research Center for Information Technology, 2001.
    [19] B. Hammer, “On the Approximation Capability of Recurrent Neural Networks. Neurocomputing,” in Neurocomputing, vol.31(1-4), pp. 107–123, 2000.
    [20] Y. Bengio, P. Simard, and P. Frasconi, “Learning Long-Term Dependencies with Gradient Descent is Difficult,” in IEEE Transactions on Neural Networks, vol.5, no.2, pp. 157–166, Mar. 1994.
    [21] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” in Neural Computation, vol.9, no.8, pp. 1735-1780, 1997.
    [22] F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to Forget: Continual Prediction with LSTM,” in Neural Computation, vol.12(10), pp. 2451-2471, 2000.
    [23] Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks,” Springer-Verlag Berlin Heidelberg, 2012.
    [24] I. Sutskever, O. Vinyals, and Q. Le, “Sequence to sequence learning with Neural Networks,” in Proceedings of the 27th International Conference on Neural Information Processing Systems, vol. 2, 2014.
    [25] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio, “Learning phrase representations using rnn encoder-decoder for statistical machine translation,” Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), 2014.
    [26] Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab Ward, “Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval,” in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, Issue 4, pp. 694–707, April 2016.
    [27] Lifeng Shang, Zhengdong Lu, and Hang Li, “Neural Responding Machine for Short-Text Conversation,” in Association for Computational Linguistics, vol.1, pp. 1577-1586, July 2015.
    [28] Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, “Neural machine translation by jointly learning to align and translate,” International Conference on Learning Representations 2015, pp.15.
    [29] Oriol Vinyals, Quoc Le, “A Neural Conversational Model,” International Conference on Machine Learning 2015.
    [30] W. Zaremba, I. Sutskever, “Learning to Execute,” in arXiv preprint, arXiv:1410.4615, 2014.
    [31] Darken, J. Chang and J. Moody, “Learning rate schedules for faster stochastic gradient search,” in Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop, pp. 1–11, 1992.
    [32] D. P. Kingma, and J. L. Ba, “Adam: A Method for Stochastic Optimization. International Conference on Learning Representations,” International Conference on Learning Representations 2015.
    [33] Fei-Fei Li, “Convolutional Neural Networks for Visual Recognition,” Course of Stanford Cs231, Lecture 4: Neural Network and Backpropagation, 2021.
    [34] K You, M Long, J Wang and MI Jordan, “How does learning rate decay help modern Neural Networks,” International Conference on Learning Representations 2020.
    [35] Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song, “On the convergence rate of training Recurrent Neural Networks,” 33rd Conference on Neural Information Processing Systems 2019, Vancouver, Canada, 2019.
    [36] Herbert Robbins and Sutton Monro, “A stochastic approximation method,” in Annals of Mathematical Statistics, vol. 22, pp. 400–407, 1951.
    [37] Ilya Loshchilov and Frank Hutter, “Decoupled Weight Decay Regularization,” International Conference on Learning Representations 2019.
    [38] Daihong Fu and Kenneth C. Dyer, “A Digital Background Calibration Technique for Time-Interleaved Analog-to-Digital Converters,” in IEEE Journal of Solid-State Circuits, vol. 33, no. 12, Dec. 1998.
    [39] Che-Wei Hsu and Soon-Jyh Chang, “A 1.6-GS/s 8-bit Flash Assisted SAR Time-Interleaved Analog-to-Digital Converter with Top-plate Residue Based Gain Calibration,” M.S. thesis, National Cheng Kung University Department of Electrical Engineering, 2021.
    [40] W. C. Black, Jr. and D. A. Hodges, “Time interleaved converter arrays,” in IEEE Journal of Solid-State Circuits, vol. 15, pp. 1022–1029, Dec. 1980.
    [41] Yih-chyun Jenq, “Digital spectra of nonuniformly sampled signals: Fundamentals and high-speed waveform digitizers,” in IEEE Instrumentation and Measurement Technology Conference, vol. 37, pp. 245–251, June 1988.
    [42] Yih-chyun Jenq, “Digital spectra of nonuniformly sampled signals: A robust sampling time offset estimation algorithm for ultra-high-speed waveform digitizers using interleaving,” in IEEE Transactions on Instrumentation and Measurement, vol. 39, pp. 71–75, Feb. 1990.
    [43] A. Petraglia and S. K. Mitra, “Analysis of mismatch effects among A/D converters in a time-interleaved waveform digitizer,” in IEEE Transactions on Instrumentation and Measurement, vol. 40, pp. 831–835, Oct. 1991.
    [44] M. Yotsuyanagi, T. Etoh, and K. Hirata, “A 10-b 50-MHz pipelined CMOS A/D converter with S/H,” in IEEE Journal of Solid-State Circuits, vol. 28, pp. 292–300, Mar. 1993.
    [45] C. S. G. Conroy, D. W. Cline, and P. R. Gray, “An 8-b 85-MS/s parallel pipeline A/D converter in 1-m CMOS,” in IEEE Journal of Solid-State Circuits, vol. 28, pp. 447–454, Apr. 1993.
    [46] K. Nakamura, M. Hotta, L. R. Carley, and D. J. Allstot, “An 85 mW, 10 b, 40 M sample/s CMOS parallel-pipelined ADC,” in IEEE Journal of Solid-State Circuits, vol. 30, pp. 173–183, Mar. 1995.
    [47] K. Y. Kim, N. Kusayanagi, and A. A. Abidi, “A 10-b, 100-MS/s CMOS A/D converter,” in IEEE Journal of Solid-State Circuits, vol. 32, pp. 302–311, Mar. 1997.
    [48] Behzad Razavi, “Design Considerations for Interleaved ADCs,” in IEEE Journal of Solid-State Circuits, vol. 48, no. 8, Aug. 2013.
    [49] Shuai C and Luke W, “All-digital calibration of timing mismatch error in time-interleaved analog-to-digital converters,” in IEEE Transactions on Very Large-Scale Integration (VLSI) Systems, vol. 25, no. 9, pp. 2552–2560, Sep. 2017.
    [50] Pietro Monsurrò and Alessandro Trifiletti, “Calibration of Time-Interleaved ADCs via Hermitianity-Preserving Taylor Approximations,” in IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 64, no. 4, pp. 357–361, Apr. 2017.
    [51] Yongtao Qiu1, Jie Zhou, Youjiang Liu and Yafan Huangfu, “A Novel Calibration Method of Gain and Time-skew Mismatches for Time-interleaved ADCs Based on Neural Network,” 2019 IEEE MTT-S International Wireless Symposium (IWS), pp. 1-3, 2019.
    [52] Jeff Zhang, Kartheek Rangineni, Zahra Ghodsi, and Siddharth Garg, “Thunderbolt: enabling aggressive voltage under scaling and timing error resilience for energy efficient deep learning accelerators,” in Proceedings of the 55th Annual Design Automation Conference, Article 19, pp. 1–6, 2018.
    [53] Ananda Samajdar, “SCALE-Sim: Systolic CNN Accelerator Simulator,” in Proceedings of the 26th Asia and South Pacific Design Automation Conference, pp. 235–241, 2021.
    [54] M. A. Bañuelos-Saucedo, J. Castillo-Hernández, S. Quintana-Thierry, R. Damián-Zamacona, J. Valeriano-Assem, R. E. Cervantes, R. Fuentes-González, G. Calva-Olmos & J. L. Pérez-Silva, “Implementation of A Neuron Model using FPGAS,” in Journal of Applied Research and Technology, vol.1, no.3, 2003.
    [55] L. Ranganath, D. Jay Kumar, P. Siva Nagendra Reddy, “Design of MAC unit in artificial Neural Network architecture using Verilog HDL,” 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), pp. 607-612, 2016.
    [56] Mustafa Gündüzalp, Çağla Sarvan, “Implementation of ANN Training Module on Field Programmable Gate Arrays,” 2019 Innovations in Intelligent Systems and Applications Conference (ASYU), pp. 1-6, 2019.

    下載圖示
    2026-09-14公開
    QR CODE