隨著人工智慧（AI）技術的快速發展，深度神經網路與大型語言模型在各領域展現了卓越的預測性能，但其背後龐大的計算資源消耗與碳排放問題也日益受到關注，催生了「綠色人工智慧」（Green AI）的研究領域。現有的 Green AI 動態模型選擇方法（如模型級聯與模型路由）雖然能透過在輕量模型與大型模型間切換來節省能源，但往往採用固定的參數設定（例如固定深度的決策樹），導致無法靈活適應不同難度與特性的資料集。
本研究旨在改進現有的動態模型選擇機制，提出一種名為「Auto-G」的自動最佳化演算法。Auto-G 能夠針對不同的資料集特性，自動遍歷並搜尋 Green Model（（決策樹）的最佳深度，並引入「指數級安全性懲罰」機制，防止模型在準確率顯著下降時仍強行使用輕量模型，從而確保系統的可靠性。
本研究設計了三種不同複雜度的測試場景：手寫數字辨識（Digits）、肺炎 X 光影像（PneumoniaMNIST）與心電圖資料（ECG），並將 Auto-G 與傳統固定策略（Fixed Strategy）及單一模型策略進行實證比較。實驗結果顯示，在 Digits 資料集中，Auto-G 路由方法相較於固定策略，在維持高準確率的同時顯著降低了運算時間；在複雜度較高的 Pneumonia 資料集中，Auto-G 級聯方法成功將準確率從原方法的 82% 提升至 85%，運運算速度比純大型模型快 50%。而在時間序列的 ECG 任務中，Auto-G的優化效益尤為顯著：其級聯方法較固定策略大幅縮減了 35% 的總推論時間，路由方法更達成了全面性的性能超越，不僅推論時間進一步加快，準確率更反超提升了0.44%，實現了節能與精準度的雙贏。研究結果證實，Auto-G 演算法能有效解決固定策略缺乏彈性的痛點，在計算成本與模型準確率之間取得更佳的平衡，為邊緣運算與資源受限環境下的 AI 部署提供具可行性的解決方案。

With the rapid development of artificial intelligence (AI) technology, deep neural networks and large language models have demonstrated outstanding predictive performance in various fields. However, the enormous computational resources consumed and carbon emissions associated with these technologies have increasingly drawn attention, giving rise to the research field of "Green AI." Existing dynamic model selection methods for Green AI (such as model cascading and model routing) can save energy by switching between lightweight and large models, but they often employ fixed parameter settings (e.g., fixed-depth decision trees), resulting in an inflexible ability to adapt to datasets of varying difficulty and characteristics.

This research aims to improve existing dynamic model selection mechanisms by proposing an automated optimization algorithm called "Auto-G." Auto-G automatically traverses and searches for the optimal depth of the Green Model (decision tree) based on different dataset characteristics and introduces an "exponential safety penalty" mechanism to prevent the model from forcibly using a lightweight model when accuracy significantly decreases, thereby ensuring system reliability.
This study designed three test scenarios with different complexities: handwritten digit recognition (Digits), pneumonia X-ray images (PneumoniaMNIST), and electrocardiogram (ECG) data, and empirically compared Auto-G with traditional fixed strategies and single-model strategies. Experimental results show that on the Digits dataset, the Auto-G routing method significantly reduced computation time compared to the fixed strategy while maintaining high accuracy. On the more complex Pneumonia dataset, the Auto-G cascade method successfully improved accuracy from 82% to 85% compared to the original method, and its computation speed was 50% faster than a purely large model. In the time-series ECG task, the optimization benefits of Auto-G were particularly significant: its cascade method reduced the total inference time by 35% compared to the fixed strategy, and the routing method achieved a comprehensive performance improvement, not only further accelerating inference time but also surpassing it in accuracy by 0.44%, achieving a win-win situation of energy saving and accuracy. The research results confirm that the Auto-G algorithm can effectively solve the pain point of the lack of flexibility in fixed strategies, achieve a better balance between computational cost and model accuracy, and provide a feasible solution for AI deployment in edge computing and resource-constrained environments.

[1] C.-J. Wu, R. Raghavendra, U. Gupta, B. Acun, et al., “Sustainable AI: Environmental implications, challenges and opportunities,” Proceedings of Machine Learning and Systems (MLSys), vol. 4, pp. 795-813, 2022. https://doi.org/10.48550/arXiv.2111.00364
[2] R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, "Green AI," Communications of the ACM, vol. 63, no. 12, pp. 54–63, 2020. doi: 10.1145/3381831. https://doi.org/10.1145/3381831
[3] R. Verdecchia, J. Sallou, and L. Cruz, "A systematic review of green AI," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 13, no. 4, p. e1507, 2023. https://doi.org/10.1002/widm.1507
[4] G. Victor Daniel et al., "AI Model Optimization Techniques," in Model Optimization Methods for Efficient and Edge AI: Federated Learning Architectures, Frameworks and Applications, Springer, 2025, pp. 87–108. https://doi.org/10.1002/9781394219230.ch5
[5] E. Cruciani and R. Verdecchia, " Choosing to Be Green: Advancing Green AI via Dynamic Model Selection," in Proc. Green-aware AI 2025 @ ECAI 2025, 2025. https://doi.org/10.48550/arXiv.2509.19996
[6] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. 2001 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), 2001, vol. 1, pp. I-511–I-518. https://doi.org/10.1109/CVPR.2001.990517
[7] R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Comput., vol. 3, no. 1, pp. 79–87, 1991. https://doi.org/10.1162/neco.1991.3.1.79
[8] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 25, pp. 1097–1105, 2012. https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
[9] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Represent. (ICLR), 2015. https://doi.org/10.48550/arXiv.1409.1556
[10] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
[11] A. Vaswani et al., "Attention is all you need," in Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
[12] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in Proc. Conf. North Amer. Chapter Assoc. Comput. Linguist.: Human Lang. Technol. (NAACL-HLT), 2019, pp. 4171–4186. https://doi.org/10.18653/v1/N19-1423
[13] T. B. Brown et al., "Language models are few-shot learners," in Adv. Neural Inf. Process. Syst. (NeurIPS), 2020, vol. 33, pp. 1877–1901. https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
[14] J. Kaplan et al., "Scaling laws for neural language models," arXiv preprint arXiv:2001.08361, 2020. https://doi.org/10.48550/arXiv.2001.08361
[15] E. Strubell, A. Ganesh, and A. McCallum, "Energy and policy considerations for deep learning in NLP," in Proc. 57th Annu. Meeting Assoc. Comput. Linguist. (ACL), 2019, pp. 3645–3650. https://doi.org/10.18653/v1/P19-1355
[16] D. Patterson et al., "Carbon emissions and large neural network training," arXiv preprint arXiv:2104.10350, 2021. https://doi.org/10.48550/arXiv.2104.10350
[17] Cushman & Wakefield, "2022 Global data center market comparison," Cushman & Wakefield Research, 2022.
[18] A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres, "Quantifying the carbon emissions of machine learning," arXiv preprint arXiv:1910.09700, 2019. https://doi.org/10.48550/arXiv.1910.09700
[19] P. Henderson, J. Hu, J. Romoff, E. Brunskill, D. Jurafsky, and J. C. Duchi, "Towards the systematic reporting of the energy and carbon footprints of machine learning," J. Mach. Learn. Res. (JMLR), vol. 21, no. 248, pp. 1–43, 2020. https://jmlr.org/papers/v21/20-312.html
[20] P. Dhar, "The carbon impact of artificial intelligence," Nat. Mach. Intell., vol. 2, no. 8, pp. 423–425, 2020. https://doi.org/10.1038/s42256-020-0219-9
[21] N. P. Jouppi et al., "In-datacenter performance analysis of a tensor processing unit," in Proc. 44th Annu. Int. Symp. Comput. Archit. (ISCA), 2017, pp. 1–12. https://doi.org/10.1145/3079856.3080246
[22] C.-J. Wu et al., "Machine learning at Facebook: Understanding inference at the edge," in Proc. 2019 IEEE Int. Symp. High Perform. Comput. Archit. (HPCA), 2019, pp. 331–344. https://doi.org/10.1109/HPCA.2019.00048
[23] A. S. Luccioni, S. Viguier, and A.-L. Ligozat, "Estimating the carbon footprint of BLOOM, a 176B parameter language model," J. Mach. Learn. Res. (JMLR), vol. 24, no. 253, pp. 1–15, 2023. https://jmlr.org/papers/v24/23-0069.html
[24] G. Bertazzini, C. Albisani, D. Baracchi, D. Shullani, and R. Verdecchia, "The hidden cost of an image: Quantifying the energy consumption of AI image generation," arXiv preprint arXiv:2506.17016, 2025. https://doi.org/10.48550/arXiv.2506.17016
[25] E. Garcia-Martin, C. F. Rodrigues, G. Riley, and H. Grahn, "Estimation of energy consumption in machine learning," J. Parallel Distrib. Comput., vol. 134, pp. 75–88, 2019. https://doi.org/10.1016/j.jpdc.2019.07.007
[26] J. Dodge et al., "Measuring the carbon intensity of AI in cloud instances," in Proc. 2022 ACM Conf. Fairness, Accountability, and Transparency (FAccT), 2022, pp. 1877–1894. https://doi.org/10.1145/3531146.3533234
[27] W. A. Wulf and S. A. McKee, "Hitting the memory wall: implications of the obvious," ACM SIGARCH Comput. Archit. News, vol. 23, no. 1, pp. 20–24, 1995. https://doi.org/10.1145/216585.216588
[28] D. A. Patterson, “Latency lags bandwidth,” Communications of the ACM, vol. 47, no. 10, pp. 71-75, 2004. https://doi.org/10.1145/1022594.1022596
[29] M. Horowitz, "1.1 Computing's energy problem (and what we can do about it)," in 2014 IEEE Int. Solid-State Circuits Conf. Dig. Tech. Papers (ISSCC), 2014, pp. 10–14. https://doi.org/10.1109/ISSCC.2014.6757323
[30] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, "Efficient processing of deep neural networks: A tutorial and survey," Proc. IEEE, vol. 105, no. 12, pp. 2295–2329, 2017. https://doi.org/10.1109/JPROC.2017.2761740
[31] J. Lin et al., "MCUNet: Tiny deep learning on IoT devices," in Adv. Neural Inf. Process. Syst. (NeurIPS), 2020, vol. 33, pp. 11711–11722. https://proceedings.neurips.cc/paper/2020/hash/87b5a83a0593452f3605a60b943715d3-Abstract.html
[32] H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, "Once-for-all: Train one network and specialize it for efficient deployment," in Proc. Int. Conf. Learn. Represent. (ICLR), 2020. https://doi.org/10.48550/arXiv.1908.09791
[33] S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding," in Proc. Int. Conf. Learn. Represent. (ICLR), 2016. https://doi.org/10.48550/arXiv.1510.00149
[34] Y.-H. Chen, J. Emer, and V. Sze, "Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks," ACM SIGARCH Comput. Archit. News, vol. 44, no. 3, pp. 367–379, 2016. https://doi.org/10.1145/3007787.3001177
[35] C. Ji et al., "Memory-efficient deep learning inference with incremental weight loading and data layout reorganization on edge systems," J. Syst. Archit., vol. 118, p. 102183, 2021. https://doi.org/10.1016/j.sysarc.2021.102183
[36] A. Gholami et al., "A survey of quantization methods for efficient neural network inference," arXiv preprint arXiv:2103.13630, 2021. https://doi.org/10.48550/arXiv.2103.13630
[37] L. F. Anthony, B. Kanding, and R. Selvan, "Carbontracker: Tracking and predicting the carbon footprint of training deep learning models," arXiv preprint arXiv:2007.03051, 2020. https://doi.org/10.48550/arXiv.2007.03051
[38] K. Lottick, S. Susarla, A. Mathur, and L. Mackey, "Energy usage reports: Environmental awareness as part of algorithmic accountability," arXiv preprint arXiv:1911.08354, 2019. https://doi.org/10.48550/arXiv.1911.08354
[39] F. N. Iandola et al., "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size," arXiv preprint arXiv:1602.07360, 2016. https://doi.org/10.48550/arXiv.1602.07360
[40] A. G. Howard et al., "MobileNets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017. https://doi.org/10.48550/arXiv.1704.04861
[41] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "MobileNetV2: Inverted residuals and linear bottlenecks," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 4510–4520. https://doi.org/10.1109/CVPR.2018.00474
[42] X. Zhang, X. Zhou, M. Lin, and J. Sun, "ShuffleNet: An extremely efficient convolutional neural network for mobile devices," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 6848–6856. https://doi.org/10.1109/CVPR.2018.00716
[43] K. Han et al., "GhostNet: More features from cheap operations," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2020, pp. 1580–1589. https://doi.org/10.1109/CVPR42600.2020.00165
[44] G. Huang, S. Liu, L. Van der Maaten, and K. Q. Weinberger, "CondenseNet: An efficient DenseNet using learned group convolutions," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 2752–2761. https://doi.org/10.1109/CVPR.2018.00291
[45] M. Tan and Q. Le, "EfficientNet: Rethinking model scaling for convolutional neural networks," in Proc. 36th Int. Conf. Mach. Learn. (ICML), 2019, pp. 6105–6114. https://proceedings.mlr.press/v97/tan19a.html
[46] H. Cai, L. Zhu, and S. Han, "ProxylessNAS: Direct neural architecture search on target task and hardware," in Proc. 7th Int. Conf. Learn. Represent. (ICLR), 2019. https://doi.org/10.48550/arXiv.1812.00332
[47] A. Howard et al., "Searching for MobileNetV3," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2019, pp. 1314–1324. https://doi.org/10.1109/ICCV.2019.00140
[48] Y. LeCun, J. Denker, and S. Solla, "Optimal brain damage," in Adv. Neural Inf. Process. Syst. (NeurIPS), 1989, vol. 2, pp. 598–605. https://proceedings.neurips.cc/paper/1989/hash/6c9882bbac1c7093bd25041881277658-Abstract.html
[49] S. Han, J. Pool, J. Tran, and W. Dally, "Learning both weights and connections for efficient neural network," in Adv. Neural Inf. Process. Syst. (NeurIPS), 2015, vol. 28, pp. 1135–1143. https://papers.nips.cc/paper_files/paper/2015/hash/ae0eb3eed39d2bcef4622b2499a05fe6-Abstract.html
[50] J. Frankle and M. Carbin, "The lottery ticket hypothesis: Finding sparse, trainable neural networks," in Proc. 7th Int. Conf. Learn. Represent. (ICLR), 2019. https://doi.org/10.48550/arXiv.1803.03635
[51] T. Hoefler, D. Alistarh, T. Ben-Nun, N. Dryden, and A. Peste, "Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks," arXiv preprint arXiv:2102.00554, 2021. https://doi.org/10.48550/arXiv.2102.00554
[52] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, "Pruning filters for efficient convnets," in Proc. 5th Int. Conf. Learn. Represent. (ICLR), 2017. https://doi.org/10.48550/arXiv.1608.08710
[53] W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, "Learning structured sparsity in deep neural networks," in Adv. Neural Inf. Process. Syst. (NeurIPS), 2016, vol. 29, pp. 2074–2082. https://proceedings.neurips.cc/paper_files/paper/2016/hash/41bfd20a38bb1b0bec75acf0845530a7-Abstract.html
[54] Y. He, X. Zhang, and J. Sun, “Channel pruning for accelerating very deep neural networks,” in Proceedings of the IEEE international conference on computer vision (ICCV), 2017, pp. 1389-1397. https://doi.org/10.1109/ICCV.2017.155
[55] P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, "Importance estimation for neural network pruning," arXiv preprint arXiv:1906.10771, 2019. https://doi.org/10.48550/arXiv.1906.10771
[56] D. Blalock, J. J. G. Ortiz, J. Frankle, and J. Guttag, "What is the state of neural network pruning?," arXiv preprint arXiv:2003.03033, 2020. https://doi.org/10.48550/arXiv.2003.03033
[57] Z. Liu, M. Sun, T. Zhou, G. Huang, and T. Darrell, "Rethinking the value of network pruning," in Proc. 7th Int. Conf. Learn. Represent. (ICLR), 2019. https://doi.org/10.48550/arXiv.1810.05270
[58] R. Krishnamoorthi, "Quantizing deep convolutional networks for efficient inference: A whitepaper," arXiv preprint arXiv:1806.08342, 2018. https://doi.org/10.48550/arXiv.1806.08342
[59] H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius, "Integer quantization for deep learning inference: Principles and empirical evaluation," arXiv preprint arXiv:2004.09602, 2020. https://doi.org/10.48550/arXiv.2004.09602
[60] M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort, "Up or Down? Adaptive Rounding for Post-Training Quantization," in Proc. 37th Int. Conf. Mach. Learn. (ICML), 2020, pp. 7192–7200. https://proceedings.mlr.press/v119/nagel20a.html
[61] B. Jacob et al., "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 2704–2713. https://doi.org/10.1109/CVPR.2018.00286
[62] Y. Bengio, N. Léonard, and A. Courville, "Estimating or propagating gradients through stochastic neurons for conditional computation," arXiv preprint arXiv:1308.3432, 2013. https://doi.org/10.48550/arXiv.1308.3432
[63] S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy, and D. S. Modha, "Learned Step Size Quantization," in Proc. 8th Int. Conf. Learn. Represent. (ICLR), 2020. https://doi.org/10.48550/arXiv.1902.08153
[64] M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio, "Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1," arXiv preprint arXiv:1602.02830, 2016. https://doi.org/10.48550/arXiv.1602.02830
[65] M. Kim, W. Saad, M. Mozaffari, and M. Debbah, "On the Tradeoff Between Energy, Precision, and Accuracy in Federated Quantized Neural Networks," in Proc. 2022 IEEE Int. Conf. Commun. (ICC), 2022, pp. 2194–2199. https://doi.org/10.1109/ICC45855.2022.9838362
[66] Z. Dong, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer, "HAWQ: Hessian Aware Quantization of Neural Networks with Mixed-Precision," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2019, pp. 293–302. https://doi.org/10.1109/ICCV.2019.00038
[67] C. Buciluă, R. Caruana, and A. Niculescu-Mizil, "Model Compression," in Proc. 12th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2006, pp. 535–541. https://doi.org/10.1145/1150402.1150464
[68] G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," arXiv preprint arXiv:1503.02531, 2015. https://doi.org/10.48550/arXiv.1503.02531
[69] J. Gou, B. Yu, S. J. Maybank, and D. Tao, "Knowledge Distillation: A Survey," Int. J. Comput. Vis., vol. 129, no. 6, pp. 1789–1819, 2021. https://doi.org/10.1007/s11263-021-01453-z
[70] A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, "FitNets: Hints for Thin Deep Nets," in Proc. 3rd Int. Conf. Learn. Represent. (ICLR), 2015. https://doi.org/10.48550/arXiv.1412.6550
[71] J. Tang, R. Shivanna, Z. Zhao, D. Lin, A. Singh, E. H. Chi, and S. Jain, "Understanding and Improving Knowledge Distillation," arXiv preprint arXiv:2002.03532, 2020. https://doi.org/10.48550/arXiv.2002.03532
[72] J. Yim, D. Joo, J. Bae, and J. Kim, "A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 4133–4141. https://openaccess.thecvf.com/content_cvpr_2017/html/Yim_A_Gift_From_CVPR_2017_paper.html
[73] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter," arXiv preprint arXiv:1910.01108, 2019. https://doi.org/10.48550/arXiv.1910.01108
[74] A. Polino, R. Pascanu, and D. Alistarh, "Model Compression via Distillation and Quantization," in Proc. 6th Int. Conf. Learn. Represent. (ICLR), Vancouver, Canada, 2018. https://doi.org/10.48550/arXiv.1802.05668
[75] J. H. Cho and B. Hariharan, "On the Efficacy of Knowledge Distillation," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2019, pp. 4794–4802. https://doi.org/10.1109/ICCV.2019.00489
[76] S. I. Mirzadeh et al., "Improved Knowledge Distillation via Teacher Assistant," in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 04, 2020, pp. 5191–5198. https://doi.org/10.1609/aaai.v34i04.5963