簡易檢索 / 詳目顯示

研究生: 蘇可維
Su, Ko-Wei
論文名稱: 利用大型語言模型於低精度穿戴式感測行為辨識
Utilizing Large Language Model for Human Activity Recognition from Coarse-Grained Wearable Sensors
指導教授: 莊坤達
Chuang, Kun-Ta
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 資訊工程學系
Department of Computer Science and Information Engineering
論文出版年: 2025
畢業學年度: 113
語文別: 英文
論文頁數: 41
中文關鍵詞: 人類行為辨識大型語言模型穿戴式感測器低精度數據時間序列分析多模態融合
外文關鍵詞: Human Activity Recognition, Large Language Models, Wearable Sensors, Coarse-grained Data, Time Series Analysis, Multi-modal Fusion
相關次數: 點閱:7下載:3
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 人類行為辨識(HAR)是穿戴式感測技術中的重要研究領域,對於健康監測、運動分析和智慧生活應用具有重要意義。傳統的穿戴式感測器往往受限於低精度數據和計算資源限制,難以實現準確的行為辨識。隨著大型語言模型(LLM)的快速發展,其強大的序列理解和模式識別能力為解決此問題提供了新的可能。

    本論文提出了一種利用大型語言模型進行低精度穿戴式感測行為辨識的創新方法。我們設計了一個適應性框架,將穿戴式感測器的粗粒度時間序列數據轉換為語言模型可理解的表示形式,並利用預訓練語言模型的強大表示學習能力來提升行為辨識的準確性。

    這項研究為穿戴式設備的智能化發展提供了新的技術路徑,推動了大型語言模型在物聯網和普適計算領域的應用創新。

    Human Activity Recognition (HAR) is a critical research area in wearable sensing technology, with significant implications for health monitoring, sports analysis, and smart living applications. Traditional wearable sensors are often limited by low-precision data and computational constraints, making accurate activity recognition challenging. The rapid advancement of Large Language Models (LLMs) offers new possibilities with their powerful sequence understanding and pattern recognition capabilities.

    This thesis proposes an innovative approach that utilizes Large Language Models for human activity recognition from coarse-grained wearable sensors. We design an adaptive framework that transforms coarse-grained time-series data from wearable sensors into representations comprehensible by language models, leveraging the powerful representation learning capabilities of pre-trained language models to enhance activity recognition accuracy.

    This research provides a new technical pathway for the intelligent development of wearable devices and advances the application innovation of Large Language Models in the Internet of Things and ubiquitous computing domains.

    中文摘要 i Abstract ii Acknowledgment iii Contents iv List of Tables vi List of Figures vii 1 Introduction 1 2 Related Work 4 2.1 Human Activity Recognition with Wearable Sensors 4 2.1.1 Deep Learning Approaches for HAR 4 2.2 Transformer and Attention Mechanisms in HAR 5 2.3 Large Language Models for Sensor Data 5 2.3.1 High-Level Reasoning with LLMs 6 2.4 Energy Efficiency and Resource Constraints 6 2.4.1 Data Sampling and Downsampling Strategies 7 2.5 Time Series Imputation and Data Enhancement 7 2.6 Research Gaps and Motivation 8 3 Methodology 9 3.1 Stage 1: SAITS-based Time Series Imputation 10 3.1.1 Motivation and Advantages 10 3.1.2 SAITS Architecture 11 3.1.3 Integration into the HAR Pipeline 11 3.1.4 SAITS Implementation Details 12 3.1.5 Mathematical Formulation 12 3.2 Stage 2: SensorLLM Training for Human Activity Recognition 14 3.2.1 SensorLLM Framework 14 3.2.2 Task-Aware HAR Training and Objective 15 3.3 Integration and Evaluation 16 4 Experiments 19 4.1 Baseline Methods 19 4.1.1 Stage 1: SAITS Imputation Training 20 4.1.2 Stage 2: SensorLLM Training 20 4.1.3 HAR Performance Metrics 21 4.1.4 Stage 2: Human Activity Recognition Performance 22 4.1.5 Augmented QA with Advanced Feature Context 24 4.1.6 SensorLLM-Only Baseline: Effect of Downsampling Granularity 26 5 Conclusion 29 Bibliography 30

    [1] Andreas Bulling, Ulf Blanke, and Bernt Schiele. “A tutorial on human activity recognition using body-worn inertial sensors”. ACM Computing Surveys, 46(3), 2014, pp. 1–33.

    [2] Oscar D. Lara and Miguel A. Labrador. “A survey on human activity recognition using wearable sensors”. IEEE Communications Surveys & Tutorials, 15(3), 2013, pp. 1192–1209.

    [3] Muhammad Shoaib et al. “A survey of online activity recognition using mobile phones”. Sensors, 15(1), 2015, pp. 2059–2085.

    [4] Kaixuan Chen et al. “A comprehensive survey of deep learning-based human activity recognition”. Proceedings of the IEEE, 109(9), 2021, pp. 1601–1622.

    [5] Jindong Wang et al. “Deep learning for sensor-based activity recognition: A survey”. Pattern Recognition Letters, 119, 2019, pp. 3–11.

    [6] Jianbo Yang et al. “Deep convolutional neural networks on multichannel time series for human activity recognition”. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI), 2015.

    [7] Nils Y. Hammerla, Shane Halloran, and Thomas Ploetz. “Deep, convolutional, and recurrent models for human activity recognition using wearables”. arXiv preprint arXiv:1604.08880, 2016.

    [8] Charissa Ann Ronao and Sung-Bae Cho. “Human activity recognition with smartphone sensors using deep learning neural networks”. Expert Systems with Applications, 59, 2016, pp. 235–244.

    [9] Andrey Ignatov. “Real-time human activity recognition from accelerometer data using convolutional neural networks”. Applied Soft Computing, 62, 2018, pp. 915–922.

    [10] Ashish Vaswani et al. “Attention is all you need”. Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008.

    [11] Jacob Devlin et al. “BERT: Pre-training of deep bidirectional transformers for language understanding”. arXiv preprint arXiv:1810.04805, 2019.

    [12] Tom Brown et al. “Language models are few-shot learners”. Advances in Neural Information Processing Systems (NeurIPS), 33, 2020, pp. 1877–1901.

    [13] Shuangjian Li et al. “P2LHAP: Wearable sensor-based human activity recognition, segmentation and forecast through Patch-to-Label Seq2Seq Transformer”. arXiv preprint arXiv:2403.08214, 2024.

    [14] Wenjie Du et al. “SAITS: Self-attention-based imputation for time series”. Proceedings of the AAAI Conference on Artificial Intelligence, 37(8), 2023, pp. 7029–7037.

    [15] Hao Zhang et al. “MoPFormer: Motion-Primitive Transformer for wearable-sensor activity recognition”. arXiv preprint arXiv:2505.20744, 2025.

    [16] Kunpeng Zhao, Asahi Miyazaki, and Tsuyoshi Okita. “Detecting informative channels: ActionFormer”. arXiv preprint arXiv:2505.20739, 2025.

    [17] Zechen Li et al. “SensorLLM: Aligning large language models with motion sensors for human activity recognition”. arXiv preprint arXiv:2410.10624, 2024.

    [18] Yuwei Zhang et al. “SensorLM: Learning the language of wearable sensors”. arXiv preprint arXiv:2506.09108, 2025.

    [19] Gabriele Civitarese et al. “Large language models are zero-shot recognizers for activities of daily living”. arXiv preprint arXiv:2407.01238, 2024.

    [20] Sijie Ji, Xinzhe Zheng, and Chenshu Wu. “HARGPT: Are LLMs zero-shot human activity recognizers?” Proceedings of the 2024 IEEE International Workshop on Foundation Models for Cyber-Physical Systems & Internet of Things (FMSys), IEEE, 2024, pp. 38–43.

    [21] Emilio Ferrara. “Large language models for wearable sensor-based human activity recognition, health monitoring, and behavioral modeling: A survey of early trends, datasets, and challenges”. Sensors, 24(15), 2024, p. 5045.

    [22] Xiaomin Ouyang and Mani Srivastava. “LLMSense: Harnessing LLMs for high-level reasoning over spatiotemporal sensor traces”. arXiv preprint arXiv:2403.19857, 2024.

    [23] Yuan Sun and Jorge Ortiz. “An AI-based system utilizing IoT-enabled ambient sensors and LLMs for complex activity tracking”. arXiv preprint arXiv:2407.02606, 2024.

    [24] Shentong Mo et al. “IoT-LM: Large multisensory language models for the Internet of Things”. arXiv preprint arXiv:2407.09801, 2024.

    [25] Franklin Y. Ruan et al. “Foundation models for wearable movement data in mental health research”. arXiv preprint arXiv:2411.15240, 2024.

    [26] Mohammad Rafid Ul Islam, Prasad Tadepalli, and Alan Fern. “Self-attention-based diffusion model for time-series imputation in partial blackout scenarios”. Proceedings of the AAAI Conference on Artificial Intelligence, 2025.

    [27] Shing Chan et al. “CAPTURE-24: A large dataset of wrist-worn activity tracker data collected in the wild for human activity recognition”. Scientific Data, 11(1), 2024, p. 1135.

    下載圖示 校內:立即公開
    校外:立即公開
    QR CODE