| 研究生: |
吳彧賢 Wu, Yu-Hsien |
|---|---|
| 論文名稱: |
基於數位孿生的多相機人體動作檢測系統及其在精確老年步態分析的應用 Digital Twin-Based Multi-Camera Human Motion Detection System with Application to Precise Elder Gait Analysis |
| 指導教授: |
蘇文鈺
Su, Wen-Yu |
| 學位類別: |
碩士 Master |
| 系所名稱: |
電機資訊學院 - 人工智慧科技碩士學位學程 Graduate Program of Artificial Intelligence |
| 論文出版年: | 2024 |
| 畢業學年度: | 112 |
| 語文別: | 英文 |
| 論文頁數: | 348 |
| 中文關鍵詞: | 數位孿生 、動作捕捉 、步態分析 、姿態估計 、多相機 、AlphaPose |
| 外文關鍵詞: | Digital Twin Technology, Motion Capture, Gait Analysis, Pose Estimation, Multi-Camera, AlphaPose |
| 相關次數: | 點閱:103 下載:15 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
傳統的數位孿生技術通常包括製造實體機器人並在虛擬空間中創建一個匹配的模型。然後調整兩個模型以匹配彼此,確保虛擬空間中的實驗準確可靠。然而,當模型基於人類時,保持實體和虛擬版本的一致性是一個挑戰。本文利用動作捕捉技術進行校準,精確記錄並分析人類運動數據,並將其應用於虛擬模型中,以確保實體與虛擬結果之間的一致性,從而在虛擬實驗中獲得可靠的結果。此外,虛擬空間中的實驗有助於配置真實世界環境的設置。
其中的一個應用是老年人的步態分析。首先,我們使用動作捕捉技術捕捉他們的步行動作三維數據,然後在虛擬世界中創建一個相似的環境。實體運動數據隨後導入虛擬模型進行步態分析模擬。步態分析基於多相機姿態估計模型(AlphaPose),重建精確的三維骨架結果。完成步態分析後,持續調整虛擬條件以模擬現實世界的限制和情況,例如相機數量的影響,並管理現實世界環境設置的權衡。最後,我們將虛擬環境設置在實體端重現,相應地定位相機角度。本論文也報告了誤差分析。
Traditional digital twin technologies usually involve making a physical robot and creating a matching model in a virtual space. Both models are then adjusted to match each other, ensuring that experiments in virtual space are accurate and reliable. However, when the model is based on a human, keeping the physical and virtual versions consistent is a challenge. This paper utilizes motion capture technology for calibration, precisely recording and analyzing human motion data, and applying it to virtual models to ensure consistency between the physical and virtual outcomes, thus achieving reliable results in virtual experiments. Furthermore, the experiments in virtual space help to configure the real word environment setup.
One application of this method is the gait analysis of the elderly. First, we capture their walking motion in three dimensions using motion capture technology, then create a similar environment in the virtual world. The physical motion data is then imported into the virtual model for gait analysis simulation. Gait analysis is based on a multi-camera pose estimation model (AlphaPose), which reconstructs accurate three-dimensional skeletal outcomes. After completing the gait analysis, continuous adjustments are made to the virtual conditions to simulate real-world limitations and circumstances, such as the impact of the number of cameras, and manage trade-offs in the setup of real-world environment. Finally, we replicate the virtual environment setup on the physical with six cameras, positioning the camera angles accordingly. The error analysis is also reported in this thesis.
[1] Tao, Fei, et al. "Digital twin in industry: State-of-the-art." IEEE Transactions on industrial informatics 15.4 (2018): 2405-2415.
[2] NEURON MOTION CAPTURE https://neuronmocap.com/
[3] Sedgwick, Philip, and Nan Greenwood. "Understanding the Hawthorne effect." Bmj 351 (2015).
[4] D’Antonio, Erika, et al. "Validation of a 3D markerless system for gait analysis based on OpenPose and two RGB webcams." IEEE Sensors Journal 21.15 (2021): 17064-17075.
[5] Cao, Zhe, et al. "Realtime multi-person 2d pose estimation using part affinity fields." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
[6] Seel, Thomas, Jorg Raisch, and Thomas Schauer. "IMU-based joint angle measurement for gait analysis." Sensors 14.4 (2014): 6891-6909.
[7] Vargas-Valencia, Laura Susana, et al. "An IMU-to-body alignment method applied to human gait analysis." Sensors 16.12 (2016): 2090.
[8] Lugaresi, Camillo, et al. "Mediapipe: A framework for building perception pipelines." arXiv preprint arXiv:1906.08172 (2019).
[9] Bazarevsky, Valentin, et al. "Blazepose: On-device real-time body pose tracking." arXiv preprint arXiv:2006.10204 (2020).
[10] Fang, Hao-Shu, et al. "Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time." IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
[11] Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[12] Jin, Sheng, et al. "Whole-body human pose estimation in the wild." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16. Springer International Publishing, 2020.
[13] Xiu, Yuliang, et al. "Pose Flow: Efficient online pose tracking." arXiv preprint arXiv:1802.00977 (2018).
[14] Fang, Hao-Shu, et al. "Rmpe: Regional multi-person pose estimation." Proceedings of the IEEE international conference on computer vision. 2017.
[15] Berndt, Donald J., and James Clifford. "Using dynamic time warping to find patterns in time series." Proceedings of the 3rd international conference on knowledge discovery and data mining. 1994.
[16] Unity https://unity.com/
[17] OpenCV https://opencv.org/
[18] Amber Free High Poly 3D Model https://www.cgtrader.com/free-3d-models/character/woman/amber-free-high-poly-3d-model Author: Martinthedark-
[19] St, Lars, and Svante Wold. "Analysis of variance (ANOVA)." Chemometrics and intelligent laboratory systems 6.4 (1989): 259-272.
[20] Hartley, Richard I., and Peter Sturm. "Triangulation." Computer vision and image understanding 68.2 (1997): 146-157.
[21] Direct linear transformation https://en.wikipedia.org/wiki/Direct_linear_transformation
[22] Golub, Gene H., and Christian Reinsch. "Singular value decomposition and least squares solutions." Handbook for Automatic Computation: Volume II: Linear Algebra. Berlin, Heidelberg: Springer Berlin Heidelberg, 1971. 134-151.
[23] Schafer, Ronald W. "What is a Savitzky-Golay filter?[lecture notes]." IEEE Signal processing magazine 28.4 (2011): 111-117.
[24] Whittle, Michael W. Gait analysis: an introduction. Butterworth-Heinemann, 2014.
[25] Di Gregorio, Raffaele, and Lucas Vocenas. "Identification of gait-cycle phases for prosthesis control." Biomimetics 6.2 (2021): 22.