本論文旨在研製一套基於雙視角影像深度學習之運動訓練歷程的關節角度估測系統，該系統包含兩個子系統：子系統一：身體運動關節角度捕捉攝影系統，子系統二：深度學習關節活動度估測系統。子系統一由兩組攝影機、電腦和Wi-Fi無線基地台所組成，一組攝影機放置於運動員前方左右45度夾角約4公尺的位置，另一組攝影機放置於運動員前方及側面4公尺的位置，兩組攝影機可以涵蓋運動員及其運動範圍，同步記錄運動員的運動歷程影像訊號，攝影機透過有線傳輸方式至電腦寫成影片檔，再透過Wi-Fi傳輸至雲端資料庫儲存。子系統二為深度學習關節活動度估測系統，主要將子系統一上傳的影像檔進行運動員的關節活動度估測，利用兩攝影機同步取得的影像訊號先進行運動員與背景分離處理，接著使用基於OpenPose為基礎的二維人體關節點檢測演算法，抓取二維影像中人體的14個關節點位置，並將兩二維影像所估測的關節點位置轉換成三維空間的人體姿態，以定位三維空間中的關節點位置，再透過關節點投影至不同的身體剖面後，計算出球窩關節及屈戌關節的關節角度等運動指標，其中包括肩、肘、髖及膝關節角度。實驗結果顯示，二維關節點檢測的平均定位誤差為2~6個像素點，另外，左右45度視角比正面側面視角的關節角度估測更為準確。最後，實驗驗證本論文提出的深度學習關節活動度估測系統可以估測關節角度在可接受的誤差範圍內，肩、肘、髖、膝關節在屈曲-伸展的平均角度誤差分別為8.05±3.26、5.05±2.26、8.1±3.03及6.37±2.08度，而肩及髖關節在外展-內收的平均角度誤差分別為6.48±3.38及6.77±3.41度。研究結果驗證了影像身體動作捕捉系統應用於重量訓練之可行性，希冀本系統除了能幫助教練安排或調整訓練內容外，也能幫助運動員更有效率的自我訓練並得到良好的表現。

This thesis aims to develop a range-of-motion measurement system for recording and analyzing sport training courses based on a deep learning dual-view camera system. The system consists of two subsystems, subsystem 1: a body motion and joints’ angle capture system, and subsystem 2: a range-of-motion measurement system based on deep learning networks. The subsystem 1 consists of two sets of depth cameras, a personal computer and a Wi-Fi wireless station. Two cameras are placed in front of the athlete on both sides, one set is placed at an angle of 45 degrees and about 4 meters ahead of the right side of the athlete, and the other camera is placed at the same distance ahead of the left side of the athlete. Those two cameras simultaneously record the video of the athlete's training courses, and the video is segmented into a series of files transmitted to the cloud database via WiFi communication. The subsystem 2 is the cloud computing server with data storage that executes several algorithms to estimate the angles of joints of the athlete from the video files uploaded by the subsystem 1. The images from the video files obtained synchronously by the two cameras are first processed to remove the image of the athlete from the background environment, and then an OpenPose-based two-dimensional human joints detection algorithm is used to position the locations of 14 joints of the human body in the two-dimensional image. The estimated joint locations from the two images captured by the cameras are converted into a three-dimensional human body posture with the locations of the joints in the three-dimensional space, and then projected to different body planes to calculate the angles of ball-and-socket and hinge joints including shoulder, elbow, hip and knee angles. The experimental results show that the mean error of two-dimensional joints detection is 2 to 6 pixels, and the range of motion estimated by the view of 45 degrees is better than by that of the view of front and side. Finally, the proposed system can estimate the angles of the range of motion with an acceptable accuracy. The average root mean square error (RMSE) of the shoulder, elbow, hip and knee joints in flexion-extension were 8.05±3.26, 5.05±2.26, 8.1±3.03, and 6.37±2.08 degrees, respectively, and the average root mean square error (RMSE) of the shoulder and hip joints in abduction-adduction were 6.48±3.38 and 6.77±3.41 degrees, respectively. The experimental results have successfully validated the proposed system. In the future, we hope this system can help coaches to arrange or modify training programs, and help athletes improve self-training efficiency for obtaining better performance.

[1] T. Akimoto, Y. Suenaga, and R. S. Wallace, “Automatic creation of 3D facial models,” IEEE Computer Graphics and Applications, vol. 13, no. 5, pp. 16-22, 1993.
[2] M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele, “2d human pose estimation: New benchmark and state of the art analysis,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3686-3693.
[3] R. Bahr and T. Krosshaug, “Understanding injury mechanisms: a key component of preventing injuries in sport,” British Journal of Sports Medicine, vol. 39, no. 6, pp. 324-329, 2005.
[4] S. T. Barnard and M. A. Fischler, “Computational stereo,” ACM Computing Surveys, vol. 14, no. 4, pp. 553-572, 1982.
[5] K. E. Bartonietz, “Biomechanics of the snatch: Toward a higher training efficiency,” Strength & Conditioning Journal, vol. 18, no. 3, pp. 24-31, 1996.
[6] Z. Cao, T. Simon, S. E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7291-7299.
[7] C. L. Chen, C. L. Tai, and Y. F. Lio, “Virtual binocular vision systems to solid model reconstruction,” The International Journal of Advanced Manufacturing Technology, vol. 35, no. 3-4, pp. 379-384, 2007.
[8] M. El-Gohary and J. McNames, “Shoulder and elbow joint angle tracking with inertial sensors,” IEEE Transactions on Biomedical Engineering, vol. 59, no. 9, pp. 2635-2641, 2012.
[9] H. S. Fang, S. Xie, Y. W. Tai, and C. Lu, “Rmpe: Regional multi-person pose estimation,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2334-2343.
[10] Y. Furukawa and C. Hernández, “Multi-view stereo: A tutorial,” Foundations and Trends® in Computer Graphics and Vision, vol. 9, no. 1-2, pp. 1-148, 2015.
[11] J. Geng, “Structured-light 3D surface imaging: A tutorial,” Advances in Optics and Photonics, vol. 3, no. 2, pp. 128-160, 2011.
[12] H. Ghasemzadeh, V. Loseu, and R. Jafari, “Wearable coach for sport training: A quantitative model to evaluate wrist-rotation in golf,” Journal of Ambient Intelligence and Smart Environments, vol. 1, no. 2, pp. 173-184, 2009.
[13] V. Gourgoulis, N. Aggeloussis, P. Antoniou, C. Christoforidis, G. Mavromatis, and A. Garas, “Comparative 3-dimensional kinematic analysis of the snatch technique in elite male and female Greek weightlifters,” The Journal of Strength & Conditioning Research, vol. 16, no. 3, pp. 359-366, 2002.
[14] D. H. Hubel and T. N. Wiesel, “Receptive fields, binocular interaction and functional architecture in the cat's visual cortex,” The Journal of Physiology, vol. 160, no. 1, pp. 106-154, 1962.
[15] E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, and B. Schiele, “Deepercut: A deeper, stronger, and faster multi-person pose estimation model,” in European Conference on Computer Vision, 2016, pp. 34-50.
[16] U. Jørgensen and S. Winge, “Epidemiology of badminton injuries,” International Journal of Sports Medicine, vol. 8, no. 6, pp. 379-382, 1987.
[17] S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 221-231, 2012.
[18] H. Koda, K. Sagawa, K. Kuroshima, T. Tsukamoto, K. Urita, and Y. Ishibashi, “3D measurement of forearm and upper arm during throwing motion using body mounted sensor,” Journal of Advanced Mechanical Design, Systems, and Manufacturing, vol. 4, no. 1, pp. 167-178, 2010.
[19] S. McCann, “3d reconstruction from multiple images,” Foundations and Trends® in Computer Graphics and Vision, vol. 4, no. 4, pp. 287-404, 2015.
[20] D. D. Mccrady, P. Cummiskey, L. J. Doyle, and H. Forstrom, “Method and apparatus for determining the position of a mobile communication device,” ed: U.S. Patent 6 801 782, Oct. 5, 2004.
[21] D. A. Migliore, M. Matteucci, and M. Naccari, “A revaluation of frame difference in fast and robust motion detection,” in Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks, 2006, pp. 215-218.
[22] A. Newell, Z. Huang, and J. Deng, “Associative embedding: End-to-end learning for joint detection and grouping,” in Advances in Neural Information Processing Systems, 2017, pp. 2277-2287.
[23] C. C. Norkin and D. J. White, Measurement of joint motion: A guide to goniometry. Philadelphia: FA Davis, 2016.
[24] G. Papandreou, T. Zhu, N. Kanazawa, A. Toshev, J. Tompson, C. Bregler, and K. Murphy, “Towards accurate multi-person pose estimation in the wild,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4903-4911.
[25] M. Piccardi, “Background subtraction techniques: A review,” in 2004 IEEE International Conference on Systems, Man and Cybernetics, 2004, vol. 4, pp. 3099-3104.
[26] K. A. Russell, R. M. Palmieri, S. M. Zinder, and C. D. Ingersoll, “Sex differences in valgus knee angle during a single-leg drop jump,” Journal of Athletic Training, vol. 41, no. 2, p. 166, 2006.
[27] S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, pp. 519-528.
[28] S. Shi, Q. Wang, and X. Chu, “Performance modeling and evaluation of distributed deep learning frameworks on GPUs,” in 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 2018, pp. 949-957.
[29] M. H. Swartz, Textbook of Physical Diagnosis E-Book: History and Examination With Student Consult Online Access. Philadelphia: W. B. Saunders, 2014.
[30] H. Tang, K. Abe, K. Katoh, and M. Ae, Three-dimensional cinematographical analysis of the badminton forehand smash: Movements of the forearm and hand, Science and Racket Sports, Cambridge: E&FN Spon, 1995.
[31] C. Tomasi and T. Kanade, “Shape and motion from image streams under orthography: A factorization method,” International Journal of Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[32] C. M. Tran and L. M. Silverberg, “Optimal release conditions for the free throw in men's basketball,” Journal of Sports Sciences, vol. 26, no. 11, pp. 1147-1155, 2008.
[33] H. K. Unthoff, Shoulder injury and disability, The Workplace Safety and Insurance Appeals Tribunal, University of Ontario, 2010.
[34] S. E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, “Convolutional pose machines,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 4724-4732.
[35] J. A. Zelisko, H. B. Noble, and M. Porter, “A comparison of men's and women's professional basketball injuries,” The American Journal of Sports Medicine, vol. 10, no. 5, pp. 297-299, 1982.
[36] H. Y. Zhang, “Multiple moving objects detection and tracking based on optical flow in polar-log images,” in 2010 International Conference on Machine Learning and Cybernetics, 2010, pp. 1577-1582.
[37] 台灣趨勢研究網, 2018. [Online].Available: http://www.twtrend.com/share_cont.php?id=62
[38] 吳柏昇,“中風患者量化評估之穿戴式無線身體感測網路系統,” 國立成功大學電機工程學系碩士論文, 2014.
[39] 黄群星、王飛、張強、馬增益、嚴建華以及岑可法, “基於高速雙視角圖像 SVD 匹配的湍流預混火焰面三维擴散速度場的重建,” 燃燒科學與技術, vol. 13, no. 3, pp. 238-242, 2007.