本研究以網球訓練系統為例，建立一個基於人體三維姿態來評估揮拍姿勢好壞之系統。在人體三維姿態擷取方面，當使用者進行揮拍時，先以Kinect彩色相機與深度感測器取得彩色影像與深度影像，然後將OpenPose提取出彩色影像中的人體二維姿態與深度影像資訊進行映射與內插，最後得到人體骨架之關節三維座標。一般來說，不同的人做同一個動作時之快慢存在差異，因此由動態時間規整的方法來解決動作不對齊的問題。將動作對齊後，能更合理的對兩動作的相似度進行評估。此外，本研究建立之系統可將使用者的揮拍動作和標準動作進行比對，並以定量指標給出關鍵動作處的相似度與擊球點時機之好壞判定，作為使用者動作改善之方向。

In this study, a system based on 3-D human pose estimation is established to evaluate users’ swing posture for tennis training. For 3-D human pose extraction, when the user is swinging, color images and depth images are captured at first with the Kinect color camera and depth sensor respectively. Next, utilizing OpenPose to extract 2-D human pose data and map to the corresponding depth image so that the depth information can be obtained via interpolation. At last, combining the depth information and the 2-D data to yield the 3-D data for all joints. Generally speaking, there exists difference in speed when different people perform the same exercise. Therefore, dynamic time warping (DTW) is used to solve the unaligned problem when comparing postures of different people. Once all postures can be aligned on a time axis, evaluation for judging the similarity between two postures becomes more reasonable. Furthermore, the established system is able to compare users’ swing with the built-in standard postures, and provides quantitative indices for judging the similarity of key postures as well as timing of contact point. All these information can help users to improve themselves.

[1] Z. Cao et al. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
[2] T. Simon et al. Hand Keypoint Detection in Single Images using Multiview Bootstrapping. In CVPR, 2017.
[3] Z. Zhang. Microsoft Kinect Sensor and its Effect. IEEE multimedia, 19(2):4-10, 2012.
[4] J. Shotton et al. Real-time Human pose Recognition in Parts from Single Depth Images. In CVPR 2011, pages 1297-1304. IEEE, 2011.
[5] Confidence Maps. https://www.programmersought.com/article/37047327059/.
[6] W. Chen et al. Fall Detection Based on Key Points of Human-skeleton using OpenPose. Symmetry, 12(5):744, 2020.
[7] M. Nakai et al. Prediction of Basketball Free Throw Shooting by OpenPose. In JSAI International Symposium on Artificial Intelligence, pages 435-446. Springer, 2018.
[8] 李嘉晃林建良. 人的姿勢與偵測. PhD thesis, 2005.
[9] Z. Yang and J. Luo. Personalized pose estimation for body language understanding. In 2017 IEEE International Conference on Image Processing (ICIP), pages 126-130. IEEE, 2017.
[10] Y. K. Ryu and C. Oh. Rf Signal Synchronized Low Cost Motion Capture Device. In 2007 Asia-Pacific Microwave Conference, pages 1-4. IEEE, 2007.
[11] H. Ukida et al. Human Motion Capture System using Color Markers and Silhouette. In 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings, pages 151-156. IEEE, 2006.
[12] R. A. El-laithy et al. Study on the Use of Microsoft Kinect for Robotics Applications. In Proceedings of the 2012 IEEE/ION Position, Location and Navigation Symposium, pages 1280-1288. IEEE, 2012.
[13] Z. Cao et al. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR, 2017.
[14] Y. Cheng. Mean Shift, Mode Seeking, and Clustering. IEEE transactions on pattern analysis and machine intelligence, 17(8):790-799, 1995.
[15] E. J. Keogh and M. J. Pazzani. Derivative Dynamic Time Warping. In Proceedings of the 2001 SIAM international conference on data mining, pages 1-11. SIAM, 2001.
[16] V. U. Thompson et al. Performance Evaluation of Similarity Measures on Similar and Dissimilar Text Retrieval. In 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K), volume 1, pages 577-584. IEEE, 2015.
[17] S. Chen et al. Pose Trainer: Correcting Exercise Posture using Pose Estimation.
arXiv preprint arXiv:2006.11718, 2020.
[18] L. Buthe et al. A Wearable Sensing System for Timing Analysis in Tennis. In 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pages 43-48. IEEE, 2016.
[19] P. Xia et al. Learning Similarity with Cosine Similarity Ensemble. Information Sciences, 307:39-52, 2015.
[20] G. Bradski. The OpenCV Library. Dr. Dobb's Journal of Software Tools, 2000.
[21] Y. Jia et al. Caffe: Convolutional Architecture for Fast Feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, pages 675-678, 2014.
[22] S. E. Wei et al. Convolutional Pose Machines. In CVPR, 2016.
[23] M. Andriluka et al. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. In Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pages 3686-3693, 2014.
[24] T. K. Ho et al. Random Decision Forests. In Proceedings of 3rd international conference on document analysis and recognition, volume 1, pages 278-282. IEEE, 1995.
[25] A. Newell et al. Stacked Hourglass Networks for Human Pose Estimation. In European conference on computer vision, pages 483-499. Springer, 2016.
[26] E. Insafutdinov et al. Deepercut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model. In European Conference on Computer Vision, pages 34-50. Springer, 2016.
[27] T. M. Mitchell. Machine Learning. McGraw-Hill, 1997.
[28] K. Simonyan and A. Zisserman. Very Deep Convolutional networks for Large-Scale Image Recognition. arXiv preprint arXiv:1409.1556, 2014.
[29] D. Jungnickel. The Greedy Algorithm, pages 129-153. Springer Berlin Heidelberg, 1999.
[30] R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.
[31] Z. Zhang. Flexible Camera Calibration by Viewing a Plane from Unknown Orientations. In Proceedings of the seventh ieee international conference on computer vision, volume 1, pages 666-673. IEEE, 1999.
[32] Z. Zhang. A Flexible New Technique for Camera Calibration. IEEE Transactions on pattern analysis and machine intelligence, 22(11):1330-1334, 2000.
[33] Y. M. et al. Wang. A camera calibration technique based on OpenCV. In The 3rd International Conference on Information Sciences and Interaction Sciences, pages 403-406, 2010.
[34] Nokia Corp. Qt : Cross-platform Application and UI Framework. http://qt.nokia.com/, 2012.
[35] T. Wiedemeyer. IAI Kinect2. https://github.com/code-iai/iai kinect2, 2014-2015. Accessed June 12, 2015.
[36] 韓麗王露晨. 基於特征向量匹配的舞蹈實時姿態分析方法. 計算機應用研究, 2016.
[37] CMU. Carnegie-Mellon Motion Capture Database. Created with funding from NSF EIA-0196217(2003), http://mocap.cs.cmu.edu/.
[38] M. Muller et al. Documentation Mocap Database HDM05. Technical ReportCG-2007-2, Universitat Bonn, June 2007.