本論文中，我們提出一套創新的運動裁判訓練系統，此系統基於穿戴式傳感器與一套可即時識別官方裁判手勢的方法，在ORS-1資料集的65種籃球裁判手勢中達到96.6%的準確率。為了透過慣性傳感器訊號識別裁判手勢，我們設計了一套混合式神經網路-ORSNet，ORSNet結合了卷積層與遞歸層，卷積層幫助模型學習更多代表性的局部特徵，而遞歸層則學習訊號在時域上的關聯性。我們提出一個新穎的損失函數與權重共享策略，使得裁判手勢辨識模型更加強健可靠。此外，論文中也探討了半監督式學習對於ORSNet的影響。最後，我們利用ORSNet建構了一個即時識別系統，並能成功的在連續動作中辨識出籃球裁判手勢。

In this work, we propose a novel sports referee training system based on wearable sensors and a real-time Official Referee Signal (ORS) segmentation/recognition method which can recognize 65 kinds basketball ORSs with the accuracy of 96.6% in ORS-1 dataset. A hybrid neural network named ORSNet is designed for recognizing gestures based on IMU signals. The proposed ORSNet involves convolution layers and recurrent layers to learn more representative features and correlations in temporal domain, respectively. A novel loss function and a weight sharing strategy are proposed to learn a more robust ORS recognition model. Moreover, we investigate the influence of applying a semi-supervised network in the proposed ORSNet. Finally, we build a real-time ORS recognition system based on ORSNet, and it can recognize basketball ORSs in continuous motion successfully.

[1] Leap motion. https://www.leapmotion.com/.
[2] Mhealth dataset. http://archive.ics.uci.edu/ml/datasets/mhealth+dataset.
[3] Mircosoft kinect. https://developer.microsoft.com/zh-tw/windows/kinect.
[4] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh. Realtime multi-person 2d pose
estimation using part affinity fields. In CVPR, volume 1, page 7, 2017.
[5] J. C. Chan, H. Leung, J. K. Tang, and T. Komura. A virtual reality dance training
system using motion capture technology. IEEE Transactions on Learning
Technologies, 4(2):187–195, 2011.
[6] S. Duffner, S. Berlemont, G. Lefebvre, and C. Garcia. 3d gesture classification
with convolutional neural networks. In Acoustics, Speech and Signal
Processing (ICASSP), 2014 IEEE International Conference on, pages 5432–
5436. IEEE, 2014.
[7] C. et al. Gesture recognition-based wireless intelligent judgment system.
https://patents.google.com/patent/CN101667059A/en, 2008.
[8] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber.
Lstm: A search space odyssey. IEEE Transactions on Neural Networks and
Learning Systems, 28(10):2222–2232, 2017.
[9] S. Ha and S. Choi. Convolutional neural networks for human activity recognition
using multiple accelerometer and gyroscope sensors. In Neural Networks
(IJCNN), 2016 International Joint Conference on, pages 381–388. IEEE,
2016.
[10] S. Ha, J.-M. Yun, and S. Choi. Multi-modal convolutional neural networks
for activity recognition. In Systems, Man, and Cybernetics (SMC), 2015 IEEE
International Conference on, pages 3017–3022. IEEE, 2015.
[11] J. Han, L. Shao, D. Xu, and J. Shotton. Enhanced computer vision with microsoft
kinect sensor: A review. IEEE transactions on cybernetics, 43(5):
1318–1334, 2013.
[12] M.-C. Hu, C.-W. Chen, W.-H. Cheng, C.-H. Chang, J.-H. Lai, and J.-L. Wu.
Real-time human movement retrieval and assessment with kinect sensor. IEEE
transactions on cybernetics, 45(4):742–753, 2015.
[13] Y. Li, X. Chen, X. Zhang, K. Wang, and Z. J. Wang. A sign-componentbased
framework for chinese sign language recognition using accelerometer
and semg data. IEEE Transactions on Biomedical Engineering, 59(10):2695–
2704, 2012.
[14] Y. Li, D. Shi, B. Ding, and D. Liu. Unsupervised feature learning for human
activity recognition using smartphone sensors. In Mining Intelligence and
Knowledge Exploration, pages 99–107. Springer, 2014.
[15] C. Lo, Q. Cao, X. Zhu, and Z. Zhang. Gesture recognition system based on acceleration
data for robocup referees. In Natural Computation, 2009. ICNC’09.
Fifth International Conference on, volume 2, pages 149–153. IEEE, 2009.
[16] Y. Miao, L. Wang, C. Xie, and B. Zhang. Gesture recognition based on deep
belief networks. In Chinese Conference on Biometric Recognition, pages 439–
446. Springer, 2017.
[17] P. Muneesawang, N. M. Khan, M. Kyan, R. B. Elder, N. Dong, G. Sun, H. Li,
L. Zhong, and L. Guan. A machine intelligence approach to virtual ballet
training. IEEE MultiMedia, 22(4):80–92, 2015.
[18] F. J. Ordóñez and D. Roggen. Deep convolutional and lstm recurrent neural
networks for multimodal wearable activity recognition. Sensors, 16(1):115,
2016.
[19] T.-Y. Pan, L.-Y. Lo, C.-W. Yeh, J.-W. Li, H.-T. Liu, and M.-C. Hu. Real-time
sign language recognition in complex background scene based on a hierarchical
clustering classification method. In Proceedings of the IEEE International
Conference on Multimedia Big Data, pages 64–67. IEEE, 2016.
[20] A. Rasmus, M. Berglund, M. Honkala, H. Valpola, and T. Raiko. Semisupervised
learning with ladder networks. In Advances in Neural Information
Processing Systems, pages 3546–3554, 2015.
[21] S. Shin and W. Sung. Dynamic hand gesture recognition for wearable devices
with low complexity recurrent neural networks. In Circuits and Systems
(ISCAS), 2016 IEEE International Symposium on, pages 2274–2277. IEEE,
2016.
[22] P. Trigueiros, F. Ribeiro, and L. P. Reis. Vision based referee sign language
recognition system for the robocup msl league. In Robot Soccer World Cup,
pages 360–372. Springer, 2013.
[23] P. Trigueiros, F. Ribeiro, and L. P. Reis. Hand gesture recognition system
based in computer vision and machine learning. In Developments in Medical
Image Processing and Computational Vision, pages 355–377. Springer, 2015.
[24] J. Vales-Alonso, D. Chaves-Diéguez, P. López-Matencio, J. J. Alcaraz, F. J.
Parrado-García, and F. J. González-Castaño. Saeta: A smart coaching assistant
for professional volleyball training. IEEE Transactions on Systems, Man, and
Cybernetics: Systems, 45(8):1138–1150, 2015.
[25] J. Wang, Y. Chen, S. Hao, X. Peng, and L. Hu. Deep learning for sensor-based
activity recognition: A survey. arXiv preprint arXiv:1707.03502, 2017.
[26] S. B. Wang, A. Quattoni, L.-P. Morency, D. Demirdjian, and T. Darrell. Hidden
conditional random fields for gesture recognition. In Computer Vision and
Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 2,
pages 1521–1527. IEEE, 2006.
[27] Z. Wang, M. Guo, and C. Zhao. Badminton stroke recognition based on body
sensor networks. IEEE Transactions on Human-Machine Systems, 46(5):769–
775, 2016.
[28] J. Wu, L. Sun, and R. Jafari. A wearable system for recognizing american
sign language in real-time using imu and surface emg sensors. IEEE Journal
of Biomedical and Health Informatics, 20(5):1281–1290, 2016.
[29] R. Xie and J. Cao. Accelerometer-based hand gesture recognition by neural
network and similarity matching. IEEE Sensors Journal, 16(11):4537–4545.
[30] S. Yao, S. Hu, Y. Zhao, A. Zhang, and T. Abdelzaher. Deepsense: A unified
deep learning framework for time-series mobile sensing data processing. In
Proceedings of the 26th International Conference on World Wide Web, pages
351–360. International World Wide Web Conferences Steering Committee,
2017.
[31] X. Zhang, X. Chen, Y. Li, V. Lantz, K. Wang, and J. Yang. A framework
for hand gesture recognition based on accelerometer and emg sensors. IEEE
Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans,
41(6):1064–1076, 2011.
[32] Y. Zheng, Q. Liu, E. Chen, Y. Ge, and J. L. Zhao. Time series classification
using multi-channels deep convolutional neural networks. In International
Conference on Web-Age Information Management, pages 298–310. Springer,
2014.