本論文討論從我們實驗室發展出的居家服務型機器人May身上實現人類跟隨功能。為了確實跟隨操作者，影像追蹤的部分由Kinect骨架與TLD (Tracking-Learning-Detection)協同組成，TLD負責重新偵測操作者被阻擋或消失於機器人的視野後再現的情況；一般無遮蔽的狀況由Kinect骨架主要追蹤，TLD學習擴大操作者影像的資料庫，以增加重新偵測的成功率。為了使追蹤系統更完善，影像追蹤加入模糊頭部運動控制，以補足May的底盤移動無法及時反應追蹤操作者。透過模糊控制，操作者瞬間快速的移動皆能被及時捕捉到。Q-learning被應用於探索底盤姿態轉換的能力使May具有更強健的跟隨功能。Q-learning的部分，狀態的設定基於三維空間中的位置、動作則是各項四輪獨立轉向四輪獨立驅動(4WIS4WID)的姿態，而動作的獎賞則是建立在狀態的轉換。最後經由實驗室的實驗結果與RoboCup 2013日本公開賽居家組的跟隨任務比賽，證明本文提出的架構能讓機器人學習如何順暢切換姿態以跟隨使用者並在此競賽項目取得優異的成績。

This thesis mainly implements human following function for home service robot, May, developed in our laboratory. In order to follow the operator accurately, visual tracking is composed by Tracking-Learning-Detection (TLD) and Kinect skeleton, where TLD plays the role as re-detecting the situation that operator is occluded or disappeared, and Kinect skeleton is adopted to track all other situations while TLD is learning how to enlarge operator image patterns in order to enhance recognition rates. For the sake of improving tracking capability, fuzzy head motion control is added in the visual tracking system to compensate the constraints that the mobile platform of May cannot react rapidly. Every instant movement of the operator can be captured by fuzzy head motion control in real time. Q-learning is applied to discover the pose switching of the mobile platform such that May possesses more robust following ability. By Q-learning, states setting are based on three dimensional position, actions are created by the pose of four wheel independent steering and four wheel independent driven (4WIS4WID) platform, and rewards are established on state transitions. Finally, both the experimental results in the laboratory and competition consequents of Follow Me Mission in robot@home league at RoboCup Japan Open 2013 Tokyo demonstrate that our robot May can fluently switch its poses to follow operator by utilizing the proposed schemes.

[1] A. G. Billard, S. Calinon, and F. Guenter, "Discriminative and Adaptive Imitation in Uni-Manual and Bi-Manual Tasks," Robotics and Autonomous Systems, vol. 54, no. 5, pp. 370-384, 2006.
[2] M. Cakmak, N. DePalma, R. I. Arriaga, and A. L. Thomaz, "Exploiting Social Partners in Robot Learning," Autonomous Robots, vol. 29, no. 3-4, pp. 309-329, 2010.
[3] A. G. Billard, S. Calinon, and F. Guenter, "On Learning, Representing, and Generalizing a Task in a Humanoid Robot," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 2, pp. 286-298, 2007.
[4] H. Y. Chan, K. Y. Young, and H. C. Fu, "Intention Learning from Human Demonstration," Journal of Information Science And Engineering, vol. 27, no. 3, pp. 1123-1136, 2011.
[5] Y. Wang, H. Lang, and C. W. De Silva, "A Hybrid Visual Servo Controller for Robust Grasping by Wheeled Mobile Robots," IEEE/ASME Transactions on Mechatronics, vol. 15, no. 5, pp. 757-769, 2010.
[6] H. Xu, W. Huang, F. Peng, K. Xue, S. Yu, X. Gao, Q. Ouyang, Q. Chang, and Z. Lu, "Maneuver Control and Kinematical Energy Analysis of a Robot Based on Instantaneous Center of Rotation," in Proc. IEEE International Conference on E-Learning in Industrial Electronics, pp. 101-106, 2006.
[7] H. Xu, W. Huang, F. Peng, K. Xue, and S. Yu, "Maneuver Control of Mobile Robot Based on Equivalent Instantaneous Center of Rotation in Rough Terrain," in Proc. International Conference on Mechatronics and Automation, pp. 405-410, 2007.
[8] M.A. Sharbafi, C. Lucas, and R. Daneshvar, "Motion Control of Omni-Directional Three-Wheel Robots by Brain-Emotional-Learning-Based Intelligent Controller," IEEE Trans. Syst., Man, Cybern., Part C, vol. 40, no. 6, pp. 630-638, 2010.
[9] Z. Kalal, J. Matas, and K. Mikolajczyk, "Tracking-learning-detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1409-1422, 2012.
[10] Z. Kalal, J. Matas, and K. Mikolajczyk, "Online Learning of Robust Object Detectors During Unstable Tracking," in Proc. International Conference on Computer Vision Workshops (ICCV Workshops), 2009.
[11] B.D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. International Joint Conferences on Artificial Intelligence, vol. 81, pp. 674–679, 1981.
[12] V. Lepetit, P. Lagger, and P. Fua, "Randomized Trees for Real Time Keypoint Recognition," in Proc. Conf. Computer Vision and Pattern Recognition, 2005.
[13] M. Andriluka, S. Roth, and B. Schiele, "People-Tracking-By Detection and People-Detection-By-Tracking," in Proc. Conf. Computer Vision and Pattern Recognition, 2008.
[14] D. Ramanan, D. Forsyth, and A. Zisserman, "Strike a Pose: Tracking People by Finding Stylized Poses," in Proc. Conf. Computer Vision and Pattern Recognition, 2005.
[15] S. Avidan, "Ensemble Tracking," Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 261–271, 2007.
[16] H. Grabner and H. Bischof, "On-Line Boosting and Vision," in Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 260-267, 2006.
[17] Z. Kalal, J. Matas, and K. Mikolajczyk, "Forward-Backward Error: Automatic Detection of Tracking Failures," in Proc. International Conference on Pattern Recognition, pp. 2756-2759, 2010.
[18] J. P. Lewis, "Fast Normalized Cross-Correlation," Vision Interface, vol. 10, no. 1, pp. 120-123, 1995.
[19] J. L. Rodgers and W. A. Nicewander, "Thirteen Ways to Look at the Correlation Coefficient," The American Statistician, vol. 42, no. 1, pp. 59–66, 1988.
[20] J. Shi and C. Tomasi, "Good Features To Track," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600, 1994.
[21] T. Tommasini, A. Fusiello, E. Trucco, and V. Roberto, "Making Good Features Track Better," in Proc. 1998 IEEE Conf. Computer Vision and Pattern Recognition, pp. 178 -183, 1998.
[22] G. Bradski and A. Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library. Cambridge, MA: O'Reilly, 2008.
[23] J. Y. Bouguet, "Pyramidal Implementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm," Technical Report, OpenCV Document, Intel Microprocessor Research Labs, 2001.
[24] Z. Kalal, J. Matas, and K. Mikolajczyk, "P-N Learning: Bootstrapping Binary Classifiers by Structural Constraints," in Proc. Conf. Computer Vision and Pattern Recognition, pp. 49-56, 2010.
[25] Y. Freund and R. E. Schapire, "A Desicion-Theoretic Generalization of On-Line Learning and an Application to Boosting," Computational Learning Theory. , pp. 23-37, 1995.
[26] P. Viola and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features," in Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. I-511,I-518, 2001.
[27] H. Grabner, M. Grabner, and H. Bischof, "Real-Time Tracking via On-Line Boosting," in Proc. British Machine Vision Conference, pp. 47-56, 2006.
[28] H. Grabner, C. Leistner, and H. Bischof, "Semi-Supervised On-Line Boosting for Robust Tracking," in Proc. European Conference on Computer Vision, pp. 234-247, 2008.
[29] S. Stalder, H. Grabner, and L. Van Gool, "Beyond Semi-Supervised Tracking: Tracking Should Be as Simple as Detection, but Not Simpler Than Recognition," in Proc. Workshop Online Learning in Computer Vision, pp. 1409-1416, 2009.
[30] V. Lepetit and P. Fua, "Keypoint Recognition using Randomized Trees," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1465-1479, 2006.
[31] L. Breiman, "Random Forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001.
[32] A. Bosch, A. Zisserman, and X. Muoz, "Image Classification using Random Forests and Ferns," in Proc. International Conference on Computer Vision, pp. 1-8, 2007.
[33] M. Ozuysal, M. Calonder, V. Lepetit, and P. Fua, "Fast Keypoint Recognition using Random Ferns," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 3, pp. 448-461, 2010.
[34] M. Ozuysal, P. Fua, and V. Lepetit, "Fast Keypoint Recognition in Ten Lines of Code," in Proc. Conf. Computer Vision and Pattern Recognition, 2007.
[35] T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, 2002.
[36] V. Utkin, "Variable Structure Systems with Sliding Modes," IEEE Transactions on Automatic Control, vol. 22, no. 2, pp. 212-222, 1977.
[37] S. V. Drakunov and V. I. Utkin, "Sliding Mode Control in Dynamic Systems," International Journal of Control, vol. 55, no. 4, pp. 1029-1037, 1992.
[38] L. A. Zadeh, "Fuzzy Sets," Information and Control, vol. 8, no. 3, pp. 338-353, 1965.
[39] E. H. Mamdani, "Application of Fuzzy Algorithms for Control of Simple Dynamic Plant," Proc. Institution of Electrical Engineers, vol. 121, no. 12, pp. 1585-1588, 1974.
[40] C. C. Lee, "Fuzzy Logic in Control Systems: Fuzzy Logic Controller-Part I," IEEE Tran. on Syst. Man and Cyber., vol. 20, no. 2, pp. 404-418, 1990.
[41] T. Takagi and M. Sugeno, "Fuzzy Identification of Systems and Its Applications to Modeling and Control," IEEE Tran. on Syst. Man and Cybern., vol. 1, pp. 116-132, 1985.
[42] C. C. Lee, "Fuzzy Logic in Control Systems: Fuzzy Logic Controller-Part II," IEEE Tran. on Syst. Man and Cybern., vol. 20, no. 2, pp. 419-435, 1990.
[43] C.P. Connette, A. Pott, M. Hägele, and A. Verl, "Control of an Pseudo-Omnidirectional, Non-Holonomic, Mobile Robot Based on an ICM Representation in Spherical Coordinates," in Proc. IEEE Conference on Decision and Control, pp. 4976-4983, 2008.
[44] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
[45] E. Alpaydin, Introduction to Machine Learning. Cambridge, MA: MIT Press, 2004.
[46] L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement Learning: A Survey," J. Artif. Intell. Res., vol. 4, pp. 237 -285, 1996.
[47] C. J. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, no. 3-4, pp. 279-292, 1992.
[48] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 2nd ed. Upper Saddle River, NJ: Pearson Education, 2003.
[49] http://www.microsoft.com/en-us/kinectforwindows/.