手語辨識系統可以幫助一般人跟聽障人士溝通，它也可以被使用來當作學習手語的主要工具，鑑於台灣手語是台灣最普及的手語，我們發展的系統乃以台灣手語之語料為採集與測試的對象。在這篇研究論文中，我們發展一個視覺型的次系統，針對形變式手語進行追蹤與辨識。形變式手語乃定義為需要從一個手型變化到另一個手型才能完整表達它的意義的手語字。此形變式手語辨識次系統由四個部分組成: 皮膚顏色的偵測、手型的追蹤、手型輪廓的表示、和形變式手勢的辨識。
物體輪廓的追蹤常利用「主動輪廓模型」方法。然而由於手型的手指屬於長又瘦的物體，造成輪廓的縱深「凹口」，主動輪廓模型往往無法在追蹤的過程中，追蹤出完整的手型。為了解決這個問題，我們結合了梯度向量流場和外加膨脹壓力來當外部力量，提出以「增強式主動輪廓模型」方法改善手型輪廓的追蹤。接下來，我們將輪廓點利用仿射不變性的傅立葉描述子來描述手型輪廓，借用它對物體輪廓之旋轉、放大和縮小不變的優點。在形變式手勢辨識的過程中，我們也加入了一個形變轉移機率去改善我們的辨識結果。
在台灣手語的手型被定義總共有49種，但是並非每一個手型都是常用的。我們選擇了20組常用的形變式手語字進行實驗，其中包含有21種基本的台灣手語手型。實驗顯示，「增強式主動輪廓模型」成功追蹤出形變式手語的完整手型，新發展的次系統可以合理地辨識台灣手語中的形變式手語字。

A recognition system for sign language can help people communicate with people with hearing impairments. The recognition system can also be used as an e-learning kit to learn the sign language. Taiwanese sign language (TSL) is one of the main languages for people with hearing impairments in Taiwan. In this study, we developed a vision-based approach to tracking and recognizing shape-changing hand gestures in TSL. A shape-changing hand gesture is defined herein as a gesture that changes from one shape to another for conveying a sign word. The shape-changing hand gesture recognition subsystem consists of four phases: skin color detection, hand shape tracking, hand shape representation, and gesture recognition.
Contours of objects are commonly tracked using the active contour model (ACM). However, the active contour model often fails to track the objects with long knobs, such as hands with fingers because of the difficulty progressing into boundary concavities. To solve the problem, the developed system enhanced the ACM by combining the gradient vector flow and the inflation pressure force as the external forces. Subsequently, the tracked hand shape contours were represented by the affine-invariant Fourier descriptors, which are invariant to translation, rotation and scaling. In the phase of gesture recognition, we included transition probabilities to characterize the change in hand shape.
In the experiment, we chose 20 shape-changing TSL hand gestures, which involved 21 fundamental hand shapes. Experiments showed that the enhanced active contour model was reliable in tracking changes in hand shapes and that the developed system was feasible for recognizing the shape-changing hand gestures of TSL.

[1] V. I. Pavlovic, R.Sharma, and T. S. Huang, “Visual interpretation of hand gestures for human-computer interaction: a review,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 677-695, July 1997.
[2] R. H. Liang and M. Ouhyoung, “A real-time continuous gesture recognition system for sign language,” Proc. of IEEE Int’l Conf. Automatic Face and Gesture Recognition, pp. 558-567, Apr. 1998.
[3] F. S. Chen, C. M. Fu, and C. L. Huang, “Hand gesture recognition using a real-time tracking method and hidden Markov models,” Journal of Image and Vision Computing, vol. 1, pp. 745-758, Mar. 2003.
[4] A. Ramamoorthy, N. Vaswani, S. Chaudhury, and S. Banerjee, “Recognition of dynamic hand gestures,” Pattern Recognition, vol. 36, pp. 2069-2081, 2003.
[5] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour model,” Int’l J. Computer Vision, vol. 1, pp. 321-331, 1988.
[6] C. Xu and J. L. Prince, “Snakes, shapes and gradient vector flow,” IEEE Trans. Image Processing, vol. 7, no. 3, pp. 359-369, Mar. 1998.
[7] C. Xu and J. L. Prince, “Generalized gradient vector flow external forces for active contours,” Signal Processing, vol. 71, pp. 131-139, 1998.
[8] L. D. Cohen, “On active contour models and balloons,” CVGIP: Image Understanding, vol. 53, no. 2, pp. 221-228, Mar. 1991.
[9] L. D. Cohen, and I. Cohen, “Finite-element methods for active contour models and balloons for 2D and 3D images,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 15, no. 11, pp. 1131-1147, 1993.
[10] K. Arbter, W. E. Snyder, H. Burkhardt and G. Hirzinger, “Application of affine-invariant Fourier descriptors to recognition of 3-D objects,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 12, no. 7, July 1990.
[11] 史文漢、丁立芬，手能生橋，第一冊~第二冊，中華民國聾人協會發行，2004.
[12] C. H. Wu, Y. H. Chiu, and C. S. Guo, “Text generation from Taiwanese sign language using a PST-based language model for augmentative communication,” IEEE Trans. Neural Systems and Rehabilitation Engineering, vol. 12, no. 4, Dec. 2004.
[13] S. Ahmad, “A usable real-time 3D hand tracker,” IEEE Asilomar Conf. Signals, Systems, and Computers, pp. 1257-1261, 1994.
[14] D. A. Becker and A. Pentland, “Using a virtual environment to teach cancer patients T’ai Chi, relaxation, and self-imagery,” Proc. of IEEE Int’l Conf. Automatic Face and Gesture Recognition, Oct. 1996.
[15] U. Brockl-Fox, “Real-time 3D interaction with up to 16 degrees of freedom from monocular image flows,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 172-178, June 1995.
[16] J. L. Crowley, F. Berard, and J. Coutaz, “Finger tracking as an input device for augmented reality,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 195-200, June 1995.
[17] T. Darrell and A. P. Pentland, “Attention-driven expression and gesture analysis in an interactive environment,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 135-140, June 1995.
[18] W. T. Freeman, K. Tanaka, J. Ohta, and K. Kyuma, “Computer vision for computer games,” Proc. Int’l Conf. Automatic Face and Gesture Recognition, pp. 100-105, Oct. 1996.
[19] W. T. Freeman and C. D. Weissman, “Television control by hand gestures,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 179-183, June 1995.
[20] M. Fukumoto, Y. Suenaga, and K. Mase, “Finger-pointer: pointing interface by image processing,” Computers and Graphics, vol. 18, no. 5, pp. 633-642, 1994.
[21] R. Kjeldsen and J. Kender, “Visual hand gesture recognition for window system control,” Proc . Int’l Workshop Automatic Face-and-Gesture-Recognition, pp. 184-188, June 1995.
[22] C. Maggioni, “Gesture computer-new ways of operating a computer,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 166-171, June 1995.
[23] F. K. H. Quek, T. Mysliwiec, and M. Zhao, “Finger mouse: a freehand pointing interface,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 372-377, June 1995.
[24] J. M. Rehg and T. Kanade, “DigitEyes: vision-based human hand tracking,” Technical Report CMU-CS-93-220, School of Computer Science, Carnegie Mellon Univ., 1993.
[25] R. Cipolla and N. J. Hollinghurst, “Human-robot interface by pointing with uncalibrated stereo vision,” Image and vision Computing, vol. 14, pp. 171-178, Mar. 1996.
[26] E. Hunter, J. Schlenzig, and R. Jain, “Posture estimation in reduced-model gesture input systems,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, June 1995.
[27] A. Torige and T. Kono, “Human-interface by recognition of human gestures with image processing recognition of gesture to specify moving directions,” IEEE Int’l Workshop on Robot and Human Communication, pp. 105-110, 1992.
[28] Y. Cui and J. J. Weng, “Hand segmentation using learning-based prediction and verification for hand sign recognition,” Proc. Int’l Conf. Automatic Face and Gesture Recognition, pp. 88-93, Oct. 1996.
[29] T. E. Starner and A. Pentland, “Visual recognition of American sign language using hidden Markov models,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 189-194, June 1995.
[30] T. S. Huang and V. I. Pavlovic, “Hand gesture modeling, analysis, and synthesis,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, pp. 73-79, June 1995.
[31] C. C. Lien and C. L. Huang, “Model-based articulated hand motion tracking for gesture recognition,” Journal of Image and Vision Computing., vol. 16, no. 2, pp. 121-134, 1998
[32] A. A. Amini, S. Tehrani, and T. E. Weymouth, “Using dynamic programming for minimizing the energy of active contours in the presence of hard constraints,” Proc. of IEEE Int’l Conf. Computer Vision, pp. 95-99, 1998.
[33] A. A. Amini, T. E. Weymouth, and R. C. Jain, “Using dynamic programming for solving variational problems in vision,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 12, no. 9, pp. 855-867, 1990.
[34] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” Int’l J. Computer Vision, vol. 22, no. 1, pp. 61-79, 1997.
[35] A. F. Bobick and A. D. Wilson, “A state-based approach to the representation and recognition of gesture,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 19, no. 12, pp. 1325-1337, Dec. 1997.
[36] K. Vaananen and K. Boehm, “Gesture driven interaction as a human factor in virtual environments-an approach with Neural Networks,” Virtual Reality systems, pp. 93-106, 1993.
[37] H. K. Lee and J. H. Kim, “An HMM-based threshold model approach for gesture recognition,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 21, no. 10, pp. 961-973, Oct. 1999.
[38] P. V. O’Neil, Advanced engineering mathematics, 3rd ed., Wadsworth publishing, pp. 510~535, 1991.
[39] J. D. Foley et al., Computer Graphics, Addition Wesley, 1992.
[40] D.P. Huttenlocher, G. A. Klanderman, and W. J. Rucklidge, “Comparing images using the Hausdorff distance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 850-863, Sept. 1993.