手語與中文存在著語言結構的差異，聽人若以中文型態的表達，期與聾人進行溝通，則無法廣泛或確切地為聾人所接受或理解。現階段啟聰教育之教學法，主要採手語為主，然而，現有靜態圖文教材，難以具體呈現手語複雜的肢體動作特徵；真人影像教材又僅侷限於小量的詞彙教學。倘若能於學習或互動層面上，以手語型態及真人影像呈現，對於聾人而言，將會是最親切、自然的模式。因此，本研究之目的為實現以中文轉譯成手語序列，再以真人影像的自然比劃動作來達成聽人對聾人的溝通目標

本研究之架構，主要包含：1).中文/手語對照語料剖析：根據對照語料及手語語言分析，提出一組語法類別單元集及二階段轉譯處理策略來描述對照句；2).轉譯機制建立：提出一統計式轉譯模型，同時估算句型主結構轉譯機率及片語對譯機率，另外，經由轉譯樣版及對應樣版的組合與衍生，可解決句型集外的問題；3).手語動作轉移平衡影像庫建立：設計涵蓋手語詞彙間轉移資訊之平衡語句資料庫，進而，建置含轉移資訊之手語真人影像資料庫；4.)手語影帶串接生成機制建立：提出具同時估算手部位置及移動方向差異的串接生成演算法，以呈現連續且平順之手語動作；5).系統整合與個案實測評估。

實驗中，選取2159個手語詞彙及4554對照語句(平均句長為4.5個詞)為訓練與測試語料，依此訓練而得1518個句型主結構轉譯樣版，867個片語對譯樣版。在轉譯效能評估部分，隨機選取90 %對照句為訓練語料，其餘測試語料，Top-1 之正確率為82.4 %；Top-5之正確率為92.3%。轉移平衡語料共計選出409句，其中，涵蓋轉移次數非為1之詞彙的平均轉移數為2.57次。在串接方法比較部分，選取5名聾生進行個案實測，分三等級評量影像接流暢度，以本文所提串接方法之總體表現為最佳。在系統整合測試部分，由教師選取課文試題經系統自動轉譯並產生相對應之手語影像，再由教師分別詢問5位聾生之反應，分三等級評量句意理解度，整體之句意理解度可達82 %。本文提出一套中文轉譯手語及手語影像生成之手語教學輔助系統，透過相關的實驗探討，顯現本研發系統之實用性與前瞻性。

The hearing-impaired people generally use sign language to express their intention. Sign language can be learned from teachers, books or videotapes. Presently, the use of either photographs or drawings overlaid with textual descriptions to represent sign language makes the reader harder to understand the process of signing. Videotapes better represent the signing process, but they have difficulty for flexible access and are limited in a small coverage of sign words. Besides, there exist many language differences between Chinese and Taiwanese sign language (TSL). The able-bodied people present sign language by way of the use of Chinese grammar. This situation frequently occurs and makes it hard to communicate with the hearing impaired. In this thesis, we propose a Chinese to TSL translation and sign image synthesis system for TSL learning.
More specially, we focuses on 1) proposing a set of syntactic clusters and a two-stage processing strategy to parse the Chinese and TSL parallel sentence into the sentence translation pattern and the phrase alignment pattern, 2) developing a novel statistical translation model to estimate the optimal TSL sentence and deal with the out-of-pattern issue, 3) developing a sign image database based on the balanced characteristics of sign motion features, 4) proposing an image synthesis mechanism to concatenate video clips by simultaneously considering the distance of palm positions and movement directions, and 5) integrating the above approaches into an image-based sign language learning system.
In order to evaluate our proposed approaches, 2159 sign words and 4554 parallel sentences, in which the mean length of sentence is 4.5 words, were collected. Of this database, 90% was used as the training corpus and the remainder for testing. 1518 sentence patterns and 867 phrase patterns were obtained. The trained balanced corpus consists of 409 sentences and 1776 sign words. The translation performance achieved 92.3% accuracy for top-5 candidates. In subjective and objective evaluation of image synthesis, our proposed approach achieved better performance. In case study, the reading comprehension performance of our proposed system also achieved 82% satisfactory degree. Consequently, our proposed system can provide the deaf and the able-bodied to learn TSL and be used as an alternative method for the exam in deaf schools.

[1]. 行政院主計處，國情統計通報。民88年。
[2]. Richard Dye, Norman Alm, John L. Arnott, Gillian Harper, “A script-based AAC system for transactional interaction,” Cambridge University Press on Natural Language Engineering, vol.4 , no. 1, pp. 57-71, 1998.
[3]. Alan Newell, Stefan Langer, Marianne Hickey, “The role of natural language processing in alternative and augmentative communication,” Cambridge University Press on Natural Language Engineering, vol.4 , no. 1, pp. 1-16, 1998.
[4]. 陳勝良，語言溝通障礙者數位溝通輔具系統之研發，國立成功大學醫學工程研究所碩士論文，民90年。
[5]. 林寶貴，語言障礙者就學、就業之科技支援，台灣師範大學特殊教育學系，1997.
[6]. Mu-Chun Su, Yu-Xiang Zhao, Hai Huang, and Hsuan-Fan Chen, “A fuzzy rule-based approach to recognizing 3-D Arm Movements,” IEEE Trans. On Neural System and Rehab. Eng., vol. 9, no. 2, 2001.
[7]. Clayton Valli and Cell Lucas, Linguistics of American Sign Language: An Introduction. Gallaudet University Press, 1995.
[8]. C. Vogler and D. Metzxas, “Adapting Hidden Markov Models for ASL recognition by using three-dimensional computer vision methods,” in Proc. Of the IEEE international Conf. on System, Man and Cybernetics, pp. 156-161, 1997.
[9]. Chih-Ming Fu, Chung-Lin Huang and Jyh-Woei Tsi, “A Real-Time Hand Gesture Tracking For Human Computer Interface,” 15th IPPR Conf. on Computer Vision, Graphics and Image Processing, 2002.
[10]. Wayne H. Smith, Morphological Characteristics of Verbs in Taiwan Sign Language. Ph.D. dissertion, Indiana University, 1989.
[11]. 郭啟祥，應用自然語言處理於台灣手語轉語音系統之研究，民國88年。
[12]. 鄭功偉，具容錯之手語檢索機制於手語至中文翻譯之應用，民國91年。
[13]. Wolfgang Wahlster, “Vermobil: Foundations of Speech-to-Speech Translation,” Springer-Verlag Press, 2000.
[14]. 游大緯，統計式機器對應與翻譯模型，民國91年。
[15]. 趙玉平，手語大師系列，現代經典出版。
[16]. 教育部特殊教育工作小組，2001年手語教學與應用研討會論文集，民90年。
[17]. 余秀敏，劉繼謚，國語語音特性平衡句之建立，電信研究季刊，第19卷第1期，民國78年3月。
[18]. 中文詞類分析(三版)，技術報告 93-05，中央研究院資訊科學研究所詞庫小組。
[19]. Heitou ZEN, Tameharu HASEGAWA, Shinji OZAWA, “Moving Object Detection From Mpeg Coded Picture,” Proceeding of IEEE International Conference on Image Processing, 28AS1, vol. IV, pp.25-29, Oct. 1999
[20]. Franc Solina, Slavko K. “Multimedia Dictionary and Synthesis of Sign Language,” Idea Group Publishing , Hershey PA, p268-281, 2001.
[21]. Slavko K., Franc Solina, “Synthesis of the Sign Language of the Deaf from the Sign Video Clips,” Electrotechnical Conf, Ljubljana, Slovenija, 1999.
[22]. Richard Kennaway, “Synthetic Animation of Deaf Signing Guestures,” Lecture notes in Computer Science, 4th International Workshop on Gesture and Sign Language Based Human-Computer Interaction, London (2001). Lecture Notes in Artificial Intelligence vol. 2298.
[23]. James N.K.Liu and Lina Zhou,” A Hybrid Model for Chinese-English Machine Translation,” Systems, Man, and Cybernetics, IEEE Interantional Conference on , Volume: 2, 11-14 Oct 1998.
[24]. Li Yujian and Zhong Yixin, “Machine Translation Based on Templates Matching and Replacing,” Systems, Man, and Cybernetics, IEEE Interantional Conference on , Volume: 2, 7-10 Oct 2001.
[25]. Franz Josef Och ,”An Efficient Method for Determining Bilingual Word Classes,” In Proc. of the Ninth Conf. of the Europ. Chapter of the Association of Computational Linguistics, Bergen, Norway, pp. 71-76, June 1999.
[26]. Peter F.Brown, Stephen A. Dela Pietra, Robert L. Mercer, “The Mathematics of Statistical Machine Translation: Parameter Estimation,” Association for Computational Linguistics 1993.
[27]. D’Armond L. Speers, “Representation of American Sign Language for Machine Translation,” Ph.D. Dissertation, Georgetown Univ., 2001.
[28]. William B. Dolan, “Word Sense Ambiguation: Clustering Related Sense,” Microsoft Research. In Proceedings of COLING94, 712-716
[29]. Christoph Tillmann and Hermann Ney, “Word Re-ordering and DP-based Search in Statistical Machine Translation,” Proceedings of COLING 2000 , The 18th International Conference on Computational Linguistics, 2000.
[30]. Patrick Pantel and Dekang Lin, “Discovering Word Senses from Text,” In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining- 2002.
[31]. Dekang Lin, “Automatic Retrieval and Clustering of Similar Words,” Proceedings of COLING/ACL-98. pp. 768 774. Montreal, Canada
[32]. Xin Li, “Data Clustering and the Application in Discovering Word Senses,” Cognitive Computation Group, 2002.
[33]. Dawn MacLaughlin, Carol Meidle, “SignStream User’s Guide ,”Version 1.5, Report No.8. October 1999.
[34]. ASL 3D Animation, http://www.vcom3d.com/SignSmithDemo.htm
[35]. http://www.handspeak.com/