臉部表情是一種直接而明白的情感表達方式，而且在情感計算研究上，臉部表情辨識也是一個非常重要的領域。在躁鬱症的診斷中，病患被誤診為憂鬱症的機率相當的高，不論是確診或者是修正誤診，往往需要對病患的狀態進行長時間的追蹤。本篇論文的研究，是收集病患們觀看情緒性的刺激影片時所呈現出來的臉部表情反應，藉由分析短期性臉部表情資料的特徵來進行躁鬱症與憂鬱症之偵測。
本論文研究如何分析這兩類病患在受到情緒性的刺激時表現出來的臉部表情之差異。首先利用影片給與受測者情緒上的刺激，並收集其臉部表情的影像片段。從臉部表情的影像中萃取出臉部特徵參數，並利用臉部特徵參數來估測出與其相對應的臉部動作單元剖面。藉由對一段影像片段在臉部動作單元剖面隨著時間之起伏變化估算其調變頻譜，以得到臉部動作單元剖面在頻域上的特徵，再利用雙層支持向量機(two-layer SVM)建立受測者的影像片段模型，訓練受測者的影像片段所呈現出來的臉部動作單元剖面之特徵。
為了驗證本論文提出的方法，我們透過K-Fold交叉驗證的方式，對收集自台灣台南市奇美醫院的24位受測者，包含8位躁鬱症患者、8位憂鬱症患者以及8位無相關症狀者，進行症狀偵測並取得了68.3%的準確率。另外，同時也比較了使用不同的分類器如高斯模型(Gaussian Model)以及深度神經網路(DNN)。此外亦與未經過調變頻譜的臉部動作單元剖面資訊做比較，所得到的實驗結果，證實本論文提出的方法可以得到較好的效能。

Facial expression is a direct and natural way for affective expression. In the field of affective computing, recognition of facial expression is an important and popular research topic. In mood disorder diagnosis, a high percentage of bipolar disorder (BD) patients are initially misdiagnosed as having unipolar depression (UD). This misdiagnosis carries significant negative consequences for the treatment of the BD patients. Therefore, it is crucial to establish an accurate distinction between BD and UD in order to make an accurate and early diagnosis, leading to improvements in treatment and course of illness. The research goal of this thesis is to collect the facial expression of the patients with mood disorder when they were watching video clips for emotion elicitation. The features extracted from the elicited facial expressions are used for the detection of BD and UD.
This thesis focuses on detecting the difference in facial expressions among the BD patients, UD patients and healthy people responding to emotional stimuli. In this study, first, the subjects are elicited by emotional video clips, and the videos with facial expression of the subjects are recorded. The corresponding facial action unit (AU) profiles are obtained using the support vector machines (SVMs) as the facial features. The Modulation Spectrum (MS) characterizing the fluctuation of AU profile sequence over a video segment are further extracted. Finally, a two-layer SVM is constructed for mood disorder detection based on the extracted MSs of the elicited facial expressions in the video segments of the subject.
In order to evaluate the proposed method, the video segments from 24 subjects, 8 for BD, 8 for UD and the remaining 8 subjects for the control group were collected at CHI-MEI Hospital, Tainan, Taiwan. K-fold (K=8) cross validation method was performed for evaluation and the detection accuracy achieved 68.3% for mood disorder detection. Compared with the well-known classifiers, such as Gaussian Model and deep neural network (DNN), the experimental results confirmed that our proposed method can achieve the best performance. Furthermore, the AU profiles and MS features are also beneficial to improve mood disorder detection performance.

[1] A. P. Association, Diagnostic and Statistical Manual of Mental Disorders (DSM-5®): American Psychiatric Pub, 2013.
[2] T. S.-T. Fu, C.-S. Lee, D. Gunnell, W.-C. Lee, and A. T.-A. Cheng, "Changing trends in the prevalence of common mental disorders in Taiwan: a 20-year repeated cross-sectional survey," The Lancet, vol. 381, pp. 235-241, 2013.
[3] 行政院衛生署. (2012). 民國91年至100年全國自殺死亡統計暨95年至100年自殺通報統計. Available: http://www.mohw.gov.tw/MOHW_Upload/doc/民國91年至100年全國自殺死亡統計暨95年至100年自殺通報統計_0031277000.pdf
[4] N. I. o. M. Health. Major Depression Among Adults. Available: http://www.nimh.nih.gov/health/statistics/prevalence/major-depression-among-adults.shtml
[5] N. I. o. M. Health. Bipolar Disorder Among Adults. Available: http://www.nimh.nih.gov/health/statistics/prevalence/bipolar-disorder-among-adults.shtml
[6] 國立雲林科技大學諮商輔導中心. 生命中的暗潮－男性與女性的憂鬱指數. Available: http://scc.yuntech.edu.tw/column/AA/a/a_01/a_01_12.htm
[7] 成大醫院護理部. 躁鬱症－疾病篇. Available: http://www.ncku.edu.tw/~nursedp/hygiene/hygiene_05_01.htm
[8] 自由時報. 躁鬱症被誤診 男子整天喊賭 1晚輸3萬. Available: http://news.ltn.com.tw/news/local/paper/104467/print
[9] 自由時報. 躁鬱症常被誤診 吃錯藥病更重. Available: http://news.ltn.com.tw/news/life/paper/634434
[10] R. H. Perlis, "Misdiagnosis of bipolar disorder," The American journal of managed care, vol. 11, pp. S271-4, 2005.
[11] J. F. Cohn, T. S. Kruez, I. Matthews, Y. Yang, M. H. Nguyen, M. T. Padilla, F. Zhou, and F. D. La Torre, "Detecting depression from facial actions and vocal prosody," in Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009. 3rd International Conference on, 2009, pp. 1-7.
[12] K. E. B. Ooi, M. Lech, and N. B. Allen, "Multichannel weighted speech classification system for prediction of major depression in adolescents," Biomedical Engineering, IEEE Transactions on, vol. 60, pp. 497-506, 2013.
[13] J. Kim, T. Nakamura, H. Kikuchi, K. Yoshiuchi, and Y. Yamamoto, "Co-variation of depressive mood and spontaneous physical activity evaluated by ecological momentary assessment in major depressive disorder," in Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, 2014, pp. 6635-6638.
[14] J. H. Kim, D. Diamond, and K. T. Lau, "Development of portable device for monitoring the lithium level from bipolar disorder patients," in Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2011 5th International Conference on, 2011, pp. 230-233.
[15] J. H. Kim, D. Diamond, and K. T. Lau, "Development of non-invasive biochemical device for monitoring the lithium level from saliva for bipolar disorder patients," in Sensors, 2011 IEEE, 2011, pp. 1744-1747.
[16] G. Valenza, M. Nardelli, G. Bertschy, A. Lanata, and E. P. Scilingo, "Complexity modulation in heart rate variability during pathological mental states of bipolar disorders," in Cardiovascular Oscillations (ESGCO), 2014 8th Conference of the European Study Group on, 2014, pp. 99-100.
[17] M. Sabatelli, V. Osmani, O. Mayora, A. Gruenerbl, and P. Lukowicz, "Correlation of significant places with self-reported state of bipolar disorder patients," in Wireless Mobile Communication and Healthcare (Mobihealth), 2014 EAI 4th International Conference on, 2014, pp. 116-119.
[18] Z. N. Karam, E. M. Provost, S. Singh, J. Montgomery, C. Archer, G. Harrington, and M. G. Mcinnis, "Ecologically valid long-term mood monitoring of individuals with bipolar disorder using speech," in Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, 2014, pp. 4858-4862.
[19] A. Grunerbl, A. Muaremi, V. Osmani, G. Bahle, S. Ohler, G. Troster, O. Mayora, C. Haring, and P. Lukowicz, "Smartphone-based recognition of States and state changes in bipolar disorder patients," Biomedical and Health Informatics, IEEE Journal of, vol. 19, pp. 140-148, 2015.
[20] D. P. David, M. G. Soeiro-de-Souza, R. A. Moreno, and D. S. Bio, "Facial emotion recognition and its correlation with executive functions in bipolar I patients and healthy controls," Journal of affective disorders, vol. 152, pp. 288-294, 2014.
[21] V. M. Goghari and S. R. Sponheim, "More pronounced deficits in facial emotion recognition for schizophrenia than bipolar disorder," Comprehensive psychiatry, vol. 54, pp. 388-397, 2013.
[22] A. C. Vederman, S. L. Weisenbach, L. J. Rapport, H. M. Leon, B. D. Haase, L. M. Franti, M.-P. Schallmo, E. F. Saunders, M. M. Kamali, and J.-K. Zubieta, "Modality-specific alterations in the perception of emotional stimuli in bipolar disorder compared to healthy controls and major depressive disorder," cortex, vol. 48, pp. 1027-1034, 2012.
[23] V. P. Bozikas, M. H. Kosmidis, T. Tonia, C. Andreou, K. Focas, and A. Karavatos, "Impaired perception of affective prosody in remitted patients with bipolar disorder," The Journal of neuropsychiatry and clinical neurosciences, vol. 19, pp. 436-440, 2007.
[24] C.-H. Chen, B. Lennox, R. Jacob, A. Calder, V. Lupson, R. Bisbrown-Chippendale, J. Suckling, and E. Bullmore, "Explicit and implicit facial affect recognition in manic and depressed states of bipolar disorder: a functional magnetic resonance imaging study," Biological psychiatry, vol. 59, pp. 31-39, 2006.
[25] S. A. Surguladze, A. W. Young, C. Senior, G. Brébion, M. J. Travis, and M. L. Phillips, "Recognition accuracy and response bias to happy and sad facial expressions in patients with major depression," Neuropsychology, vol. 18, p. 212, 2004.
[26] S. A. Langenecker, L. A. Bieliauskas, L. J. Rapport, J.-K. Zubieta, E. A. Wilde, and S. Berent, "Face emotion perception and executive functioning deficits in depression," Journal of Clinical and Experimental Neuropsychology, vol. 27, pp. 320-333, 2005.
[27] K. L. Schaefer, J. Baumann, B. A. Rich, D. A. Luckenbaugh, and C. A. Zarate, "Perception of facial emotion in adults with bipolar or unipolar depression and controls," Journal of psychiatric research, vol. 44, pp. 1229-1235, 2010.
[28] J. Gray, H. Venn, B. Montagne, L. Murray, M. Burt, E. Frigerio, D. Perrett, and A. H. Young, "Bipolar patients show mood‐congruent biases in sensitivity to facial expressions of emotion when exhibiting depressed symptoms, but not when exhibiting manic symptoms," Cognitive Neuropsychiatry, vol. 11, pp. 505-520, 2006.
[29] M. Summers, K. Papadopoulou, S. Bruno, L. Cipolotti, and M. A. Ron, "Bipolar I and bipolar II disorder: cognition and emotion processing," Psychological Medicine, vol. 36, pp. 1799-1809, 2006.
[30] G. Bersani, E. Polli, G. Valeriani, D. Zullo, C. Melcore, E. Capra, A. Quartini, P. Marino, A. Minichino, and L. Bernabei, "Facial expression in patients with bipolar disorder and schizophrenia in response to emotional stimuli: a partially shared cognitive and social deficit of the two disorders," Neuropsychiatric disease and treatment, vol. 9, p. 1137, 2013.
[31] Y. Li, S. Wang, Y. Zhao, and Q. Ji, "Simultaneous facial feature tracking and facial expression recognition," Image Processing, IEEE Transactions on, vol. 22, pp. 2559-2573, 2013.
[32] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, "Active shape models-their training and application," Computer vision and image understanding, vol. 61, pp. 38-59, 1995.
[33] T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," IEEE Transactions on Pattern Analysis & Machine Intelligence, pp. 681-685, 2001.
[34] M. Black and Y. Yacoob, "Recognizing facial expressions under rigid and non-rigid facial motions," in International Workshop on Automatic Face and Gesture Recognition, Zurich, 1995, pp. 12-17.
[35] C. Padgett, G. W. Cottrell, and B. Adolphs, "Categorical perception in facial emotion classification," in Proceedings of the Cognitive Science Conference, 1996, pp. 249-253.
[36] J. J.-J. Lien, T. Kanade, J. F. Cohn, and C.-C. Li, "Detection, tracking, and classification of action units in facial expression," Robotics and Autonomous Systems, vol. 31, pp. 131-146, 2000.
[37] M. S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Recognizing facial expression: machine learning and application to spontaneous behavior," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 2005, pp. 568-573.
[38] Y.-l. Tian, T. Kanade, and J. F. Cohn, "Recognizing action units for facial expression analysis," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 23, pp. 97-115, 2001.
[39] P. Ekman and W. V. Friesen, "Facial action coding system," 1977.
[40] M. Pantic and I. Patras, "Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences," Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 36, pp. 433-449, 2006.
[41] M. H. Mahoor, M. Zhou, K. L. Veon, S. M. Mavadati, and J. F. Cohn, "Facial action unit recognition with sparse representation," in Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, 2011, pp. 336-342.
[42] S. W. Chew, R. Rana, P. Lucey, S. Lucey, and S. Sridharan, "Sparse temporal representations for facial expression recognition," in Advances in Image and Video Technology, ed: Springer, 2012, pp. 311-322.
[43] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, "The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression," in Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, 2010, pp. 94-101.
[44] P. Philippot, "Inducing and assessing differentiated emotion-feeling states in the laboratory," Cognition & Emotion, vol. 7, pp. 171-193, 1993.
[45] 梁育綺, 謝淑蘭, 翁嘉英, and 孫蒨如, "台灣地區華人情緒與相關心理生理資料庫—標準化華語版情緒電影短片材料庫與主觀評量常模," Chinese Journal of Psychology, vol. 55, pp. 601-621, 2013.
[46] J. J. Gross and R. W. Levenson, "Emotion elicitation using films," Cognition & emotion, vol. 9, pp. 87-108, 1995.
[47] C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, pp. 273-297, 1995.
[48] Z. Weifa, "A SVM Text Classification Approch Based on Binary Tree," in Computer Science-Technology and Applications, 2009. IFCSTA'09. International Forum on, 2009, pp. 455-458.
[49] A. Ganapathiraju, J. Hamaker, and J. Picone, "Hybrid SVM/HMM architectures for speech recognition," in INTERSPEECH, 2000, pp. 504-507.
[50] C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Transactions on Intelligent Systems and Technology (TIST), vol. 2, p. 27, 2011.
[51] G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural computation, vol. 18, pp. 1527-1554, 2006.
[52] A.-r. Mohamed, G. E. Dahl, and G. Hinton, "Acoustic modeling using deep belief networks," Audio, Speech, and Language Processing, IEEE Transactions on, vol. 20, pp. 14-22, 2012.
[53] A. Fischer and C. Igel, "An introduction to restricted Boltzmann machines," in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, ed: Springer, 2012, pp. 14-36.
[54] G. Tzimiropoulos and M. Pantic, "Gauss-newton deformable part models for face alignment in-the-wild," in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, 2014, pp. 1851-1858.
[55] C. Computer Science Department. FACS - Facial Action Coding System. Available: http://www.cs.cmu.edu/~face/facs.htm