近年來，臉部表情辨識是許多專家學者在研究發展的議題，本論文主要是發展一套自動化的表情辨識系統，可以從影像輸入之後自動化地偵測人臉、擷取特徵到表情辨識。藉由自動人臉偵測、特徵區域的概念與光流追蹤的方法，可以簡單快速地組成一套自動化的表情辨識系統，達成我們的目標。 大部分傳統的表情辨識系統是尋求自動追蹤臉部某些特徵(如眼角、眉尖、嘴角)的方法且以這樣的擷取特徵作為表情辨識的基礎。但是實驗的結果顯示影像品質、光線或其他干擾因素都會造成無法確實擷取臉部特徵，某些影像屬性的影響就足以造成不小的誤差，即使能夠克服也會付出一定的運算時間。雖然清楚的特徵點可以帶來極大的貢獻，但是臉部非特徵點的某些微小肌肉變化也可以讓人感覺出表情的變化，所以我們採用特定特徵區域且平均特徵點的做法，藉由這些特徵點在臉部的移動判別出人臉的表情。 根據這樣的構想，我們在取得一段序列影像之後，以第一張影像做人臉偵測，並依幾何比例關係取得雙眼及嘴巴三個特徵區域，為求特徵區域選取的正確性，我使用了Sobel邊緣偵測、水平投影兩方法，將三個特徵區域的範圍做更精確的選取。特徵區域定義完成之後，將特徵點做平均的分配，三個特徵區域共定義了84個點，接著利用光流(Optical Flow)演算法追蹤之後序列影像中的這84個點，追蹤完畢我們即可取得84個臉部移動向量，因此，基於這84個移動向量就可以進行表情辨識。我們的表情辨識系統包含兩個階段，第一個階段裡，訓練三個多層感知機來辨識三個區域（眉毛、眼睛和嘴巴）的基本動作單元，接著基於上述三個多層感知機的輸出，我們使用五個單層感知機來辨識基本情緒表情。最後我們以實驗來測試本表情辨識系統的效果，而且有不錯的結果。 Recently, researchers have put a lot of efforts on the recognition of facial expressions. The goal of the thesis is to develop an automatic facial expression recognition system to automatically perform human face detection, feature extraction and facial expression recognition after the images are faded. Via the use of the automatic human face detection, the region of facial features and the optical flow tracking algorithm, we can construct an automatic facial expression recognition system to achieve our goal. Most of the traditional facial expression systems are first to look for a way to automatically track some facial feature points (ex: canthus, eyebrows, and mouth) and then recognize expressions based on these extracted facial features. But experimental results exhibited that the facial features cannot always be obtained reliably because of the quality of images, illumination, and some other disturbing factors. Some properties of images contribute a lot of errors or bias and cost a lot of process time to overcome them if possible. Although the clear features can make a lot of contribution on the performance, we can also feel the changes of facial expression according to some slight muscle variations of facial area. So the way to utilize some specified feature regions and the uniform-distributed feature points is used to for the facial expression from the motion of these feature points. After a series of images are derived, according to the proposed idea, the first frame is used to perform human face detection, and get the three feature regions (eyes and mouth) by their geometrical ratio relationships. To increase the accuracy of locating feature regions, the Sobel edge detection incorporated with the horizontal projection is used. After three feature regions have been located 84 feature points are uniformly distributed in the specified feature regions. Then we use the optical flow algorithm to track these 84 feature points on the following image series. Therefore, 84 facial motion vectors can be derived from the tracking procedure. Then the facial expression recognition is based on these 84 facial motion vectors. The facial recognition procedures involves in two stages. At the first stage, three multi-layer perceptrons are trained to recognize the action units in the eyebrow, the eye and the mouth regions. Then another five single-layer perceptrons are used to recognize the facial expressions based on the outputs computed from the aforementioned three MLPs. Experiments were conducted to test the performance of the proposed facial recognition system.
