Award Date

5-1-2021

Degree Type

Thesis

Degree Name

Master of Science in Electrical Engineering (MSEE)

Department

Electrical and Computer Engineering

First Committee Member

Brendan Morris

Second Committee Member

Venkatesan Muthukumar

Third Committee Member

Ebrahim Saberinia

Fourth Committee Member

Jennifer Rennels

Number of Pages

79

Abstract

Facial Emotion Recognition (FER) has become a popular computer vision topic and attracted a lot of researchers. However, these researches mostly focus on adult facial emotion. Infants’ facial structure and expression of emotions differ from adults’. That is why our target is to create a model that particularly focuses on infants’ facial emotions. In this work, we classify three types of emotions: positive, negative, and neutral. Two datasets are used in this work. One is an image-based dataset named ‘The City’ that consists of 154 images of infants aged 0-12 months. Another is the Rebel Dataset from the Department of Psychology of the University of Nevada, Las Vegas (UNLV). It consists of 50 videos of infants aged 6-10 months. Rebel dataset has a large number of unlabeled videos of children that need to be annotated. But manual annotation is time-consuming and expensive. We aim to investigate traditional feature-based Machine Learning (ML) FER approaches as well as Deep Learning (DL) approaches that can be used to label the datasets. One of the challenges for this task is dataset inadequacy. Most of the available FER datasets are on adult emotions. The few datasets that focus on infants’ facial emotions are all quite small for modern ML/DL approaches. The traditional feature-based methods work well on moderate-sized datasets. In our feature-based approaches, we extract features from our datasets and pass it through a classifier. Here we use Histogram of Oriented Gradients (HOG), Facial Action Units (AU), and Facial Landmarks (LM) as the features with Support Vector Machine (SVM) as the classifier. In our DL approach that requires large training data, we use transfer learning to overcome the dataset limitation. We used a pre-trained Convolutional Neural Network (CNN) with 1. ImageNet dataset 2. CK+ dataset 3. FER 2013 dataset before fine-tuning with the infant dataset. CK+ is the most popular posed adult FER dataset and FER 2013 is one of the largest wild FER datasets. We used these two to improve the model’s learning parameters to get a better result. We use CNN as fine-tuning and as Off-the-shelf with SVM classifier. Lastly, we use CNN-RNN (Recurrent Neural Network) network to classify the emotion from video sequences of our video dataset.

Keywords

CNN; Deep learning; Emotion recognition; Infant FER; LSTM GRU; Off the-shelf

Disciplines

Electrical and Computer Engineering

File Format

pdf

File Size

2400 MB

Degree Grantor

University of Nevada, Las Vegas

Language

English

Rights

IN COPYRIGHT. For more information about this rights statement, please visit http://rightsstatements.org/vocab/InC/1.0/

Download


Share

COinS