Introduction

  The ability to predict emotions based on static and dynamic images has improved the possibilities in the fields of computer vision and robotics. However, this remains an important area of research and is continuously explored. Computer vision includes, among other things detecting, classifying, and recognizing objects and people on images (static) or videos (dynamic). To achieve this, visual information has to be extracted and processed accordingly by using Convolutional Neural Networks (CNN). In this project, two pre-trained CNN’s, the VGG16 and the Inception-v3 network were trained using the FER-2013 dataset from Kaggle, which consists of 48×48 pixels grayscale images containing different faces with emotions. Both networks were trained using the Transfer Learning method. To achive a better performance of the networks, the dataset was augmented accordingly.

 

Purpose

  The aim of the work was to categorize various pictures showing students with different facial expressions who were writing an exam into the seven emotion classes of anger, disgust, fear, joy, sadness, surprise, and neutral. The result of the training process and the prediction of the emotions was plotted on graphs and analyzed accordingly. Detailed information about the training process and the evaluation of the networks can be found in my master thesis. Nevertheless, there are many other use cases where FER can be applied. Some of them are listed bellow.

• Playing specific song types (i.e. Soul, Blues, Rock) based on the mood of a person.
• Detection of a suspicious persons before he can harm someone.
• Detecting a person if he is lying such like a lie detector, but only by analyzing his expression.
• Applying a weatherforecast based on the expression of one person or people.

 

Development

Development system	Ubuntu 18.04.2 LTS, Windows 10, Interl i7, 32GB RAM, Nvidia GTX1080, 1TB HDD
Development tools	PyCharm
Programming languages	Python
Libraries	NumPy, Matplotlib, OpenCV, dlib, scikit-learn, HDF5, pandas, TensorFlow, Keras
Repository

 

Images