How Facial Features Convey Attention in Stationary Environments

Category

Health & Natural Sciences & Engineering > Engineering > Computer Science

Received

June 8, 2022

Accepted

August 19, 2022

Published

August 31, 2022

Authors

Janelle Domantay, University of Nevada, Las VegasFollow
Brendan Morris, University of Nevada, Las VegasFollow

Authors

Janelle Domantay (JD)^1* & Brendan Morris, Ph.D. (BM)¹

Author Affiliations

¹Department of Computer Science, University of Nevada, Las Vegas, Las Vegas, NV, USA.

Corresponding Author

*Janelle Domantay, domantay@unlv.nevada.edu

DOI

https://doi.org/10.9741/2766-7227.1021

Author Contributions

JD: Principal investigator, performed literature review, analysis, and interpretation, drafted and revised manuscript

BM: Supervised principal investigation, advised research direction, and provided manuscript revisions

Copyright

Articles in Spectra are freely available under a Creative Commons Attribution License (CC BY 4.0) which allows others to re-use the work without permission as long as the work is properly cited.

Data Availability Statement

The authors of this article confirm that all data underlying the findings are fully available without restrictions.

Conflicts of Interest

The authors declare that they have no conflict of interest.

Ethical Considerations

Given that this project did not involve human or animal subjects, no IRB or IACUC approval was needed. All research was derived from publicly shared sources.

Funding

This research was funded by the UNLV Office of Undergraduate Research’s (OUR) 2021 Undergraduate Research Stimulus Program (URSP) Track 1: Undergraduate Research Mentor-Mentee Program.

Abstract

Awareness detection technologies have been gaining traction in a variety of enterprises; most often used for driver fatigue detection, recent research has shifted towards using computer vision technologies to analyze user attention in environments such as online classrooms. This paper aims to extend previous research on distraction detection by analyzing which visual features contribute most to predicting awareness and fatigue. We utilized the open-source facial analysis toolkit OpenFace in order to analyze visual data of subjects at varying levels of attentiveness. Then, using a Support-Vector Machine (SVM) we created several prediction models for user attention and identified the Histogram of Oriented Gradients (HOG) and Action Units to be the greatest predictors of the features we tested. We also compared the performance of this SVM to deep learning approaches that utilize Convolutional and/or Recurrent neural networks (CNNs and CRNNs). Interestingly, CRNNs did not appear to perform significantly better than their CNN counterparts. While deep learning methods achieved greater prediction accuracy, SVMs utilized less resources and, using certain parameters, were able to approach the performance of deep learning methods.

Keywords

Machine learning, neural networks, drowsiness, attention, computer vision

Submission Type

Primary research article

Recommended Citation

Domantay, J., & Morris, B. (2022). How facial features convey attention in stationary environments. Spectra Undergraduate Research Journal, 2(2), 66-88. https://doi.org/10.9741/2766-7227.1021