Facial Movements as Indicators of Fatigue in Air Traffic Control Tasks

Vivi Triyanti; Hastian Abdul Azis; Hardianto Iridiastadi; Yassierli Yassierli

doi:10.15866/irease.v13i5.18371

Facial Movements as Indicators of Fatigue in Air Traffic Control Tasks

Vivi Triyanti^(1*), Hastian Abdul Azis⁽²⁾, Hardianto Iridiastadi⁽³⁾, Yassierli Yassierli⁽⁴⁾

^(*) Corresponding author

Authors' affiliations

DOI: https://doi.org/10.15866/irease.v13i5.18371

Abstract

Air Traffic Control is a work that needs sustained vigilance for a certain period. This condition may result in mental fatigue that can lead to errors or incidents. Facial points could be potential indicators of fatigue that can be used in a real work environment. The objective of this research is to identify the parameters of facial points in order to detect fatigue considering correlation and accuracy value. An Experiment laboratory has been used to capture and recognize five facial points around the eyes of 12 participants when doing moderate and high traffic of simulated Air Traffic Control tasks. Six parameters that are related to movement and fixation of the facial points have been analyzed at five facial points. As a result, among 14 parameters-facial points that have a significant correlation to fatigue, only 4 of them at 2 facial points have good accuracy (>70%) in detecting the increase of fatigue level. Decreasing of movement velocity and increasing of fixation duration of points related to head and eyebrow have been found out when fatigue level increased. It can be concluded that some movement and fixation parameters of facial points are promising to be used in fatigue detection in Air Traffic Control tasks.
Copyright © 2020 Praise Worthy Prize - All rights reserved.

Keywords

Facial Movement; Mental Fatigue; Air Traffic Control Tasks

Full Text:

PDF

References

X. Zhang et al, Modeling pilot mental workload using information theory, The Aeronautical Journal 123 (2019), 828–839.

Santos, L., Melicio, R., Stress, Pressure and Fatigue on Aircraft Maintenance Personal, (2019) International Review of Aerospace Engineering (IREASE), 12 (1), pp. 35-45.
https://doi.org/10.15866/irease.v12i1.14860

Zgodavová, Z., Rozenberg, R., Szabo, S., Sabo, J., The Impact of Changes of Psychophysiological Factors on the Flight Crew Performance, (2019) International Review of Aerospace Engineering (IREASE), 12 (3), pp. 150-158.
https://doi.org/10.15866/irease.v12i3.16480

International Civil Aviation Organization (ICAO), International Federation of Air Traffic Controllers Associations (IFATCA), and Civil Air Navigation Services Organization (CANSO), Fatigue Management Guide for Air Traffic Service Providers, First edition, International Civil Aviation Organization, 2016.
https://doi.org/10.4324/9780429435614-3

Y.H. Chang et al, Effects of Work Shifts on Fatigue Levels of Air Traffic Controllers, Journal of Air Transport Management 76 (2019), 1-9.
https://doi.org/10.1016/j.jairtraman.2019.01.013

I. Tomic and J. Liu, Strategies to Overcome Fatigue in Air Traffic Control Based on Stress Management, The International Journal of Engineering and Science 6 (2017), 48-57.
https://doi.org/10.9790/1813-0604014857

L.A. Fowler and D. Gustafson, Video Game Play as a Fatigue Countermeasure in Air Traffic Controllers, Aerospace Medicine and Human Performance 90 (2019), 540-545.
https://doi.org/10.3357/amhp.5308.2019

M. Finke and T. H. Stelkens-Kobsch, Comparing Different Workload and Stress Assessment Methods in Air Traffic Control Simulations, The German Aerospace Congress (DLRK) (2017).

M. Marchitto et al, Air Traffic Control: Ocular Metrics Reflect Cognitive Complexity, International Journal of Industrial Ergonomics 54 (2016), 120-130.
https://doi.org/10.1016/j.ergon.2016.05.010

D. Dasari et al, ICA-Derived EEG Correlates to Mental Fatigue, Effort, and Workload in a Realistically Simulated Air Traffic Control Task, Frontiers in Neuroscience 11 (2017), 1-12.
https://doi.org/10.3389/fnins.2017.00297

L.L. Di Stasi et al, Microsaccade and Drift Dynamics Reflect Mental Fatigue, European Journal of Neuroscience (2013), 1–10.

M. Poursadeghiyan et al, Determination the Levels of Subjective and Observer Rating of Drowsiness and Their Associations with Facial Dynamic Changes, Iran J Public Health 46 (2017), 93-102.

D.Tianhong et al, Study on the Preferred Application-Oriented Index for Mental Fatigue Detection, Int. J. Environ. Res. Public Health 15 (2018), 1-26.

A.S. Le et al, A Novel Method for Classifying Driver Mental Workload Under Naturalistic Conditions With Information From Near-Infrared Spectroscopy, Frontiers in Neuroscience 12 (2018), 1-12.
https://doi.org/10.3389/fnhum.2018.00431

A.H. Molina et al, Using Psychophysiological Sensors to Assess Mental Workload in Web Browsing, Sensors 18 (2018), 1-26.
https://doi.org/10.3390/s18020458

A. S. Jayswal and R.V. Modi, Face and Eye Detection Techniques for Driver Drowsiness Detection, International Research Journal of Engineering and Technology 04 (2017) 2508-11.

W. B. Zhu et al, A Method for Recognizing Fatigue Driving Based on Dempster-Shafer Theory and Fuzzy Neural Network Mathematical Problems in Engineering 2017 (2017), 1-10.
https://doi.org/10.1155/2017/6191035

Othman, N., Romli, F., Mental Workload Evaluation of Pilots Using Pupil Dilation, (2016) International Review of Aerospace Engineering (IREASE), 9 (3), pp. 80-84.
https://doi.org/10.15866/irease.v9i3.9541

Y. Yamada and M. Kobayashi, Detecting Mental Fatigue from Eye-Tracking Data Gathered While Watching Video: Evaluation in Younger And Older Adults, Artificial Intelligence in Medicine 91 (2018), 39-48.
https://doi.org/10.1016/j.artmed.2018.06.005

L. Das et al, Towards Preventing Accidents in Process Industries by Inferring the Cognitive State of Control Room Operators through Eye Tracking, ACS Sustainable Chem. Eng 6 (2018), 2517–2528.
https://doi.org/10.1021/acssuschemeng.7b03971

Y. Zheng et al, Predicting Workload Experienced in a Flight Test by Measuring Workload in a Flight Simulator, Aerospace Medicine and Human Performance 90 (2019), 618-623.
https://doi.org/10.3357/amhp.5350.2019

M. Gavrilescu and N. Vizireanu, Neural Network Based Architecture for Fatigue Detection Based on the Facial Action Coding System, in O. Fratu, N. Militaru, S. Halunga (ed), Future Access Enablers for Ubiquitous and Intelligent Infrastructures, (Springer, 2017, pp 113-123).
https://doi.org/10.1007/978-3-319-92213-3_18

N. Irtija et al, Fatigue Detection Using Facial Landmarks, International Society of Affective Science and Engineering (2018), 41-47.

L Zhao et al, Facial Expression Recognition from Video Sequences Based on Spatial-Temporal Motion Local Binary Pattern and Gabor Multiorientation Fusion Histogram, Hindawi Mathematical Problems in Engineering 2017 (2017), 1-12.
https://doi.org/10.1155/2017/7206041

V. Triyanti et al, Eye Segment Movement as Indicators of Mental Workload in Air Traffic Controller, iMEC-APCOMS 2019, Lecture Notes in Mechanical Engineering, 238-244 (2020).
https://doi.org/10.1007/978-981-15-0950-6_37

V Triyanti et al, Fatigue-related Differences in Human Facial Dimensions Based on Static Images, IOP Conference Series: Materials Science and Engineering 528, 1 (2019).
https://doi.org/10.1088/1757-899x/528/1/012029

A. Bandini et al, Analysis Of Facial Expressions In Parkinson's Disease Through Video-Based Automatic Methods, Journal of Neuroscience Methods 281 (2017), 7-20.

G. Szirtesa et al. Behavioral Cues Help Predict Impact of Advertising on Future Sales, Image and Vision Computing 65 (2018), 49-57.
https://doi.org/10.1016/j.imavis.2017.03.002

V. Triyanti et al, Basic Emotion Recogniton using Automatic Facial Expression Analysis Software, Jurnal Optimasi Sistem Industri 18, 1 (2019), 55-64.
https://doi.org/10.25077/josi.v18.n1.p55-64.2019

B. Shishov, Mental Workload Estimation on Facial Video Using LSTM Network, 3rd IEEE International Conference on formation Management (2017).

R. Kawamura et al, Mental Fatigue Estimation Based on Facial Expression Change During Speech, Transactions of the Society of Instrument and Control Engineers 53 (2017), 90-98.

Y. Fang et al, Eye-Head Coordination for Visual Cognitive Processing, PLoS One 10 (2015),1-17.

E.G. Freedman, Coordination of The Eyes and Head During Visual Orienting, Exp Brain Res. 190 (2008), 369–387.
https://doi.org/10.1007/s00221-008-1504-8

D. Cazzoli et. al, Eye Movements Discriminate Fatigue Due to Chronotypical Factors and Time Spent On Task - A Double Dissociation, PLoS One 9 (2014), 1-5.
https://doi.org/10.1371/journal.pone.0087146

M. F. Pascual and B.G. Zapirain, Assessing Visual Attention using Eye Tracking Sensors in Intelligent Cognitive Therapies Based on Serious Games, Sensors 15 (2015), 11092-11117.
https://doi.org/10.3390/s150511092

V. Kazemi and J. Sullivan, One millisecond face alignment with an ensemble of regression trees, Proceeding of 27th IEEE Conference on Computer Vision and Pattern Recognition, Columbus, United States, 1867–1874 (2014).
https://doi.org/10.1109/cvpr.2014.241

X. Wang and C. Xu, Driver Drowsiness Detection Based on Non-Intrusive Metrics Considering Individual Specifics, Accident Analysis and Prevention 95 (2016), 350–357.
https://doi.org/10.1016/j.aap.2015.09.002

Fas-Millán, M., Pastor, E., Dynamic Workload Management for Multi-RPAS Pilots, (2019) International Review of Aerospace Engineering (IREASE), 12 (2), pp. 57-69.
https://doi.org/10.15866/irease.v12i2.15334

Refbacks

There are currently no refbacks.

Username
Password
Remember me