Multimodal Cueing In Attitude Tracking Tasks: Toward The Prediction Of Pilot Cognitive Workload Via Physiological Measurements

Abstract

This paper investigates the relationship between variations in physiological signals, pilot performance, and mental workload during the execution of a roll-attitude compensatory tracking task. In this task, the pilot acts on the displayed error between a desired input and the comparable vehicle output motion to produce a control action. The error is displayed to the pilot using multiple cueing modalities: visual, haptic, audio, and permutations of these. In this experiment heart activity, respiration, brain activity, skin temperature, and electrodermal activity (EDA) are measured and correlated with both the participants' performance in the task, as well as with perceived mental workload through the Bedford workload questionnaire. Specific goals of the study include: (i) assessment of the perceived mental workload (MWL) for each of the cueing modalities in consideration, (ii) analysis of the relationship and correlation between performance and perceived MWL, (iii) investigation of the ability of physiological signals to distinguish different levels of MWL, and (iv) providing preliminary insights into which physiological signals are most sensitive to cognitive effort. Statistical analysis revealed that while perceived MWL is not significantly affected by multimodal cueing in good visual conditions, multimodal cueing reduces MWL in degraded/denied visual environments. Additionally, physiological responses consistently distinguished between different MWL levels across all eleven cueing modalities studied. Respiration emerged as the most statistically significant indicator, with a robust correlation with MWL variations. Conversely, EDA and skin temperature showed limited discriminatory capability in this context. The behaviors of ECG and fNIRS signals provided less consistent results and require further investigation to validate their utility in MWL assessment.