Isomorphic spatial visual-auditory displays for operations in DVE for obstacle avoidance

Abstract

Helicopter military missions such as combat search and rescue, medical evacuation and landing on unprepared sites can involve operating in hostile, low-altitude, and degraded visual environments (DVE). These conditions may significantly reduce the pilot's capability to use the natural out of the window (OTW) perceptual cues, increase workload and increase the risk of collision with terrain and natural or man-made obstacles. In modern helicopter cockpits, synthetic vision systems (SVSs) can employ conventional non-conformal two-dimensional (2D), egocentric three-dimensional (3D) conformal symbology (CS) and laser detection and ranging (LADAR)/ radio detection and ranging (RADAR)/ forward looking infrared (FLIR) imagery support guidance and control, especially during operations in DVE. Although 3D CS can decrease pilot workload, it can also produce attentional tunneling (cognitive capture) and may not provide maximally effective depiction of the environment around the helicopter. In this context, it is crucial to develop integrated multimodal interfaces that extend the current operational envelope while enhancing flight safety. Several flight simulator studies have investigated the use of spatial auditory displays (SADs) in combination with spatially and temporally congruent visual displays in tasks as diverse as collision avoidance, intruding aircraft detection, or system malfunction warning. In this paper we propose a novel approach to spatial sonification design based on the premises that perception-based synthetic cueing can increase situation awareness (SA), improve overall performance, and allow mental workload to be kept at operationally effective levels. This paper discusses the development, implementation, and evaluation of a sensor-based augmented-reality spatial auditory display (ARSAD) and its visual analog, an integrated collision avoidance display (ICAD) for all phases of flight. Five UH60M Army pilots participated in a low-level flight simulation evaluating the visual and the auditory displays, alone or in combination in low-visibility and zero visibility environments. The results are discussed in the context of pilot cueing synergies for DVE.