JAZZ-novo multisensor

operator's behavior capture system BASIC instrument for CognitoMetrics research

The CognitoMetrics is a new "start up" research area. Its basic assumption is, that the cognitive processes involved in learning and skill acquisition and its execution can be indirectly accessed by the quantitative evaluation of the human attention management. It further assumes that the "human error" is caused by the failure of human operator to adequately manage his/her attention resources, in particular due to the "physiological" attention anomalies.

The JAZZ-novo is the multisensor system that allows the acquisition of the eye movements with excellent spatial and temporal resolution, together with other physiological and environmental signals. The main idea behind developing the JAZZ-novo multisensor was to gather different kinds of information about pilot’s interaction in the cockpit, using a single device. The physiological signals measured by JAZZ-novo include:

• eye movements in horizontal and vertical axis (1000 Hz sampling frequency),
• head rotation velocity in horizontal and vertical axis (1000 Hz sampling frequency),
• head acceleration in horizontal and vertical axis (1000 Hz sampling frequency),
• photoplethysmographic signals in two lengths of the light wave (500 Hz sampling frequency)
• audio signal recording (8000 Hz sampling frequency).

Measurement technique

For the eye movement measurement the JAZZ-novo system utilizes the Cyclops-ODS (Oculus Dexter Sinister) technology (InfraRed Oculography - IRO) optimized for easy set-up and minimal intrusiveness, which are crucial for monitoring of subject behavior in the non-laboratory environment. Unlike most IRO setups, which require to place the infrared emitters/sensors around the eye in its close vicinity, the Cyclop-ODS’s set of optoelectronic sensors is placed between the eyes, hiding the sensor in the "shadow" of the nose. Thanks to such a setup, the limitation of the visual field are minimal, reducing the risk of JAZZ-novo interference with subject’s visual exploration of working environment. The eye movement measurements are performed with high temporal and spatial resolution, allowing the precise detection of saccades — fast eye movements used to move the point of gaze around the available field of view. Statistical processing of detected saccades over selected periods of time (their quantity, amplitude, duration of preceding fixations) provides important information about operator's visual attention involvement.

Head rotation velocity and acceleration measurements allows detection of the head movements linked with the visual exploration of the control room environment. Additionally the absolute measurement of horizontal head rotation velocity allows automatic calibration of horizontal eye movement in angular degree units.

The photoplethysmography signals measured by JAZZ-novo system, allow evaluation of the operator's heart beat and relative changes of the blood oxygenation. As these signals carry the information about the vasodilation/vasoconstriction responses regulated by the sympathetic system, its analysis can be used to access subject’s workload.

Audio signal recording carries information about experimenter’s/subject’s comments, audio signal in environment like, communication and talking with other operators in the control room.