47286: Student Informatics or similar - Integration of Eye-Tracking and Assistant Based Speech Recognition for the Interaction at the Controller Working Position

German Aerospace Center (DLR)

Braunschweig, Germany

Work group:

Institute of Flight Guidance



Area of research:

Diploma & Master Thesis



Job description:


The department controller assistance develops prototypic support systems for air traffic controllers. This also encompasses human machine interfaces and functionalities for information visualisation to the air traffic operator. Among other things, systems for assistant based speech recognition and for the detection of visual controller attention have already been developed.


An assistant based speech recognition system, for example, recognizes one or more controller commands and enters the values in the corresponding command type cell of an aircraft radar label in an emphasized form. This highlighting is intended to cause the air traffic controller to check and correct the displayed command(s) if it is / are incorrect. If all the information is correct and the controller has apparently checked this, the highlighting should disappear after a certain time (e.g. 20 seconds or at the time of looking at it). However, the controller may not have carried out a check despite the highlighting. This can be checked as far as possible by an eye tracker detecting whether the controller’s gaze has stayed at least for a short time on the highlighted label position or whether the mouse cursor has been specifically moved to the display position. If the controller has apparently not carried out any control, the highlighting should become clearer (higher visual escalation level) in order to draw the controller’s attention. After a further 20 seconds, if applicable, the highlighted values should be adopted automatically. The inspection using eye tracking and the time-dependent transfer of values are to be developed and integrated into an existing prototypic radar display. Ideally, this system can even be tested and validated with air traffic controllers in simulations at DLR.


In the course of your bachelor’s thesis, the following tasks need to be handled:


familiarisation with the concepts of assistant based speech recognition and eye-tracking based visual attention guidance development of concept for determining the visual scan of speech recognition output through the air traffic operator based on various sensor data and related displays implementation of the developed concept and creation of tests into existing source code of a software-toolbox for controller support systems including integration of gaze and speech recognition continuous documentation of working steps and results if applicable support for the evaluation of developed functionalities with air traffic controllers in simulation studies

Please apply via recruiter’s website.

Quote Reference: 47286