TU Darmstadt: The eyes have a plan
Darmstadt, January 22, 2019. A team of researchers led by Constantin Rothkopf at the Centre for Cognitive Science at the Technische Universität Darmstadt has shown that humans unconsciously plan ahead where they look. The study published in „Scientific Reports“ compared where humans direct their gaze to algorithms for planning from artificial intelligence. The results are relevant for the understanding of the brain’s information processing ability.
Making a business plan, selecting a travel itinerary, or playing a game of chess have in common, that we need to plan our actions ahead of time in order to achieve our goals. A fundamental characteristic of planning is that sometimes we may need to get away from our goal temporarily: we need to invest by spending money before getting a return and earning, sometimes we need to sacrifice a chess piece to mate the king of our opponent, or we need to take a detour to get faster to our destination.
Artificial intelligence has a precise definition of planning, which says that deciding on the course of action by considering possible future situations before they are actually experienced constitutes planning. This makes planning tough, because many future consequences may need to be taken into account when planning ahead. It would be easier to only consider the consequences of a single next action, i.e. not to plan ahead. Unfortunately, in general this may have really detrimental consequences and may even lead to missing the goal altogether. Thus, how humans take future costs and benefits into account has considerable consequences.
Now researchers at the Centre for Cognitive Science have shown in a series of experiments that the way humans choose to direct their eyes must be planned ahead. To this end, the researchers measured fast eye movements of their participants, who were instructed to look for tiny targets within irregular shapes shown on a monitor. While in one condition participants had as little time as was sufficient to only carry out a single eye movement, in a second experiment enough time was available to carry out two eye movements. The experiments revealed that if participants were given time for a single eye movement their gaze selected a target that allowed a good coverage of the shape to find the target. By contrast, when it was possible to carry out two eye movements, participants automatically directed their eyes to a different point of the shape, which did not allow to find the target as well. But together with the second eye movement, the overall sequence of gaze shift was much more successful at finding the target.
Remarkably, the gaze targets selected by the human participants were well predicted by a planning algorithm from artificial intelligence. By contrast, previously utilized models for the prediction of human gaze targets, which do not plan ahead where to look, deviated from the observed behavior.
The present study reveals the sophistication of the information processing of the human visual system: it takes future actions into account in order to plan where to look next. This result is not only relevant for the understanding and prediction of human gaze shifts, which we carry out about three times a second, but also informs researchers about the information processing going on in the brain. While conscious planning as in playing chess, selecting a travel itinerary, or investing your hard earned money require effort and may be riddled by bad choices, our eyes plan for us almost optimally.
The Technische Universität (TU) Darmstadt is one of Germany’s leading technical universities. TU Darmstadt incorporates diverse science cultures to create its characteristic profile. The focus is set on engineering and natural sciences, which cooperate closely with outstanding humanities and social sciences. We are enjoying a worldwide reputation for excellent research in our highly-relevant, focused profile areas: cybersecurity, internet and digitalisation, nuclear physics, fluid dynamics and heat- and mass transfer, energy systems and new materials for product innovation. We dynamically develop our portfolio of research and teaching, innovation and transfer, in order to continue opening up important opportunities for the future of society. Our 312 professors, 4,450 scientific and administrative employees and close to 26,000 students devote their talents and best efforts to this goal. Together with Goethe University Frankfurt and Johannes Gutenberg University Mainz, TU Darmstadt has formed the strategic Rhine-Main Universities alliance.
David Hoppe & Constantin A. Rothkopf. Multi-step planning of eye movements in visual search. Scientific Reports, volume 9, Article number: 144 (2019)