Abstract

A. Borji, D. N. Sihite, L. Itti, Modeling the Influence of Action on Spatial Attention in Visual Interactive Environments, In: Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 1-7, May 2012. [2012 acceptance rate: 40%] (Cited by 6)

Abstract: A large number of studies have been reported on top-down influences of visual attention. However, less progress have been made in understanding and modeling its mechanisms in real-world tasks. In this paper, we propose an approach for learning spatial attention taking into account influences of physical actions on top-down attention. For this purpose, we focus on interactive visual environments (video games) which are modest real-world simulations, where a player has to attend to certain aspects of visual stimuli and perform actions to achieve a goal. The basic idea is to learn a mapping from current mental state of the game player, represented by past actions and observations, to its gaze fixation. A data-driven approach is followed where we train a model from the data of some players and test it over a new subject. In particular, two contributions this paper makes are: 1) employing multimodal information including mean eye position, gist of a scene, physical actions, bottom-up saliency, and tagged events for state representation and 2) analysis of different methods of combining bottom-up and top-down influences. Comparing with other top-down task-driven and bottom-up spatio-temporal models, our approach shows higher NSS scores in predicting eye positions.

Themes: Computational Modeling, Model of Top-Down Attentional Modulation, Computer Vision