Abstract

L. Itti, P. F. Baldi, Modeling what attracts human gaze over dynamic natural scenes, In: Computational Vision in Neural and Machine Systems, (L. Harris, M. Jenkin Ed.), Cambridge, MA:Cambridge University Press, 2006. (Cited by 791)

Abstract: Attention in biological and artificial systems rapidly selects important information within massive sensory inputs, a process key to survival. When there is little time for detailed sensory analysis, finding important information must rely on heuristic computations. To characterize these computations, we propose a general Bayesian definition of important information we call surprise. Surprise quantifies how data affects a natural or artificial observer, by measuring the difference between prior and posterior beliefs of the observer. We find that surprise outperforms five other metrics previously proposed in the literature in predicting recorded gaze shifts of four humans watching 25 minutes of video stimuli (over 45,000 distinct video frames), including television broadcast and video games. The Bayesian theory of surprise presented in this chapter is general and applicable to domains beyond visual attention, across different modalities, datatypes, tasks, and abstraction levels.

Themes: Computational Modeling, Model of Bottom-Up Saliency-Based Visual Attention, Computer Vision, Bayesian Theory of Surprise, Human Eye-Tracking Research