Abstract

L. Itti, P. F. Baldi, A Principled Approach to Detecting Surprising Events in Video, In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 631-637, Jun 2005. [2005 acceptance rate: 28%] (Cited by 538)

Abstract: Primates demonstrate unparalleled ability at rapidly orienting towards important events in complex dynamic environments. During rapid guidance of attention and gaze towards potential objects of interest or potential threats, however, often there is no time for detailed visual analysis. Thus, heuristic computations are necessary to locate the most interesting events in quasi real-time. We present a new theory of sensory surprise, which provides a principled and computable shortcut to important information. We develop a model that computes instantaneous low-level surprise at every location in video streams. The algorithm significantly correlates with eye movements of two human observers watching complex video clips, including television programs (17,936 frames, 2,152 saccadic gaze shifts). The resulting system allows more sophisticated and time-consuming image analysis to be efficiently focused onto the most surprising subsets of the incoming data.

Themes: Model of Bottom-Up Saliency-Based Visual Attention, Computer Vision, Human Eye-Tracking Research, Bayesian Theory of Surprise