Abstract

L. Itti, Automatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention, IEEE Transactions on Image Processing, Vol. 13, No. 10, pp. 1304-1318, Oct 2004. [2003 impact factor: 2.642] (Cited by 992)

Abstract: We evaluate the applicability of a biologically-motivated algorithm to select visually-salient regions of interest in video streams for multiply-foveated video compression. Regions are selected based on a nonlinear integration of low-level visual cues, mimicking processing in primate occipital and posterior parietal cortex. A dynamic foveation filter then blurs every frame, increasingly with distance from salient locations. Sixty-three variants of the algorithm (varying number and shape of virtual foveas, maximum blur, and saliency competition) are evaluated against an outdoor video scene, using MPEG-1 and constant-quality MPEG-4 (DivX) encoding. Additional compression radios of 1.1 to 8.5 are achieved by foveation. Two variants of the algorithm are validated against eye fixations recorded from 4-6 human observers on a heterogeneous collection of 50 video clips (over 45,000 frames in total). Significantly higher overlap than expected by chance is found between human and algorithmic foveations. With both variants, foveated clips are on average approximately half the size of unfoveated clips, for both MPEG-1 and MPEG-4. These results suggest a general-purpose usefulness of the algorithm in improving compression ratios of unconstrained video.

Keywords: Visual attention ; video compression ; saliency ; bottom-up ; eye movements ; foveated

Themes: Model of Bottom-Up Saliency-Based Visual Attention, Computational Modeling, Computer Vision, Human Eye-Tracking Research