Abstract

L. Itti, Automatic Attention-Based Prioritization of Unconstrained Video for Compression, In: Proc. SPIE Human Vision and Electronic Imaging IX (HVEI04), San Jose, CA, (B. Rogowitz, T. N. Pappas Ed.), Vol. 5292, pp. 272-283, Bellingham, WA:SPIE Press, Jan 2004. (Cited by 17)

Abstract: We apply a biologically-motivated algorithm that selects visually-salient regions of interest in video streams to multiply-foveated video compression. Regions of high encoding priority are selected based on nonlinear integration of low-level visual cues, mimicking processing in primate occipital and posterior parietal cortex. A dynamic foveation filter then blurs (foveates) every frame, increasingly with distance from high-priority regions. Sixty-three variants of the algorithm with different parameter settings are evaluated against an outdoor video scene, using MPEG-1 and MPEG-4, yielding compression radios of 1.1 to 8.5. Two variants (one with continuously-variable blur proportional to saliency at every pixel, and the other with blur proportional to distance from three independent foveation centers) are validated against eye fixations from 4-6 human observers on 50 video clips (synthetic stimuli, video games, outdoors day and night home video, television newscast, sports, talk-shows, etc). Significant overlap is found between human and algorithmic foveations on every clip with one variant, and on 48 out of 50 clips with the other. Substantial compressed file size reductions by a factor 0.5 on average are obtained for foveated compared to unfoveated clips. These results suggest a general-purpose usefulness of the algorithm in improving compression ratios of unconstrained video.

Themes: Computational Modeling, Model of Bottom-Up Saliency-Based Visual Attention, Computer Vision