To interpret our results, it is useful to note that (not unexpectedly) the figures for the average recommended blur (over the extent of each frame, then over all frames) closely matched a compound measure of blur over a random scanpath (not shown). Thus, in the % at eye columns of the table below, a value of 100% or more would indicate that humans did not look at regions of high model priority (low suggested blur) more than a random scanpath, while a value of 0% would indicate that humans always gazed exactly at regions of maximum priority (no suggested blur).
Windows users beware: Somehow, MPEG-1 clips encoded with the reference program mpeg_encode do not play smoothly on Windows 2000 using the Windows Media Player anymore (they used to, but new versions of the Windows Media Player have broken that). They play well on Windows 98 and Windows NT (at least the versions we have here). If the movies appear choppy (playing too fast for a second, then pausing, then continuing too fast, etc), we recommend that you try playing them with the Quicktime Player rather than the Windows Media Player. All movies play smoothly at 30 frames/s under Linux using mpeg_play or mplayer.
| Maxnorm, 0 fov, depth 4 | Fancynorm, 3 fov, depth 6 | ||||||||
| Video clip | Frames | Subjects | %avg at eye | mpg1 | mpg4 | %avg at eye | mpg1 | mpg4 | |
| beverly01 |  | 490 | 5 | 52.3 +/- 18.5 | 36.4% | 64.6% | 41.4 +/- 13.2 | 38.1% | 65.1% |
| beverly03 |  | 481 | 5 | 42.9 +/- 9.5 | 37.6% | 60.2% | 21.5 +/- 4.1 | 33.9% | 58.8% |
| beverly05 |  | 546 | 4 | 40.3 +/- 8.7 | 51.3% | 91.6% | 43.5 +/- 9.3 | 48.6% | 90.4% |
| beverly06 |  | 521 | 4 | 39.0 +/- 9.9 | 40.1% | 88.9% | 23.9 +/- 11.0 | 44.3% | 87.5% |
| beverly07 |  | 357 | 4 | 48.5 +/- 11.8 | 25.7% | 69.8% | 31.2 +/- 3.9 | 32.3% | 71.0% |
| beverly08 |  | 237 | 5 | 67.7 +/- 5.6 | 25.5% | 64.5% | 36.6 +/- 15.5 | 34.2% | 68.1% |
| gamecube02 |  | 1819 | 6 | 55.1 +/- 5.0 | 73.0% | 98.9% | 42.5 +/- 4.7 | 65.4% | 98.1% |
| gamecube04 |  | 2083 | 4 | 50.5 +/- 1.4 | 77.7% | 99.3% | 27.9 +/- 9.6 | 73.0% | 98.8% |
| gamecube05 |  | 213 | 6 | 37.3 +/- 7.3 | 82.2% | 79.1% | 15.1 +/- 5.8 | 88.7% | 90.3% |
| gamecube06 |  | 2440 | 6 | 35.7 +/- 7.8 | 58.9% | 95.6% | 36.4 +/- 6.6 | 52.8% | 95.4% |
| gamecube13 |  | 898 | 5 | 72.9 +/- 3.0 | 54.6% | 80.4% | 52.3 +/- 8.8 | 50.8% | 79.5% |
| gamecube16 |  | 2814 | 4 | 81.0 +/- 7.5 | 57.2% | 96.7% | 38.8 +/- 7.9 | 53.6% | 96.7% |
| gamecube17 |  | 2114 | 5 | 44.6 +/- 8.3 | 82.5% | 100.2% | 31.2 +/- 11.0 | 83.2% | 100.0% |
| gamecube18 |  | 1999 | 5 | 52.8 +/- 4.0 | 74.4% | 99.3% | 29.1 +/- 8.5 | 74.1% | 99.7% |
| gamecube23 |  | 1429 | 4 | 28.6 +/- 9.9 | 59.6% | 95.8% | 30.1 +/- 14.5 | 58.4% | 96.6% |
| monica03 |  | 1526 | 5 | 53.3 +/- 13.8 | 51.2% | 85.5% | 40.3 +/- 3.3 | 46.1% | 85.4% |
| monica04 |  | 640 | 5 | 47.0 +/- 6.8 | 47.6% | 78.7% | 23.2 +/- 6.3 | 43.8% | 78.2% |
| monica05 |  | 611 | 4 | 60.8 +/- 6.8 | 43.4% | 77.8% | 58.5 +/- 6.3 | 40.7% | 78.9% |
| monica06 |  | 164 | 4 | 23.8 +/- 6.5 | 47.6% | 63.6% | 41.7 +/- 12.3 | 43.0% | 64.3% |
| saccadetest |  | 516 | 5 | 29.2 +/- 10.3 | 24.8% | 70.7% | 14.4 +/- 10.8 | 34.4% | 107.7% |
| standard01 |  | 254 | 4 | 58.0 +/- 6.5 | 49.9% | 68.8% | 70.1 +/- 19.3 | 38.5% | 66.0% |
| standard02 |  | 515 | 5 | 51.0 +/- 4.4 | 49.5% | 71.1% | 52.4 +/- 11.0 | 42.0% | 70.3% |
| standard03 |  | 309 | 4 | 72.8 +/- 10.5 | 52.3% | 72.0% | 78.8 +/- 9.5 | 46.5% | 73.5% |
| standard04 |  | 612 | 5 | 81.3 +/- 6.1 | 47.7% | 76.1% | 77.0 +/- 3.4 | 40.6% | 73.7% |
| standard05 |  | 483 | 5 | 52.2 +/- 1.4 | 52.7% | 75.9% | 53.6 +/- 8.1 | 43.3% | 74.0% |
| standard06 |  | 434 | 5 | 63.3 +/- 2.9 | 52.0% | 75.4% | 67.1 +/- 14.3 | 40.9% | 74.0% |
| standard07 |  | 177 | 4 | 43.2 +/- 4.9 | 43.0% | 51.1% | 42.8 +/- 9.8 | 39.3% | 52.1% |
| tv-action01 |  | 567 | 4 | 39.0 +/- 1.5 | 47.0% | 55.0% | 20.0 +/- 4.3 | 42.4% | 53.6% |
| tv-ads01 |  | 1077 | 4 | 79.4 +/- 5.4 | 59.2% | 98.4% | 52.4 +/- 10.5 | 65.2% | 98.9% |
| tv-ads02 |  | 387 | 4 | 60.3 +/- 9.3 | 52.4% | 93.0% | 48.4 +/- 5.4 | 57.0% | 93.8% |
| tv-ads03 |  | 841 | 5 | 64.9 +/- 14.3 | 48.8% | 78.0% | 44.2 +/- 15.6 | 46.3% | 77.6% |
| tv-ads04 |  | 313 | 5 | 43.9 +/- 3.3 | 56.1% | 95.4% | 44.0 +/- 9.6 | 53.8% | 92.3% |
| tv-announce01 |  | 434 | 4 | 78.0 +/- 2.2 | 60.6% | 88.5% | 51.2 +/- 10.6 | 59.0% | 89.6% |
| tv-music01 |  | 1022 | 5 | 59.9 +/- 3.9 | 51.6% | 82.6% | 51.0 +/- 5.6 | 45.8% | 82.8% |
| tv-news01 |  | 918 | 5 | 59.5 +/- 3.9 | 46.9% | 98.7% | 70.3 +/- 5.2 | 51.3% | 98.5% |
| tv-news02 |  | 1058 | 6 | 60.0 +/- 2.5 | 57.5% | 98.2% | 67.5 +/- 14.2 | 62.2% | 98.7% |
| tv-news03 |  | 1444 | 5 | 71.8 +/- 1.9 | 53.7% | 99.7% | 67.0 +/- 9.1 | 62.8% | 98.6% |
| tv-news04 |  | 491 | 5 | 33.5 +/- 9.5 | 52.1% | 85.9% | 31.4 +/- 8.0 | 55.7% | 87.7% |
| tv-news05 |  | 1341 | 5 | 60.7 +/- 4.3 | 65.3% | 98.9% | 46.1 +/- 5.6 | 70.1% | 99.4% |
| tv-news06 |  | 1643 | 5 | 72.9 +/- 7.0 | 56.2% | 99.4% | 78.9 +/- 11.1 | 62.4% | 100.2% |
| tv-news09 |  | 1176 | 4 | 69.3 +/- 6.4 | 53.4% | 99.6% | 87.8 +/- 16.8 | 59.2% | 99.0% |
| tv-sports01 |  | 579 | 5 | 67.1 +/- 6.1 | 46.0% | 70.0% | 51.4 +/- 10.8 | 43.8% | 71.1% |
| tv-sports02 |  | 444 | 4 | 72.0 +/- 9.7 | 53.4% | 95.6% | 56.5 +/- 11.5 | 54.0% | 95.3% |
| tv-sports03 |  | 1460 | 5 | 52.7 +/- 5.6 | 46.7% | 87.2% | 50.3 +/- 7.7 | 43.5% | 86.3% |
| tv-sports04 |  | 982 | 4 | 79.4 +/- 5.2 | 43.3% | 68.0% | 56.3 +/- 4.8 | 43.9% | 68.6% |
| tv-sports05 |  | 1386 | 6 | 55.0 +/- 6.1 | 50.5% | 76.6% | 41.4 +/- 3.0 | 45.6% | 74.6% |
| tv-talk01 |  | 1651 | 4 | 37.7 +/- 3.5 | 65.7% | 98.9% | 42.8 +/- 13.2 | 61.3% | 99.1% |
| tv-talk03 |  | 783 | 5 | 56.4 +/- 4.2 | 44.8% | 83.7% | 43.3 +/- 5.7 | 43.7% | 84.4% |
| tv-talk04 |  | 1258 | 5 | 51.8 +/- 4.2 | 36.8% | 98.7% | 55.5 +/- 10.9 | 42.7% | 98.7% |
| tv-talk05 |  | 552 | 4 | 64.9 +/- 1.3 | 38.1% | 96.3% | 70.2 +/- 9.7 | 41.9% | 96.0% |
| SUMMARY | 46,489 | 4.7 | 55.1 +/- 7.3 | 51.7% | 84.0% | 46.5 +/- 9.7 | 50.9% | 84.8% | |
To evaluate the algorithm, we here simply used it as a front-end, applied before standard video compression algorithms: a spatially-variable blur was applied to the input frames, that was inversely proportional to the model-computed priority (lower-priority regions are more strongly blurred). Although this is not optimal in terms of expected file size gains, it has the advantage of producing compressed streams that are compatible with existing decoders. This method should be regarded as a worst-case scenario for two reasons: first, as the foveas move from frame to frame, the appearance of static objects far from the foveas will change, requiring continuous re-encoding of those changes while no re-encoding is necessary in the absence of foveation; second, even when foveas remain for some time at fixed locations, peripheral moving objects will receive variable amounts of blur, depending on their distance to the closest fovea, hence defeating any motion compensation scheme in the encoder, and also requiring continuous re-encoding of the differences in their appearance. Any file size gain obtained despite these limitations hence represents the promise that even better size gains should be obtained if a specific video codec was to be created, which would truly use the model's priority maps to prioritize encoding (e.g., encode high-priority regions first, then lower-priority regions, in a continuously-variable-bitrate encoding scheme).
Copyright © 2003 by the University of Southern California, iLab and Prof. Laurent Itti