Seeing the world through a retina


Original image from a CCD camera.	Same image seen by a retina (right eye, fixation at center).

The retina is very different, in terms of scene digitization, from a CCD camera which uniformly samples a visual scene. The retina has a specialized central region (the fovea) which images the immediate neighborhood of the point of fixation with a much higher resolution than peripheral regions of the visual field. Around the fovea, the sampling density (i.e. the actual resolution) of the retina falls-off quickly as we move towards the periphery.

Another complication is the presence of a blind spot. At the blind spot, all nerve fibers coming from the photoreception and processing layers of the retina leave the eye to form the optic nerve (about one million fibers for each eye). Because of the presence of this massive bundle of nerve fibers, there are no photoreceptors at the location where it leaves the eye. As a result, a small area of the visual field, of about 5 degrees of visual angle in diameter and at 15 degrees of eccentricity from the fovea, on the temporal side of the visual field, is not represented by the retina. Although you are not aware of the presence of your blind spots, they are easy to find: with one eye closed, fixate a point in front of you. Without moving your open eye, move your finger at arm's length in front of you, from the fixation point towards the periphery, in the horizontal plane. At an angle of about 15 degrees, your finger will disappear. At smaller and larger angles, you will be able to see your finger.

While we have three types of photoreceptors for color perception (roughly sensitive for red, green and blue wavelengths), the central 1 degree of the visual field (the fovea) is almost entirely devoid of blue-type photoreceptors.

Finally, we move our eyes on average 100,000 to 150,000 times every day. The goal of these, usually very rapid ("saccadic", up to velocities of about 800 degrees per second), eye movements is to always move the fovea towards the objects of current interest, in order to examine them with the region of the retina that has finest detail.

We are not aware of most of the phenomena mentioned above. Here is how the world would look if we were...

(2.8 Mb MPEG)

Simulation of how a video clip looks like seen by a retina. This is our original demo which we developed with Prof. Christof Koch at Caltech and which appeared in the May 2000 issue of Discover Magazine.

(11.0 Mb MPEG)

(3.0 Mb MPEG)

(3.5 Mb MPEG)

These clips are another simulation of how the raw inputs to your retina may look like when you inspect a video clip and make eye movements, now using an eye-tracker to capture the eye position of a human observer watching the video clip.

First, have a look at the first clip (top). This is a normal video sequence filmed with a camcorder. We recorded eye position from a human subject as he watched this video clip.

Now have a look at the second clip (middle). This is the same as the first one, except that each frame has been shifted to the current eye position of a human observer; thus this version shows you the distribution of light intensity that hit the subject's retina as he was making eye movements to explore the clip (assuming a black border around the clip). Interestingly, while the motion of the scene induced by eye movements is obvious and extremely annoying in the second clip, subjects do not perceive the same annoyance when they execute their own eye movements, although the visual input hitting their retinas is essentially the same as what you can see here by maintaining your gaze fixed at the center of the images.

The third clip (bottom) is the same as the second one, except for a slightly wider field of view and grey background.