2002 E3 Report (Electronics Entertainment Expo)

The 2002 E3 came and went, but here is a full report on the future trends set by the gaming industry. The gaming community, with its various salient interfaces and its much vaunted multi-million dollar booths that exhibited the current reigning consoles were captured in our exclusive footage. Presented through iLab, complete with predictions from our visual attention model (captured by yellow and green circles) and saliency maps (shown to the right of each video clip).

The purpose of this footage is to test the visual saliency algorithm on video games. iLab is investigating possible applications of the saliency software to the development of more sophisticated opponents. These opponents will use intelligent agents that would employ the iLab algorithms for visual perception. Testing is also done on the Beobot robotics platform to be released this year, with external components designed by yours truly.

At left are MPEG movie clips from the latest GameCube and PlayStation 2 games. The footage includes the following line-up: Legend of Zelda, Super Mario Sunshine, Star Wars, Eternal Darkness, Harry Potter, Tony Hawk's Pro Skater 4, Super Monkey Ball 2, and a variety of other games. Below is an explanation of the visual saliency software application. The software analyzes specifically 4 characteristics of any given visual scene: color, intensity, motion, and edges, over multiple spatial scales. Although this software can be attuned to even more specific characteristics (a specific color, for example), it is not the case here, so as to give each game the same evaluation. The saliency software also considers the 'inhibition of return,' which means that once a specific location has been visited, it will not return to the location before 1 to 2 seconds. Each movie clip is in real-time, and the software updates at 30 frames per second.

Results

After evaluating the movie clips, certain games were more likely to be useful for testing the effectiveness of creating computer opponents that use visual attention to locate and combat human players.

Player takes the cake

The best results came from: Legend of Zelda, Super Mario Sunshine, Sly Cooper, Gladius and Beach Spikers. In these games the main player was salient either because it contained a unique color that distinguished it from the background (green and red costumes for Zelda and Mario respectively), constant motion throughout the clip, or geometric shapes above their heads (Beach Spikers).

Background takes the cake

In games like Super Monkey Ball2, Harry Potter, 1080: Whitestorm, and Tony Hawk's Pro Skater 4, the main player either remained close to the center throughout the footage or the background contained constantly changing patterns and colors. Consequently, the main player was attended to occasionally due to the inhibition of return mechanism. Only when the player reacted to obstacles in the background did it move away from the center of the visual field (the inhibition of return zone) and was attended to.

What does this all mean?

Although the visual saliency algorithm was by no means developed or tuned to video games, it demonstrated a remarkable ability to focus onto the most meaningful elements of any given scene. When those elements remained at a constant position, they were attended to only occasionally (due to inhibition of return) while when they rapidly moved across the visual field, they were tracked by the software.

The Nintendo and Sony booths can be seen below, with aerial views of heavy traffic and gaming stations. While both consoles were designed to appeal to the serious gamer, the approach taken by each was dramatically different. PlayStation had more of a lounge scene setting, equipped with large double-sided projection screens on clear glass panels, glowing plastic sofas and satellite speakers behind the sofas for the full gaming experience of the showcased games. Nintendo, on the opposite end, had several intimate hidden coves with only two essential things needed to play the showcased game: a large flat-screen and the gaming device dedicated to the showcased game. Both consoles provided representatives to answer questions and included periodic games to entice visitors.

''Booth babes'' were also on hand to spice up the experience, these girls fulfill their mission as a salient feature at the expo. The movie clips below further demonstrate this fact.