SimulationViewerCompress Class Reference

Do multi-foveated saliency-based image compression. More...

#include <Neuro/SimulationViewerCompress.H>

Inheritance diagram for SimulationViewerCompress:
Inheritance graph
[legend]
Collaboration diagram for SimulationViewerCompress:
Collaboration graph
[legend]

List of all members.

Public Member Functions

Constructors and destructors

 SimulationViewerCompress (OptionManager &mgr, const std::string &descrName="Multi-Foveated Compression Simulation Viewer", const std::string &tagName="SimulationViewerCompress")
 Constructor. See ModelComponent.H.
virtual ~SimulationViewerCompress ()
 Destructor.

Protected Member Functions

 SIMCALLBACK_DECLARE (SimulationViewerCompress, SimEventRetinaImage)
 Callback for when a new retina image is available.
 SIMCALLBACK_DECLARE (SimulationViewerCompress, SimEventSaccadeStatusEye)
 Callback for when the eye moves.
 SIMCALLBACK_DECLARE (SimulationViewerCompress, SimEventSaveOutput)
 Callback for every time we should save our outputs.
Image< PixRGB< byte > > getTraj (SimEventQueue &q)
 Get the attention/eye/head trajectory image.
virtual void start1 ()
 Get started and disable any SC the Brain may want to use.
virtual void stop1 ()
 get stopped
virtual void paramChanged (ModelParamBase *const param, const bool valueChanged, ParamClient::ChangeStatus *status)
 Intercept people changing our number of foveas.

Protected Attributes

OModelParam< int > itsFOAradius
 FOA radius for object trackers.
OModelParam< int > itsNumFoveas
 number of foveas
OModelParam< bool > itsSaveTraj
 save trajectory?
OModelParam< bool > itsSaveMegaCombo
 save mega combo?
OModelParam< bool > itsSaveMask
 save mask?
OModelParam< bool > itsSaveFoveatedImage
 save foveated image?
OModelParam< float > itsDistanceFactor
 distance factor to change the fovea size
OModelParam< bool > itsSaveEyeCombo
 save eye combo?
OModelParam< bool > itsDisplayPatch
 draw patches
OModelParam< bool > itsDisplayFOA
 draw objetc outlines
OModelParam< bool > itsDisplayEye
 draw human eye movements
NModelParam< PixRGB< byte > > itsColorNormal
 patch color
NModelParam< PixRGB< byte > > itsColorEye
 patch color for human eye mvts
OModelParam< int > itsHeadRadius
 head radius
OModelParam< int > itsMultiRetinaDepth
 depth of blur pyramid
OModelParam< int > itsCacheSize
 size of our mask cache
OModelParam< bool > itsUseTRMmax
 use TRM to take max in cache
OModelParam< std::stringitsFoveaSCtype
 type of SC for foveas
OModelParam< std::stringitsOutFname
 Name of output file.
OModelParam< LevelSpecitsLevelSpec
 our levelspec
OModelParam< int > itsNumRandomSamples
 number of random samples
OModelParam< bool > itsEyeCompare
 do the eye compare with the mask
OModelParam< int > itsIFramePeriod
 This parameter is the period (in frames) for foveation mask to change.

Detailed Description

Do multi-foveated saliency-based image compression.

This viewer will consider the top N salient locations and apply blurring to the image that increases in strength (sigma) as we get farther away from any of the top N most salient locations. The resulting trajectory image hence will be crisp at the top N locations and increasingly blurred away from those. Compressing the output of this viewer, e.g., using MPEG, yields smaller file size while (hopefully) preserving high quality at the important image locations. This viewer relies on a collection of N SaccadeControllers to track the N hotspots; you can pick which type of controller to use and configure their parameters as usual, via the command line. Note that in programs like ezvision that use an StdBrain, the StdBrain also contains a SaccadeController; but here we will explicitly drop it, as we won;t be using Brain's controller, since we will decide on the N hotspots here based onthe raw saliency map. We will also drop Brain's ShapeEstimator and force Brain's IOR type to none, so that we have a clean saliency map here to work with.

Definition at line 74 of file SimulationViewerCompress.H.


Constructor & Destructor Documentation

SimulationViewerCompress::SimulationViewerCompress ( OptionManager mgr,
const std::string descrName = "Multi-Foveated Compression Simulation Viewer",
const std::string tagName = "SimulationViewerCompress" 
)
SimulationViewerCompress::~SimulationViewerCompress (  )  [virtual]

Destructor.

Definition at line 118 of file SimulationViewerCompress.C.


Member Function Documentation

Image< PixRGB< byte > > SimulationViewerCompress::getTraj ( SimEventQueue q  )  [protected]
void SimulationViewerCompress::paramChanged ( ModelParamBase *const   param,
const bool  valueChanged,
ParamClient::ChangeStatus status 
) [protected, virtual]

Intercept people changing our number of foveas.

Reimplemented from ModelComponent.

Definition at line 122 of file SimulationViewerCompress.C.

References itsFoveaSCtype, and itsNumFoveas.

SimulationViewerCompress::SIMCALLBACK_DECLARE ( SimulationViewerCompress  ,
SimEventSaveOutput   
) [protected]

Callback for every time we should save our outputs.

SimulationViewerCompress::SIMCALLBACK_DECLARE ( SimulationViewerCompress  ,
SimEventSaccadeStatusEye   
) [protected]

Callback for when the eye moves.

SimulationViewerCompress::SIMCALLBACK_DECLARE ( SimulationViewerCompress  ,
SimEventRetinaImage   
) [protected]

Callback for when a new retina image is available.

void SimulationViewerCompress::start1 (  )  [protected, virtual]

Get started and disable any SC the Brain may want to use.

Reimplemented from SimModule.

Definition at line 157 of file SimulationViewerCompress.C.

References OModelParam< T >::getVal(), itsCacheSize, itsOutFname, ModelComponent::numSubComp(), ImageCache< T >::setMaxSize(), and ModelComponent::subComponent().

void SimulationViewerCompress::stop1 (  )  [protected, virtual]

get stopped

Reimplemented from ModelComponent.

Definition at line 179 of file SimulationViewerCompress.C.


Member Data Documentation

size of our mask cache

Definition at line 120 of file SimulationViewerCompress.H.

Referenced by getTraj(), and start1().

patch color for human eye mvts

Definition at line 117 of file SimulationViewerCompress.H.

Referenced by getTraj().

patch color

Definition at line 116 of file SimulationViewerCompress.H.

Referenced by getTraj().

draw human eye movements

Definition at line 115 of file SimulationViewerCompress.H.

Referenced by getTraj().

draw objetc outlines

Definition at line 114 of file SimulationViewerCompress.H.

Referenced by getTraj().

draw patches

Definition at line 113 of file SimulationViewerCompress.H.

Referenced by getTraj().

distance factor to change the fovea size

Definition at line 111 of file SimulationViewerCompress.H.

do the eye compare with the mask

Definition at line 126 of file SimulationViewerCompress.H.

Referenced by SimulationViewerCompress().

FOA radius for object trackers.

Definition at line 105 of file SimulationViewerCompress.H.

type of SC for foveas

Definition at line 122 of file SimulationViewerCompress.H.

Referenced by paramChanged().

head radius

Definition at line 118 of file SimulationViewerCompress.H.

This parameter is the period (in frames) for foveation mask to change.

The idea here is to see whether keeping the foveation mask stable for predicted frames and changing it only for intra-coded frames may improve the compression ratio when using MPEG-1 compression. Have a look at http://www.disctronics.co.uk/technology/video/video_mpeg.htm if you are not familiar with those various types of frames. In particular (quoted from that web site):

"I-frames (Intra coded frames) use DCT encoding only to compress a single frame without reference to any other frame in the sequence. [...] P-frames (Predicted frames) are coded as differences from the last I or P frame. The new P-frame is first predicted by taking the last I or P frame and 'predicting' the values of each new pixel. P-frames use Motion Prediction and DCT encoding. As a result P-frames will give a compression ratio better than I-frames but depending on the amount of motion present. The differences between the predicted and actual values are encoded. [...] B-frames (Bidirectional frames) are coded as differences from the last or next I or P frame. B-frames use prediction as for P-frames but for each block either the previous I or P frame is used or the next I or P frame. [...]"

So, in our case, changing the foveation mask on a P or B frame may yield lots of prediction errors, that can be reduced if we force the mask to be only allowed to change on I frames. Here we assume that the first frame is an I-frame.

Definition at line 156 of file SimulationViewerCompress.H.

Referenced by getTraj().

our levelspec

Definition at line 124 of file SimulationViewerCompress.H.

Referenced by getTraj().

depth of blur pyramid

Definition at line 119 of file SimulationViewerCompress.H.

number of foveas

Definition at line 106 of file SimulationViewerCompress.H.

Referenced by paramChanged().

number of random samples

Definition at line 125 of file SimulationViewerCompress.H.

Referenced by getTraj().

Name of output file.

Definition at line 123 of file SimulationViewerCompress.H.

Referenced by start1().

save eye combo?

Definition at line 112 of file SimulationViewerCompress.H.

Referenced by getTraj().

save foveated image?

Definition at line 110 of file SimulationViewerCompress.H.

Referenced by getTraj().

save mask?

Definition at line 109 of file SimulationViewerCompress.H.

Referenced by getTraj().

save mega combo?

Definition at line 108 of file SimulationViewerCompress.H.

Referenced by getTraj().

save trajectory?

Definition at line 107 of file SimulationViewerCompress.H.

use TRM to take max in cache

Definition at line 121 of file SimulationViewerCompress.H.

Referenced by getTraj().


The documentation for this class was generated from the following files:
Generated on Sun May 8 08:43:49 2011 for iLab Neuromorphic Vision Toolkit by  doxygen 1.6.3