SimulationViewerCompress Class Reference

Do multi-foveated saliency-based image compression. More...

#include <Neuro/SimulationViewerCompress.H>

Inheritance diagram for SimulationViewerCompress:

[legend]

Collaboration diagram for SimulationViewerCompress:

[legend]

List of all members.

Public Member Functions
Constructors and destructors

	SimulationViewerCompress (OptionManager &mgr, const std::string &descrName="Multi-Foveated Compression Simulation Viewer", const std::string &tagName="SimulationViewerCompress")
	Constructor. See ModelComponent.H.
virtual	~SimulationViewerCompress ()
	Destructor.
Protected Member Functions
	SIMCALLBACK_DECLARE (SimulationViewerCompress, SimEventRetinaImage)
	Callback for when a new retina image is available.
	SIMCALLBACK_DECLARE (SimulationViewerCompress, SimEventSaccadeStatusEye)
	Callback for when the eye moves.
	SIMCALLBACK_DECLARE (SimulationViewerCompress, SimEventSaveOutput)
	Callback for every time we should save our outputs.
Image< PixRGB< byte > >	getTraj (SimEventQueue &q)
	Get the attention/eye/head trajectory image.
virtual void	start1 ()
	Get started and disable any SC the Brain may want to use.
virtual void	stop1 ()
	get stopped
virtual void	paramChanged (ModelParamBase const param, const bool valueChanged, ParamClient::ChangeStatus status)
	Intercept people changing our number of foveas.
Protected Attributes
OModelParam< int >	itsFOAradius
	FOA radius for object trackers.
OModelParam< int >	itsNumFoveas
	number of foveas
OModelParam< bool >	itsSaveTraj
	save trajectory?
OModelParam< bool >	itsSaveMegaCombo
	save mega combo?
OModelParam< bool >	itsSaveMask
	save mask?
OModelParam< bool >	itsSaveFoveatedImage
	save foveated image?
OModelParam< float >	itsDistanceFactor
	distance factor to change the fovea size
OModelParam< bool >	itsSaveEyeCombo
	save eye combo?
OModelParam< bool >	itsDisplayPatch
	draw patches
OModelParam< bool >	itsDisplayFOA
	draw objetc outlines
OModelParam< bool >	itsDisplayEye
	draw human eye movements
NModelParam< PixRGB< byte > >	itsColorNormal
	patch color
NModelParam< PixRGB< byte > >	itsColorEye
	patch color for human eye mvts
OModelParam< int >	itsHeadRadius
	head radius
OModelParam< int >	itsMultiRetinaDepth
	depth of blur pyramid
OModelParam< int >	itsCacheSize
	size of our mask cache
OModelParam< bool >	itsUseTRMmax
	use TRM to take max in cache
OModelParam< std::string >	itsFoveaSCtype
	type of SC for foveas
OModelParam< std::string >	itsOutFname
	Name of output file.
OModelParam< LevelSpec >	itsLevelSpec
	our levelspec
OModelParam< int >	itsNumRandomSamples
	number of random samples
OModelParam< bool >	itsEyeCompare
	do the eye compare with the mask
OModelParam< int >	itsIFramePeriod
	This parameter is the period (in frames) for foveation mask to change.

Detailed Description

Do multi-foveated saliency-based image compression.

This viewer will consider the top N salient locations and apply blurring to the image that increases in strength (sigma) as we get farther away from any of the top N most salient locations. The resulting trajectory image hence will be crisp at the top N locations and increasingly blurred away from those. Compressing the output of this viewer, e.g., using MPEG, yields smaller file size while (hopefully) preserving high quality at the important image locations. This viewer relies on a collection of N SaccadeControllers to track the N hotspots; you can pick which type of controller to use and configure their parameters as usual, via the command line. Note that in programs like ezvision that use an StdBrain, the StdBrain also contains a SaccadeController; but here we will explicitly drop it, as we won;t be using Brain's controller, since we will decide on the N hotspots here based onthe raw saliency map. We will also drop Brain's ShapeEstimator and force Brain's IOR type to none, so that we have a clean saliency map here to work with.

Definition at line 74 of file SimulationViewerCompress.H.

Constructor & Destructor Documentation

SimulationViewerCompress::SimulationViewerCompress	(	OptionManager &	mgr,
		const std::string &	descrName = `"Multi-Foveated Compression Simulation Viewer"`,
		const std::string &	tagName = `"SimulationViewerCompress"`
	)

Constructor. See ModelComponent.H.

Definition at line 74 of file SimulationViewerCompress.C.

References ModelComponent::getManager(), OModelParam< T >::getVal(), itsEyeCompare, and OptionManager::setOptionValString().

SimulationViewerCompress::~SimulationViewerCompress ( ) [virtual]

Destructor.

Definition at line 118 of file SimulationViewerCompress.C.

Member Function Documentation

Image< PixRGB< byte > > SimulationViewerCompress::getTraj ( SimEventQueue & q ) [protected]

Get the attention/eye/head trajectory image.

Definition at line 207 of file SimulationViewerCompress.C.

References binaryReverse(), Rectangle::bottomO(), SimEventQueue::check(), Point2D< T >::clampToDims(), Image< T >::clear(), colGreyCombo(), concatX(), contour2D(), decXY(), drawDisk(), drawPatchBB(), eye(), Image< T >::getBounds(), Image< T >::getDims(), Image< T >::getHeight(), SimulationViewer::getMap(), getMinMaxAvg(), Rectangle::getOverlap(), Image< T >::getVal(), OModelParam< T >::getVal(), NModelParam< T >::getVal(), Image< T >::getWidth(), Dims::h(), Point2D< T >::i, Image< T >::initialized(), inplaceLowThresh(), inplacePaste(), itsCacheSize, itsColorEye, itsColorNormal, itsDisplayEye, itsDisplayFOA, SimulationViewer::itsDisplayInterp, itsDisplayPatch, itsIFramePeriod, itsLevelSpec, itsNumRandomSamples, itsSaveEyeCombo, itsSaveFoveatedImage, itsSaveMask, itsSaveMegaCombo, itsUseTRMmax, Rectangle::left(), lowPass3(), LevelSpec::mapLevel(), ImageCacheAvg< T >::mean(), NO_INIT, ImageCache< T >::push_back(), randomUpToNotIncluding(), SimEventQueue::request(), Image< T >::resize(), Rectangle::rightO(), SEQ_ANY, takeMin(), thresholdedMix(), Rectangle::tlbrI(), Rectangle::top(), toRGB(), Dims::w(), and weightedBlur().

void SimulationViewerCompress::paramChanged	(	ModelParamBase *const	param,
		const bool	valueChanged,
		ParamClient::ChangeStatus *	status
	)			`[protected, virtual]`

Intercept people changing our number of foveas.

Reimplemented from ModelComponent.

Definition at line 122 of file SimulationViewerCompress.C.

References itsFoveaSCtype, and itsNumFoveas.

SimulationViewerCompress::SIMCALLBACK_DECLARE	(	SimulationViewerCompress	,
		SimEventSaveOutput
	)			`[protected]`

Callback for every time we should save our outputs.

SimulationViewerCompress::SIMCALLBACK_DECLARE	(	SimulationViewerCompress	,
		SimEventSaccadeStatusEye
	)			`[protected]`

Callback for when the eye moves.

SimulationViewerCompress::SIMCALLBACK_DECLARE	(	SimulationViewerCompress	,
		SimEventRetinaImage
	)			`[protected]`

Callback for when a new retina image is available.

void SimulationViewerCompress::start1 ( ) [protected, virtual]

Get started and disable any SC the Brain may want to use.

Reimplemented from SimModule.

Definition at line 157 of file SimulationViewerCompress.C.

References OModelParam< T >::getVal(), itsCacheSize, itsOutFname, ModelComponent::numSubComp(), ImageCache< T >::setMaxSize(), and ModelComponent::subComponent().

void SimulationViewerCompress::stop1 ( ) [protected, virtual]

get stopped

Reimplemented from ModelComponent.

Definition at line 179 of file SimulationViewerCompress.C.

Member Data Documentation

OModelParam<int> SimulationViewerCompress::itsCacheSize [protected]

size of our mask cache

Definition at line 120 of file SimulationViewerCompress.H.

Referenced by getTraj(), and start1().

NModelParam< PixRGB<byte> > SimulationViewerCompress::itsColorEye [protected]

patch color for human eye mvts

Definition at line 117 of file SimulationViewerCompress.H.

Referenced by getTraj().

NModelParam< PixRGB<byte> > SimulationViewerCompress::itsColorNormal [protected]

patch color

Definition at line 116 of file SimulationViewerCompress.H.

Referenced by getTraj().

OModelParam<bool> SimulationViewerCompress::itsDisplayEye [protected]

draw human eye movements

Definition at line 115 of file SimulationViewerCompress.H.

Referenced by getTraj().

OModelParam<bool> SimulationViewerCompress::itsDisplayFOA [protected]

draw objetc outlines

Definition at line 114 of file SimulationViewerCompress.H.

Referenced by getTraj().

OModelParam<bool> SimulationViewerCompress::itsDisplayPatch [protected]

draw patches

Definition at line 113 of file SimulationViewerCompress.H.

Referenced by getTraj().

OModelParam<float> SimulationViewerCompress::itsDistanceFactor [protected]

distance factor to change the fovea size

Definition at line 111 of file SimulationViewerCompress.H.

OModelParam<bool> SimulationViewerCompress::itsEyeCompare [protected]

do the eye compare with the mask

Definition at line 126 of file SimulationViewerCompress.H.

Referenced by SimulationViewerCompress().

OModelParam<int> SimulationViewerCompress::itsFOAradius [protected]

FOA radius for object trackers.

Definition at line 105 of file SimulationViewerCompress.H.

OModelParam<std::string> SimulationViewerCompress::itsFoveaSCtype [protected]

type of SC for foveas

Definition at line 122 of file SimulationViewerCompress.H.

Referenced by paramChanged().

OModelParam<int> SimulationViewerCompress::itsHeadRadius [protected]

head radius

Definition at line 118 of file SimulationViewerCompress.H.

OModelParam<int> SimulationViewerCompress::itsIFramePeriod [protected]

This parameter is the period (in frames) for foveation mask to change.

The idea here is to see whether keeping the foveation mask stable for predicted frames and changing it only for intra-coded frames may improve the compression ratio when using MPEG-1 compression. Have a look at http://www.disctronics.co.uk/technology/video/video_mpeg.htm if you are not familiar with those various types of frames. In particular (quoted from that web site):

"I-frames (Intra coded frames) use DCT encoding only to compress a single frame without reference to any other frame in the sequence. [...] P-frames (Predicted frames) are coded as differences from the last I or P frame. The new P-frame is first predicted by taking the last I or P frame and 'predicting' the values of each new pixel. P-frames use Motion Prediction and DCT encoding. As a result P-frames will give a compression ratio better than I-frames but depending on the amount of motion present. The differences between the predicted and actual values are encoded. [...] B-frames (Bidirectional frames) are coded as differences from the last or next I or P frame. B-frames use prediction as for P-frames but for each block either the previous I or P frame is used or the next I or P frame. [...]"

So, in our case, changing the foveation mask on a P or B frame may yield lots of prediction errors, that can be reduced if we force the mask to be only allowed to change on I frames. Here we assume that the first frame is an I-frame.

Definition at line 156 of file SimulationViewerCompress.H.

Referenced by getTraj().

OModelParam<LevelSpec> SimulationViewerCompress::itsLevelSpec [protected]

our levelspec

Definition at line 124 of file SimulationViewerCompress.H.

Referenced by getTraj().

OModelParam<int> SimulationViewerCompress::itsMultiRetinaDepth [protected]

depth of blur pyramid

Definition at line 119 of file SimulationViewerCompress.H.

OModelParam<int> SimulationViewerCompress::itsNumFoveas [protected]

number of foveas

Definition at line 106 of file SimulationViewerCompress.H.