VisualObjectMatch Class Reference

Simple class to store lists of Keypoint matches between two VisualObject. More...

#include <SIFT/VisualObjectMatch.H>

Collaboration diagram for VisualObjectMatch:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 VisualObjectMatch (const rutz::shared_ptr< VisualObject > &voref, const rutz::shared_ptr< VisualObject > &votest, const VisualObjectMatchAlgo algo, const uint thresh=7U)
 Constructor from two visual objects.
 VisualObjectMatch (const rutz::shared_ptr< KDTree > &kdref, const rutz::shared_ptr< VisualObject > &votest, const VisualObjectMatchAlgo algo, const uint thresh=7U)
 Constructor from a pre-built KDTree and a VisualObject.
 VisualObjectMatch (const rutz::shared_ptr< VisualObject > &voref, const rutz::shared_ptr< VisualObject > &votest, const std::vector< KeypointMatch > &kpm)
 Build from a precomputed list of KeypointMatch matches.
 ~VisualObjectMatch ()
 Destructor.
void push_back (const KeypointMatch &km)
 Add a Keypoint match to the back of our internal list.
const KeypointMatchoperator[] (const uint idx) const
 Access a KeypointMatch, read-only version.
KeypointMatchoperator[] (const uint idx)
 Access a KeypointMatch, read/write version.
const KeypointMatchgetKeypointMatch (const uint index) const
 get a given KeypointMatch
uint size () const
 Get number of Keypoint matches we currently have.
const rutz::shared_ptr
< VisualObject > & 
getVoRef () const
 Get the reference VisualObject.
const rutz::shared_ptr
< VisualObject > & 
getVoTest () const
 Get the test VisualObject.
uint prune (const uint maxn=25U, const uint minn=3U)
 Apply a standard series of prunings.
uint pruneByDist (const uint thresh=7U, const uint minn=3U)
 Prune our matches by ratio of best to second best distance.
uint pruneByHough (const float rangefac=0.25F, const uint minn=3U)
 Prune the matches using a Hough transform.
uint pruneByAff (const float dist=5.0F, const uint minn=3U)
 Prune our matches by consistency with affine transform.
SIFTaffine getSIFTaffine ()
 Compute least-squares affine transform between matches.
bool checkSIFTaffine (const float maxrot=10.0F, const float maxscale=30.0F, const float maxshear=10.0F)
 Check the SIFT affine for weirdness.
float getScore (const float kcoeff=0.5F, const float acoeff=0.5F)
 Get a match score, higher scores are better.
float getSalScore (const float wcoeff=0.0F, const float hcoeff=0.0F)
float getSalDist ()
 spatial distance of the salient points
float getSalDiff ()
 salient feature vector difference
float getKeypointAvgDist ()
 Compute a matching score based on average residual distsq btw keys.
float getAffineAvgDist ()
 Compute matching score based on average residual distance between keys.
Image< PixRGB< byte > > getMatchImage (const float scale=1.0F) const
 Get a combo image with SIFT Keypoint matches.
Image< PixRGB< byte > > getMatchImage (Dims frameSize, Point2D< int > refOffset, Point2D< int > testOffset, const float scale=1.0F) const
Image< PixRGB< byte > > getTransfTestImage (const Image< PixRGB< byte > > &im=Image< PixRGB< byte > >())
 Get the image of the test object transformed to match the ref object.
void getTransfTestOutline (Point2D< int > &tl, Point2D< int > &tr, Point2D< int > &br, Point2D< int > &bl)
 Get the transformed coords of the 4 corners of the test image.
Image< PixRGB< byte > > getFusedImage (const float mix=0.5F)
 Get the image of the test object fused on top of that of the ref obj.
const std::vector
< KeypointMatch > & 
getKeypointMatches () const
 Get our list of Keypoint matches.
Point2D< int > getSpatialDist (Point2D< int > objOffset1=Point2D< int >(0, 0), Point2D< int > objOffset2=Point2D< int >(0, 0))
 get the spatial distance between the two objects
Rectangle getOverlapRect ()
 get the overlap rectangle of the match
bool isOverlapping ()
 check overlap using the image rectangle overlap
bool isOverlapping2 ()
 check overlap using keypoint bounding box overlap

Detailed Description

Simple class to store lists of Keypoint matches between two VisualObject.

Definition at line 52 of file VisualObjectMatch.H.


Constructor & Destructor Documentation

VisualObjectMatch::VisualObjectMatch ( const rutz::shared_ptr< VisualObject > &  voref,
const rutz::shared_ptr< VisualObject > &  votest,
const VisualObjectMatchAlgo  algo,
const uint  thresh = 7U 
)

Constructor from two visual objects.

If using a KDTree matching algo, the KDTree will be built in 'voref' (with caching). So voref should usually be the object that has more keypoints than votest.

Definition at line 47 of file VisualObjectMatch.C.

VisualObjectMatch::VisualObjectMatch ( const rutz::shared_ptr< KDTree > &  kdref,
const rutz::shared_ptr< VisualObject > &  votest,
const VisualObjectMatchAlgo  algo,
const uint  thresh = 7U 
)

Constructor from a pre-built KDTree and a VisualObject.

The KDTree should have been filled already and is considered the 'reference' object. Matching algo must be either VOMA_KDTREE or VOMA_KDTREEBBF.

Definition at line 68 of file VisualObjectMatch.C.

VisualObjectMatch::VisualObjectMatch ( const rutz::shared_ptr< VisualObject > &  voref,
const rutz::shared_ptr< VisualObject > &  votest,
const std::vector< KeypointMatch > &  kpm 
)

Build from a precomputed list of KeypointMatch matches.

Normally you would not want to call this unless you got your KeypointMatch matches from somewhere. This is used by VisualObjectDB during KDTree-based matching.

Definition at line 90 of file VisualObjectMatch.C.

VisualObjectMatch::~VisualObjectMatch (  ) 

Destructor.

Definition at line 103 of file VisualObjectMatch.C.


Member Function Documentation

bool VisualObjectMatch::checkSIFTaffine ( const float  maxrot = 10.0F,
const float  maxscale = 30.0F,
const float  maxshear = 10.0F 
)

Check the SIFT affine for weirdness.

This returns false if our affine represents too gross of a distortion between the two matched objects.

Parameters:
maxrot max allwoed rotation; valid values are in [0..pi]; default is to allow any rotation.
maxscale max allowed scaling; valid values are > 0.0; default is to allow scaling from 1:10 to 10:1.
maxshear max allowed shearing; valid values are >= 0.0; default is to allow shearing between -0.5 and 0.5.

Definition at line 352 of file VisualObjectMatch.C.

References SIFTaffine::decompose(), and SIFTaffine::isInversible().

Referenced by BeoSub::matchSIFT().

float VisualObjectMatch::getAffineAvgDist (  ) 

Compute matching score based on average residual distance between keys.

Normally you would just use getScore() but this is made public for people who want finer control. You should call this after you have pruned. We here just get the SIFTaffine and compute the average residual distance between each ref keypoint transformed by the affine and the corresponding test keypoint. Units hence are pixels in the test image.

Definition at line 514 of file VisualObjectMatch.C.

References SIFTaffine::getResidualDistSq().

Referenced by getScore().

Image< PixRGB< byte > > VisualObjectMatch::getFusedImage ( const float  mix = 0.5F  ) 

Get the image of the test object fused on top of that of the ref obj.

The resulting image has the size of the ref object's. A mixing factor of 1.0 means that the ref image gets a coeff 1.0 and the test image 0.0. To get the affine transform between the two images, getSIFTaffine will be called, without any pruning. So you may want to prune and clean the matches before you get that final display.

Definition at line 791 of file VisualObjectMatch.C.

References Image< T >::begin(), Image< T >::beginw(), Image< T >::coordsOk(), Image< T >::getHeight(), getSIFTaffine(), Image< T >::getValInterp(), Image< T >::getWidth(), NO_INIT, and SIFTaffine::transform().

float VisualObjectMatch::getKeypointAvgDist (  ) 

Compute a matching score based on average residual distsq btw keys.

Normally you would just use getScore() but this is made public for people who want finer control. The distance here is scaled so as to become comparable to that of getKeypointAvgDist().

Definition at line 492 of file VisualObjectMatch.C.

Referenced by getScore().

const KeypointMatch & VisualObjectMatch::getKeypointMatch ( const uint  index  )  const [inline]

get a given KeypointMatch

Definition at line 327 of file VisualObjectMatch.H.

References ASSERT.

const std::vector< KeypointMatch > & VisualObjectMatch::getKeypointMatches (  )  const [inline]

Get our list of Keypoint matches.

Definition at line 363 of file VisualObjectMatch.H.

Referenced by getMatchImage().

Image< PixRGB< byte > > VisualObjectMatch::getMatchImage ( Dims  frameSize,
Point2D< int >  refOffset,
Point2D< int >  testOffset,
const float  scale = 1.0F 
) const

Get a combo image with SIFT Keypoint matches this one has a frame around it so that different size images can be reconciled and the offset can also be added

Definition at line 683 of file VisualObjectMatch.C.

References drawDisk(), getKeypointMatches(), getVoRef(), getVoTest(), Dims::h(), inplacePaste(), Dims::w(), and ZEROS.

Image< PixRGB< byte > > VisualObjectMatch::getMatchImage ( const float  scale = 1.0F  )  const

Get a combo image with SIFT Keypoint matches.

Definition at line 630 of file VisualObjectMatch.C.

References Image< T >::coordsOk(), drawDisk(), Image< T >::getHeight(), Image< T >::getWidth(), inplacePaste(), max(), and ZEROS.

Rectangle VisualObjectMatch::getOverlapRect (  ) 
float VisualObjectMatch::getSalDiff (  ) 

salient feature vector difference

Definition at line 448 of file VisualObjectMatch.C.

References getVoRef(), and getVoTest().

Referenced by getSalScore().

float VisualObjectMatch::getSalDist (  ) 

spatial distance of the salient points

Definition at line 472 of file VisualObjectMatch.C.

References Point2D< T >::distance(), getSIFTaffine(), getVoRef(), getVoTest(), Point2D< T >::i, and SIFTaffine::transform().

Referenced by getSalScore().

float VisualObjectMatch::getSalScore ( const float  wcoeff = 0.0F,
const float  hcoeff = 0.0F 
)

get a match score based on the salient feature difference * spatial distance

Definition at line 412 of file VisualObjectMatch.C.

References getSalDiff(), getSalDist(), getVoRef(), getVoTest(), and sqrt().

float VisualObjectMatch::getScore ( const float  kcoeff = 0.5F,
const float  acoeff = 0.5F 
)

Get a match score, higher scores are better.

Score returned here is defined as kcoeff / (1 + getKeypointAvgDist()) + acoeff / (1 + getAffineAvgDist()) + 0.05 * numKeypointMatches

Definition at line 382 of file VisualObjectMatch.C.

References getAffineAvgDist(), and getKeypointAvgDist().

Referenced by BeoSub::matchSIFT().

SIFTaffine VisualObjectMatch::getSIFTaffine (  )  [inline]

Compute least-squares affine transform between matches.

If we have not already done so, we will compute the affine from all our Keypoint matches. Hence it is recommended that you call prune() prior to invoking this member function.

Definition at line 341 of file VisualObjectMatch.H.

Referenced by getFusedImage(), getOverlapRect(), getSalDist(), getTransfTestImage(), getTransfTestOutline(), main(), and BeoMap::makePanorama().

Point2D< int > VisualObjectMatch::getSpatialDist ( Point2D< int >  objOffset1 = Point2D<int>(0,0),
Point2D< int >  objOffset2 = Point2D<int>(0,0) 
)

get the spatial distance between the two objects

the offsets passed in are the the coordinates of the top left corner of each image. The result is the distance of the two respective origins, which is also the coordinate change of the two objects. This is useful for visual objects that come from a cropped frame. The result can be the camera movement or egomotion

Definition at line 827 of file VisualObjectMatch.C.

References Image< T >::getVal(), Point2D< T >::i, matrixMult(), Image< T >::setVal(), SIFTaffine::ty, and ZEROS.

Image< PixRGB< byte > > VisualObjectMatch::getTransfTestImage ( const Image< PixRGB< byte > > &  im = ImagePixRGB<byte> >()  ) 

Get the image of the test object transformed to match the ref object.

If an uninitialized image is given, the resulting image has the size of the ref object's, and contains zeros everywhere except where the test object is. Otherwise, the test object is just painted into the given image (which must have the dims of the ref image.

Definition at line 734 of file VisualObjectMatch.C.

References Image< T >::beginw(), Image< T >::coordsOk(), Image< T >::getHeight(), getSIFTaffine(), Image< T >::getValInterp(), Image< T >::getWidth(), Image< T >::initialized(), Image< T >::resize(), and SIFTaffine::transform().

void VisualObjectMatch::getTransfTestOutline ( Point2D< int > &  tl,
Point2D< int > &  tr,
Point2D< int > &  br,
Point2D< int > &  bl 
)

Get the transformed coords of the 4 corners of the test image.

The returned points have coords in the coord system of the ref image, but may fall outside that image.

Definition at line 765 of file VisualObjectMatch.C.

References getSIFTaffine(), Dims::h(), Point2D< T >::i, SIFTaffine::inverse(), SIFTaffine::transform(), and Dims::w().

const rutz::shared_ptr< VisualObject > & VisualObjectMatch::getVoRef (  )  const [inline]
const rutz::shared_ptr< VisualObject > & VisualObjectMatch::getVoTest (  )  const [inline]
bool VisualObjectMatch::isOverlapping (  ) 
bool VisualObjectMatch::isOverlapping2 (  ) 

check overlap using keypoint bounding box overlap

Definition at line 949 of file VisualObjectMatch.C.

References getVoRef(), getVoTest(), and size().

KeypointMatch & VisualObjectMatch::operator[] ( const uint  idx  )  [inline]

Access a KeypointMatch, read/write version.

Definition at line 323 of file VisualObjectMatch.H.

const KeypointMatch & VisualObjectMatch::operator[] ( const uint  idx  )  const [inline]

Access a KeypointMatch, read-only version.

Definition at line 319 of file VisualObjectMatch.H.

uint VisualObjectMatch::prune ( const uint  maxn = 25U,
const uint  minn = 3U 
)

Apply a standard series of prunings.

This is a heuristic combination of calls to pruneByDist(), pruneByHough(), and pruneByAff(), so as to prune outlier matches and allow the recovery of a clean affine transform through getSIFTaffine(). Do not prune to fewer than 'minn' matches but try to prune down to fewer than 'maxn' matches (the latter is not guaranteed, as all matches may be very very good). Returns the number of outlier matches pruned away.

Definition at line 107 of file VisualObjectMatch.C.

References pruneByAff(), pruneByDist(), and pruneByHough().

Referenced by main(), BeoMap::makePanorama(), and BeoSub::matchSIFT().

uint VisualObjectMatch::pruneByAff ( const float  dist = 5.0F,
const uint  minn = 3U 
)

Prune our matches by consistency with affine transform.

Normally you should just use prune() but this is made public for people who want finer control. Compute the affine transform from the matches, and eliminate the matches that disagree with it in that the distance in the test image between an affine-transformed reference keypoint and the matching test keypoint is larger than 'dist'. During this process, we will not continue if fewer than 'minn' matches remain. The number of outlier matches that were deleted is returned.

Definition at line 266 of file VisualObjectMatch.C.

References SIFTaffine::getResidualDistSq().

Referenced by prune().

uint VisualObjectMatch::pruneByDist ( const uint  thresh = 7U,
const uint  minn = 3U 
)

Prune our matches by ratio of best to second best distance.

Normally you should just use prune() but this is made public for people who want finer control. The given thresh is in units of 10%. For example, if thresh=8, matches where the match distance is more than 0.8 the second best distance will be eliminated. Returns the number of matches pruned. Do not prune to fewer than 'minn' matches.

Definition at line 140 of file VisualObjectMatch.C.

Referenced by prune().

uint VisualObjectMatch::pruneByHough ( const float  rangefac = 0.25F,
const uint  minn = 3U 
)

Prune the matches using a Hough transform.

Normally you should just use prune() but this is made public for people who want finer control. Returns the number of matches pruned. Do not prune to fewer than 'minn' matches. Prune matches that disagree with the most popular transform by more than 'rangefac' times the range in any dimension; sensible values are between 0.05 (extremely strict; not recommended since the family of geometric transformations used here is very approximative) and 0.5 (may not prune anything).

Definition at line 161 of file VisualObjectMatch.C.

References SIFThough::addValue(), and SIFThough::getPeak().

Referenced by prune().

void VisualObjectMatch::push_back ( const KeypointMatch km  )  [inline]

Add a Keypoint match to the back of our internal list.

Definition at line 315 of file VisualObjectMatch.H.

uint VisualObjectMatch::size (  )  const [inline]

Get number of Keypoint matches we currently have.

Definition at line 312 of file VisualObjectMatch.H.

Referenced by isOverlapping2(), main(), and BeoMap::makePanorama().


The documentation for this class was generated from the following files:
Generated on Sun May 8 08:44:03 2011 for iLab Neuromorphic Vision Toolkit by  doxygen 1.6.3