Simple class to store lists of Keypoint matches between two VisualObject. More...
#include <SIFT/VisualObjectMatch.H>
Public Member Functions | |
| VisualObjectMatch (const rutz::shared_ptr< VisualObject > &voref, const rutz::shared_ptr< VisualObject > &votest, const VisualObjectMatchAlgo algo, const uint thresh=7U) | |
| Constructor from two visual objects. | |
| VisualObjectMatch (const rutz::shared_ptr< KDTree > &kdref, const rutz::shared_ptr< VisualObject > &votest, const VisualObjectMatchAlgo algo, const uint thresh=7U) | |
| Constructor from a pre-built KDTree and a VisualObject. | |
| VisualObjectMatch (const rutz::shared_ptr< VisualObject > &voref, const rutz::shared_ptr< VisualObject > &votest, const std::vector< KeypointMatch > &kpm) | |
| Build from a precomputed list of KeypointMatch matches. | |
| ~VisualObjectMatch () | |
| Destructor. | |
| void | push_back (const KeypointMatch &km) |
| Add a Keypoint match to the back of our internal list. | |
| const KeypointMatch & | operator[] (const uint idx) const |
| Access a KeypointMatch, read-only version. | |
| KeypointMatch & | operator[] (const uint idx) |
| Access a KeypointMatch, read/write version. | |
| const KeypointMatch & | getKeypointMatch (const uint index) const |
| get a given KeypointMatch | |
| uint | size () const |
| Get number of Keypoint matches we currently have. | |
| const rutz::shared_ptr < VisualObject > & | getVoRef () const |
| Get the reference VisualObject. | |
| const rutz::shared_ptr < VisualObject > & | getVoTest () const |
| Get the test VisualObject. | |
| uint | prune (const uint maxn=25U, const uint minn=3U) |
| Apply a standard series of prunings. | |
| uint | pruneByDist (const uint thresh=7U, const uint minn=3U) |
| Prune our matches by ratio of best to second best distance. | |
| uint | pruneByHough (const float rangefac=0.25F, const uint minn=3U) |
| Prune the matches using a Hough transform. | |
| uint | pruneByAff (const float dist=5.0F, const uint minn=3U) |
| Prune our matches by consistency with affine transform. | |
| SIFTaffine | getSIFTaffine () |
| Compute least-squares affine transform between matches. | |
| bool | checkSIFTaffine (const float maxrot=10.0F, const float maxscale=30.0F, const float maxshear=10.0F) |
| Check the SIFT affine for weirdness. | |
| float | getScore (const float kcoeff=0.5F, const float acoeff=0.5F) |
| Get a match score, higher scores are better. | |
| float | getSalScore (const float wcoeff=0.0F, const float hcoeff=0.0F) |
| float | getSalDist () |
| spatial distance of the salient points | |
| float | getSalDiff () |
| salient feature vector difference | |
| float | getKeypointAvgDist () |
| Compute a matching score based on average residual distsq btw keys. | |
| float | getAffineAvgDist () |
| Compute matching score based on average residual distance between keys. | |
| Image< PixRGB< byte > > | getMatchImage (const float scale=1.0F) const |
| Get a combo image with SIFT Keypoint matches. | |
| Image< PixRGB< byte > > | getMatchImage (Dims frameSize, Point2D< int > refOffset, Point2D< int > testOffset, const float scale=1.0F) const |
| Image< PixRGB< byte > > | getTransfTestImage (const Image< PixRGB< byte > > &im=Image< PixRGB< byte > >()) |
| Get the image of the test object transformed to match the ref object. | |
| void | getTransfTestOutline (Point2D< int > &tl, Point2D< int > &tr, Point2D< int > &br, Point2D< int > &bl) |
| Get the transformed coords of the 4 corners of the test image. | |
| Image< PixRGB< byte > > | getFusedImage (const float mix=0.5F) |
| Get the image of the test object fused on top of that of the ref obj. | |
| const std::vector < KeypointMatch > & | getKeypointMatches () const |
| Get our list of Keypoint matches. | |
| Point2D< int > | getSpatialDist (Point2D< int > objOffset1=Point2D< int >(0, 0), Point2D< int > objOffset2=Point2D< int >(0, 0)) |
| get the spatial distance between the two objects | |
| Rectangle | getOverlapRect () |
| get the overlap rectangle of the match | |
| bool | isOverlapping () |
| check overlap using the image rectangle overlap | |
| bool | isOverlapping2 () |
| check overlap using keypoint bounding box overlap | |
Simple class to store lists of Keypoint matches between two VisualObject.
Definition at line 52 of file VisualObjectMatch.H.
| VisualObjectMatch::VisualObjectMatch | ( | const rutz::shared_ptr< VisualObject > & | voref, | |
| const rutz::shared_ptr< VisualObject > & | votest, | |||
| const VisualObjectMatchAlgo | algo, | |||
| const uint | thresh = 7U | |||
| ) |
Constructor from two visual objects.
If using a KDTree matching algo, the KDTree will be built in 'voref' (with caching). So voref should usually be the object that has more keypoints than votest.
Definition at line 47 of file VisualObjectMatch.C.
| VisualObjectMatch::VisualObjectMatch | ( | const rutz::shared_ptr< KDTree > & | kdref, | |
| const rutz::shared_ptr< VisualObject > & | votest, | |||
| const VisualObjectMatchAlgo | algo, | |||
| const uint | thresh = 7U | |||
| ) |
Constructor from a pre-built KDTree and a VisualObject.
The KDTree should have been filled already and is considered the 'reference' object. Matching algo must be either VOMA_KDTREE or VOMA_KDTREEBBF.
Definition at line 68 of file VisualObjectMatch.C.
| VisualObjectMatch::VisualObjectMatch | ( | const rutz::shared_ptr< VisualObject > & | voref, | |
| const rutz::shared_ptr< VisualObject > & | votest, | |||
| const std::vector< KeypointMatch > & | kpm | |||
| ) |
Build from a precomputed list of KeypointMatch matches.
Normally you would not want to call this unless you got your KeypointMatch matches from somewhere. This is used by VisualObjectDB during KDTree-based matching.
Definition at line 90 of file VisualObjectMatch.C.
| VisualObjectMatch::~VisualObjectMatch | ( | ) |
Destructor.
Definition at line 103 of file VisualObjectMatch.C.
| bool VisualObjectMatch::checkSIFTaffine | ( | const float | maxrot = 10.0F, |
|
| const float | maxscale = 30.0F, |
|||
| const float | maxshear = 10.0F | |||
| ) |
Check the SIFT affine for weirdness.
This returns false if our affine represents too gross of a distortion between the two matched objects.
| maxrot | max allwoed rotation; valid values are in [0..pi]; default is to allow any rotation. | |
| maxscale | max allowed scaling; valid values are > 0.0; default is to allow scaling from 1:10 to 10:1. | |
| maxshear | max allowed shearing; valid values are >= 0.0; default is to allow shearing between -0.5 and 0.5. |
Definition at line 352 of file VisualObjectMatch.C.
References SIFTaffine::decompose(), and SIFTaffine::isInversible().
Referenced by BeoSub::matchSIFT().
| float VisualObjectMatch::getAffineAvgDist | ( | ) |
Compute matching score based on average residual distance between keys.
Normally you would just use getScore() but this is made public for people who want finer control. You should call this after you have pruned. We here just get the SIFTaffine and compute the average residual distance between each ref keypoint transformed by the affine and the corresponding test keypoint. Units hence are pixels in the test image.
Definition at line 514 of file VisualObjectMatch.C.
References SIFTaffine::getResidualDistSq().
Referenced by getScore().
Get the image of the test object fused on top of that of the ref obj.
The resulting image has the size of the ref object's. A mixing factor of 1.0 means that the ref image gets a coeff 1.0 and the test image 0.0. To get the affine transform between the two images, getSIFTaffine will be called, without any pruning. So you may want to prune and clean the matches before you get that final display.
Definition at line 791 of file VisualObjectMatch.C.
References Image< T >::begin(), Image< T >::beginw(), Image< T >::coordsOk(), Image< T >::getHeight(), getSIFTaffine(), Image< T >::getValInterp(), Image< T >::getWidth(), NO_INIT, and SIFTaffine::transform().
| float VisualObjectMatch::getKeypointAvgDist | ( | ) |
Compute a matching score based on average residual distsq btw keys.
Normally you would just use getScore() but this is made public for people who want finer control. The distance here is scaled so as to become comparable to that of getKeypointAvgDist().
Definition at line 492 of file VisualObjectMatch.C.
Referenced by getScore().
| const KeypointMatch & VisualObjectMatch::getKeypointMatch | ( | const uint | index | ) | const [inline] |
| const std::vector< KeypointMatch > & VisualObjectMatch::getKeypointMatches | ( | ) | const [inline] |
Get our list of Keypoint matches.
Definition at line 363 of file VisualObjectMatch.H.
Referenced by getMatchImage().
| Image< PixRGB< byte > > VisualObjectMatch::getMatchImage | ( | Dims | frameSize, | |
| Point2D< int > | refOffset, | |||
| Point2D< int > | testOffset, | |||
| const float | scale = 1.0F | |||
| ) | const |
Get a combo image with SIFT Keypoint matches this one has a frame around it so that different size images can be reconciled and the offset can also be added
Definition at line 683 of file VisualObjectMatch.C.
References drawDisk(), getKeypointMatches(), getVoRef(), getVoTest(), Dims::h(), inplacePaste(), Dims::w(), and ZEROS.
Get a combo image with SIFT Keypoint matches.
Definition at line 630 of file VisualObjectMatch.C.
References Image< T >::coordsOk(), drawDisk(), Image< T >::getHeight(), Image< T >::getWidth(), inplacePaste(), max(), and ZEROS.
| Rectangle VisualObjectMatch::getOverlapRect | ( | ) |
get the overlap rectangle of the match
Definition at line 864 of file VisualObjectMatch.C.
References Rectangle::bottomI(), Image< T >::getBounds(), Image< T >::getHeight(), Rectangle::getOverlap(), getSIFTaffine(), getVoRef(), getVoTest(), Image< T >::getWidth(), Rectangle::height(), Rectangle::left(), Rectangle::rightI(), Rectangle::tlbrI(), Rectangle::top(), SIFTaffine::transform(), and Rectangle::width().
Referenced by isOverlapping().
| float VisualObjectMatch::getSalDiff | ( | ) |
salient feature vector difference
Definition at line 448 of file VisualObjectMatch.C.
References getVoRef(), and getVoTest().
Referenced by getSalScore().
| float VisualObjectMatch::getSalDist | ( | ) |
spatial distance of the salient points
Definition at line 472 of file VisualObjectMatch.C.
References Point2D< T >::distance(), getSIFTaffine(), getVoRef(), getVoTest(), Point2D< T >::i, and SIFTaffine::transform().
Referenced by getSalScore().
| float VisualObjectMatch::getSalScore | ( | const float | wcoeff = 0.0F, |
|
| const float | hcoeff = 0.0F | |||
| ) |
get a match score based on the salient feature difference * spatial distance
Definition at line 412 of file VisualObjectMatch.C.
References getSalDiff(), getSalDist(), getVoRef(), getVoTest(), and sqrt().
| float VisualObjectMatch::getScore | ( | const float | kcoeff = 0.5F, |
|
| const float | acoeff = 0.5F | |||
| ) |
Get a match score, higher scores are better.
Score returned here is defined as kcoeff / (1 + getKeypointAvgDist()) + acoeff / (1 + getAffineAvgDist()) + 0.05 * numKeypointMatches
Definition at line 382 of file VisualObjectMatch.C.
References getAffineAvgDist(), and getKeypointAvgDist().
Referenced by BeoSub::matchSIFT().
| SIFTaffine VisualObjectMatch::getSIFTaffine | ( | ) | [inline] |
Compute least-squares affine transform between matches.
If we have not already done so, we will compute the affine from all our Keypoint matches. Hence it is recommended that you call prune() prior to invoking this member function.
Definition at line 341 of file VisualObjectMatch.H.
Referenced by getFusedImage(), getOverlapRect(), getSalDist(), getTransfTestImage(), getTransfTestOutline(), main(), and BeoMap::makePanorama().
| Point2D< int > VisualObjectMatch::getSpatialDist | ( | Point2D< int > | objOffset1 = Point2D<int>(0,0), |
|
| Point2D< int > | objOffset2 = Point2D<int>(0,0) | |||
| ) |
get the spatial distance between the two objects
the offsets passed in are the the coordinates of the top left corner of each image. The result is the distance of the two respective origins, which is also the coordinate change of the two objects. This is useful for visual objects that come from a cropped frame. The result can be the camera movement or egomotion
Definition at line 827 of file VisualObjectMatch.C.
References Image< T >::getVal(), Point2D< T >::i, matrixMult(), Image< T >::setVal(), SIFTaffine::ty, and ZEROS.
| Image< PixRGB< byte > > VisualObjectMatch::getTransfTestImage | ( | const Image< PixRGB< byte > > & | im = Image< PixRGB<byte> >() |
) |
Get the image of the test object transformed to match the ref object.
If an uninitialized image is given, the resulting image has the size of the ref object's, and contains zeros everywhere except where the test object is. Otherwise, the test object is just painted into the given image (which must have the dims of the ref image.
Definition at line 734 of file VisualObjectMatch.C.
References Image< T >::beginw(), Image< T >::coordsOk(), Image< T >::getHeight(), getSIFTaffine(), Image< T >::getValInterp(), Image< T >::getWidth(), Image< T >::initialized(), Image< T >::resize(), and SIFTaffine::transform().
| void VisualObjectMatch::getTransfTestOutline | ( | Point2D< int > & | tl, | |
| Point2D< int > & | tr, | |||
| Point2D< int > & | br, | |||
| Point2D< int > & | bl | |||
| ) |
Get the transformed coords of the 4 corners of the test image.
The returned points have coords in the coord system of the ref image, but may fall outside that image.
Definition at line 765 of file VisualObjectMatch.C.
References getSIFTaffine(), Dims::h(), Point2D< T >::i, SIFTaffine::inverse(), SIFTaffine::transform(), and Dims::w().
| const rutz::shared_ptr< VisualObject > & VisualObjectMatch::getVoRef | ( | ) | const [inline] |
Get the reference VisualObject.
Definition at line 334 of file VisualObjectMatch.H.
Referenced by getMatchImage(), getOverlapRect(), getSalDiff(), getSalDist(), getSalScore(), isOverlapping(), and isOverlapping2().
| const rutz::shared_ptr< VisualObject > & VisualObjectMatch::getVoTest | ( | ) | const [inline] |
Get the test VisualObject.
Definition at line 338 of file VisualObjectMatch.H.
Referenced by getMatchImage(), getOverlapRect(), getSalDiff(), getSalDist(), getSalScore(), isOverlapping(), and isOverlapping2().
| bool VisualObjectMatch::isOverlapping | ( | ) |
check overlap using the image rectangle overlap
Definition at line 918 of file VisualObjectMatch.C.
References Rectangle::bottomI(), Image< T >::getBounds(), getOverlapRect(), getVoRef(), getVoTest(), Rectangle::height(), Rectangle::isValid(), Rectangle::left(), Rectangle::rightI(), Rectangle::top(), and Rectangle::width().
| bool VisualObjectMatch::isOverlapping2 | ( | ) |
check overlap using keypoint bounding box overlap
Definition at line 949 of file VisualObjectMatch.C.
References getVoRef(), getVoTest(), and size().
| KeypointMatch & VisualObjectMatch::operator[] | ( | const uint | idx | ) | [inline] |
Access a KeypointMatch, read/write version.
Definition at line 323 of file VisualObjectMatch.H.
| const KeypointMatch & VisualObjectMatch::operator[] | ( | const uint | idx | ) | const [inline] |
Access a KeypointMatch, read-only version.
Definition at line 319 of file VisualObjectMatch.H.
Apply a standard series of prunings.
This is a heuristic combination of calls to pruneByDist(), pruneByHough(), and pruneByAff(), so as to prune outlier matches and allow the recovery of a clean affine transform through getSIFTaffine(). Do not prune to fewer than 'minn' matches but try to prune down to fewer than 'maxn' matches (the latter is not guaranteed, as all matches may be very very good). Returns the number of outlier matches pruned away.
Definition at line 107 of file VisualObjectMatch.C.
References pruneByAff(), pruneByDist(), and pruneByHough().
Referenced by main(), BeoMap::makePanorama(), and BeoSub::matchSIFT().
Prune our matches by consistency with affine transform.
Normally you should just use prune() but this is made public for people who want finer control. Compute the affine transform from the matches, and eliminate the matches that disagree with it in that the distance in the test image between an affine-transformed reference keypoint and the matching test keypoint is larger than 'dist'. During this process, we will not continue if fewer than 'minn' matches remain. The number of outlier matches that were deleted is returned.
Definition at line 266 of file VisualObjectMatch.C.
References SIFTaffine::getResidualDistSq().
Referenced by prune().
Prune our matches by ratio of best to second best distance.
Normally you should just use prune() but this is made public for people who want finer control. The given thresh is in units of 10%. For example, if thresh=8, matches where the match distance is more than 0.8 the second best distance will be eliminated. Returns the number of matches pruned. Do not prune to fewer than 'minn' matches.
Definition at line 140 of file VisualObjectMatch.C.
Referenced by prune().
Prune the matches using a Hough transform.
Normally you should just use prune() but this is made public for people who want finer control. Returns the number of matches pruned. Do not prune to fewer than 'minn' matches. Prune matches that disagree with the most popular transform by more than 'rangefac' times the range in any dimension; sensible values are between 0.05 (extremely strict; not recommended since the family of geometric transformations used here is very approximative) and 0.5 (may not prune anything).
Definition at line 161 of file VisualObjectMatch.C.
References SIFThough::addValue(), and SIFThough::getPeak().
Referenced by prune().
| void VisualObjectMatch::push_back | ( | const KeypointMatch & | km | ) | [inline] |
Add a Keypoint match to the back of our internal list.
Definition at line 315 of file VisualObjectMatch.H.
| uint VisualObjectMatch::size | ( | ) | const [inline] |
Get number of Keypoint matches we currently have.
Definition at line 312 of file VisualObjectMatch.H.
Referenced by isOverlapping2(), main(), and BeoMap::makePanorama().
1.6.3