Author |
Topic: Methods for computing orientations (Read 7597 times) |
|
Dirk Bernhardt-Walther
YaBB Full Member
Posts: 155
|
 |
Methods for computing orientations
« on: 04/09/03 at 08:03:56 » |
 Quote  Modify
|
Hi
I am putting this recent email exchange between Rob and me online - I think it is interesting for everyone:
Hi Rob
I have played around with GaborPyramid a bit. I have a few questions about this:
- In line 175 (or around there) you compute the kernel size as int(ceil(max_stddev * sqrt(-2.0F * log(exp(-5.0F)))))
> Why did you write "-2.0F * log(exp(-5.0F))" instead of 10.0F ?
> Why do you use sqrt(10) as a scaling factor? I think it is too large - the filters have a lot of gray pixels around them that don't contribute much. In my local version, I have made it possible to state the size explicitly, and with filter_period = 7 and elongation = 1, a size of 4 (kernel size 9x9) works just fine.
- What was your reason to re-write the convolution for GaborPyramid? What's the difference to convolve or convolveClean?
I thought of adding a enum ModelParam to GaborChannel that would state the method used to compute the OrientationChannel something like "steeribleFilters", "Gabor", "GaborNormalized", and also an option for the kernel size.
Thanks for the nice solution with mexopts.sh!
Dirk
|
| « Last Edit: 04/09/03 at 08:14:28 by Dirk Bernhardt-Walther » |
 Logged |
|
|
|
Dirk Bernhardt-Walther
YaBB Full Member
Posts: 155
|
|
Re: Methods for computing orientations
« Reply #1 on: 04/09/03 at 08:13:48 » |
Quote Modify
|
And here is Rob's reply:
> I have played around with GaborPyramid a bit. I have a few questions about
> this:
>
> - In line 175 (or around there) you compute the kernel size as
> int(ceil(max_stddev * sqrt(-2.0F * log(exp(-5.0F)))))
I was just copying out of the gaborFilter() function in Image_FilterOps... as you say, maybe we can do better here (and since the convolution is O(N^2) in the filter size, any size reduction would be nice).
> > Why did you write "-2.0F * log(exp(-5.0F))" instead of 10.0F ?
> > Why do you use sqrt(10) as a scaling factor? I think it is too large -
> the filters have a lot of gray pixels around them that don't contribute
> much. In my local version, I have made it possible to state the size
> explicitly, and with filter_period = 7 and elongation = 1, a size of 4
> (kernel size 9x9) works just fine.
> - What was your reason to re-write the convolution for GaborPyramid?
> What's the difference to convolve or convolveClean?
(1) My convolve function is optimized to have separate inner loops for the boundary cases and for the non-boundary cases. This gives a significant speedup over having one inner loop in which each iteration has to branch to either boundary or non-boundary code.
(2) My convolve function has a different edge treatment. Essentially, convolve() does zero-padding, convolveClean() uses a truncated filter, and my function does "mean-padding", where we fill in the missing pixels with the mean value of the actual pixels that overlap the filter. For example, let's say we're doing a five-point low-pass with the following values in the kernel:
[1/16 4/16 6/16 4/16 1/16]
If we're at a left edge, then we need to pick three values for the kernel. Our options are:
[6/16 4/16 1/16] (zero-padding, used in convolve())
[6/11 4/11 1/11] (truncated filter, used in convolveClean())
[23/48 17/48 8/48] ("mean-padding", used in optConvolve())
== [6/16+5/48 4/16+5/48 1/16+5/48]
(take the mean of the three real pixels, and apply those to the missing 1/16 and 4/16 filter elements... this is a total of 5/16 of weight, divided among three real pixels, for 5/48 of weight each)
I found that this last condition gave the best-looking (artifact-free) edges, at least for the kinds of images I was looking at. If you find that it doesn't work as well for other kinds of images, we could think about making the edge-handling style another run-time parameter...
> I thought of adding a enum ModelParam to GaborChannel that would state the
> method used to compute the OrientationChannel something like
> "steeribleFilters", "Gabor", "GaborNormalized", and also an option for the
> kernel size.
Yeah, something like that would be a nice idea for GaborChannel+OrientationChannel.
Cheers,
Rob
|
|
Logged |
|
|
|
Dirk Bernhardt-Walther
YaBB Full Member
Posts: 155
|
|
Re: Methods for computing orientations
« Reply #2 on: 04/09/03 at 17:37:51 » |
Quote Modify
|
Hi
The gabor method for extracting orientations is up and running now. However, a few things are strange:
In order to remove any remainders of the source image itself, I had to normalize the g0 and g90 gabor filters such that their mean = 0 -- i.e., I had to subtract the mean. Plus, I had to subtract the source image from the result and then clamp the whole thing - thereby, I think, sacrificing half of the phase of the Gabor.
Something is really strange here - I'll have to look at it more thoroughly tomorrow. In case you would like to have a look: around line 250 of Pyramid_Ops.C; I have marked the doubtful places with // ? ? ?
You can check the effect quickly using: test-gabor2 image.ppm angle
Or you can use option --oricomp-type=Gabor9 in ezvision and friends.
Perhaps you have some idea - otherwise I'll deal with it tomorrow.
Dirk
|
|
Logged |
|
|
|
Rob Peters
YaBB Senior Member
 
Gender: 
Posts: 398
|
|
Re: Methods for computing orientations
« Reply #3 on: 04/10/03 at 12:25:53 » |
Quote Modify
|
on 04/09/03 at 17:37:51, Dirk Walther wrote:| In order to remove any remainders of the source image itself, I had to normalize the g0 and g90 gabor filters such that their mean = 0 -- i.e., I had to subtract the mean. |
|
This makes sense.
Quote:| Plus, I had to subtract the source image from the result and then clamp the whole thing - thereby, I think, sacrificing half of the phase of the Gabor. |
|
Actually, I think we can handle this on the input end and leave buildPyrGabor() as it was. Basically we want to pre-filter the input with some kind of (pseudo-)high-pass filter... then the input already has essentially zero-mean, so the g0/g90-normalization issue disappears, and we don't need to subtract the original image from the filter output.
One way to do the high-pass filter is just to take the laplacian of the input first. Another way that I think works better is to do a "pseudo-energy-norm":
Code:ImageSet<float> gausspyr = buildPyrGaussian(...);
for (uint i = 0; i < gausspyr.size(); ++i) {
const Image<float> x = gausspyr[i] + 1.0f; // add 1.0 to prevent divide-by-zero
const Image<float> energy = sqrt(lowPass9(x * x));
const Image<float> energyNormed = x / energy;
gausspyr[i] = energyNormed - mean(energyNormed);
}
// Now build a gabor pyramid from the energy-normed gaussian pyramid:
ImageSet<float> gaborpyr = buildPyrGabor(gausspyr, theta, period);
|
|
We could make a buildPyrGaussianEnergyNorm() function that wraps those first steps. But at any rate, I'd prefer to not have one particular approach hard-coded into buildPyrGabor(), since we might want to use the laplacian or energy-norm under different circumstances.
How about this... there are two buildPyrGabor() functions: one takes a pyramid as input, and just does the gabor convolution at each pyramid level; the second takes a single image as input, then builds a (gaussian) pyramid from it, then feeds that pyramid into the first function.
We could add an enum parameter to the second function (the one that takes a single input image) to specify what kind of pyramid to feed into the gabor convolution: could be an unadorned gaussian pyramid, could be a laplacian pyramid, could be an energy-normed gaussian pyramid.
Cheers,
Rob
|
| « Last Edit: 04/10/03 at 12:26:40 by Rob Peters » |
Logged |
|
|
|
Rob Peters
YaBB Senior Member
Gender:
Posts: 398
|
|
Re: Methods for computing orientations
« Reply #4 on: 04/10/03 at 14:40:27 » |
Quote Modify
|
Well, OK there is more to this than I had thought. Dirk, your clampedDiff() operations seems to be doing something nice indeed... I might have been getting a similar effect in the contour-finding stuff by post-processing in which, after computing pyramids for all orientations, for each scale I compute the mean image across orientations, and then subtract that from each of the orientation outputs at that scale.
In any case, I added several flags (see Pyramid_Ops.H) for buildPyrGabor so that we can control all the parameters... basically we have three parameter groups:
- High-pass filtering? Here we can either do none, or we can do an energy-norm (DO_ENERGY_NORM), or do a laplacian (DO_LAPLACIAN).
- Absolute value? Whether or not to take the absolute value of the filter outputs. In principle we need this in order to get output from the four possible filter phases (i.e. xox, oxo, xo, and ox), so the default is to use the absolute value, but can be turned off with NO_ABS.
- Whether to use your clamped diff operations (DO_CLAMPED_DIFF).
So there are 12 sets of options:
Code:
OPTION(0 | 0 | NO_ABS);
OPTION(0 | 0 | 0);
OPTION(0 | DO_ENERGY_NORM | NO_ABS);
OPTION(0 | DO_ENERGY_NORM | 0); // Rob's original
OPTION(0 | DO_LAPLACIAN | NO_ABS);
OPTION(0 | DO_LAPLACIAN | 0);
OPTION(DO_CLAMPED_DIFF | 0 | NO_ABS); // Dirk's modification
OPTION(DO_CLAMPED_DIFF | 0 | 0);
OPTION(DO_CLAMPED_DIFF | DO_ENERGY_NORM | NO_ABS);
OPTION(DO_CLAMPED_DIFF | DO_ENERGY_NORM | 0);
OPTION(DO_CLAMPED_DIFF | DO_LAPLACIAN | NO_ABS);
OPTION(DO_CLAMPED_DIFF | DO_LAPLACIAN | 0);
|
|
I modified test-gabor2 to exercise all of these options... not sure what to make of it all yet though.
By the way... Pyramid_Ops.C ended up with
Code:g0 -= sum(g0);
g90 -= sum(g90); |
|
which I changed to
Code:g0 -= mean(g0);
g90 -= mean(g90); |
|
...this is probably what was making the original algorithm give such wacky results.
Now, what exactly is going on with clampedDiff()? Dirk, as you say, we are probably losing part of the phase information here... we can retain all of the phase information from the filter outputs by applying abs(f0) and abs(f90) before doing the clampedDiff(), but then we still have the problem that the image itself is not phase neutral... the operation is more likely to cancel out contours at bright parts of the image than at dark parts.
Maybe clampedDiff() is just acting as a threshold and making the output look cleaner? That seems to be the result of comparing 0|0|0 (image) vs. DO_CLAMPED_DIFF|0|0 (image) at least for my test image (which was saliency/simp/simp.ashes.pgm).
Then compare DO_CLAMPED_DIFF|0|0 (image) vs. DO_CLAMPED_DIFF|DO_ENERGY_NORM|0 (image) to see how the energy-normalization is making the output less dependent on the local image contrast... although in some cases this seems to bring up "signal" it also seems to bring up "noise" in other cases.
At this point, I'm not sure there's much difference between DO_CLAMPED_DIFF|DO_ENERGY_NORM|0 (image)
and 0|DO_ENERGY_NORM|0 (image).
Cheers,
Rob
|
|
Logged |
|
|
|
Dirk Bernhardt-Walther
YaBB Full Member
Posts: 155
|
|
Re: Methods for computing orientations
« Reply #5 on: 04/22/03 at 08:33:08 » |
Quote Modify
|
Quote: By the way... Pyramid_Ops.C ended up with
Code:
g0 -= sum(g0);
g90 -= sum(g90);
which I changed to
Code:
g0 -= mean(g0);
g90 -= mean(g90);
...this is probably what was making the original algorithm give such wacky results. |
|
Yes, this mistake had messed me up thoroughly. With that part corrected, flag configuration ( 0 | 0 | 0 ) works fine, actually. I don't think there is any need for the clampedDiff part at all, and neither for NO_ABS.
I have played around with and without energy normalization - I think the difference is minimal. But I did see a nice improvement when I subtracted the average over all orientations from each orientation - see test-gabor3. I think this works much nicer than compensating for the local image energy. I guess I will start and create a channel that does that.
I have not yet tried how much subtracting the average over all orientations may also help when using steerible filters. I'll have a look ... 
Dirk 
|
|
Logged |
|
|
|
Dirk Bernhardt-Walther
YaBB Full Member
Posts: 155
|
|
New options for computing orientations
« Reply #6 on: 04/28/03 at 07:13:01 » |
Quote Modify
|
Hey ho - It's spring in Boston, finally! 
I have built in two new options for computing orientations:
--ori-interaction=<None|SubtractMean|c1,...,cn> [None]
This describes the way in which the different orientations within the OrientationChannel interact with each other. 'None' is for no interaction (default); for 'SubtractMean', the mean of all orientation pyramids is subtracted from each orientation pyramid. You can determine your own interaction by specifying a vector with interaction coefficients: 'c1,...,cn' where n is the number of orientations. c1 is the coeffiecient for an orientation itself, c2 for the one with the next higher angle and so on. For instance, for n=4, 'None' is the same as '1,0,0,0', and 'SubtractMean' is the same as '0.75,-0.25,-0.25,-0.25'. [OriInteraction]
--oricomp-type=<Steerable|Gabor|GaborEnergyNorm> [Steerable]
Type of computation used to compute orientations [OrientComputeType]
I have experimented with them a little this weekend, trying to improve the detection of very faint alongated marine animals. I could not see much difference in using Gabor over Steerable, and only a very slight advantage in using GaborEnergyNorm over Steerable.
However, having --ori-interaction=SubtractMean made a big difference. Using this option helps to separate saliency due to an oriented edge and saliency due to luminance contrast. In my pictures I have a lot of high-contrast blobs with not much orientation. They had totally swamped out the faint alongated guys in the orientation maps. Now I get rid of them and can finally give edges a higher priority over luminance by increasing the weight for the orientation conspicuity map.
I have also built in the possibility to enter your own interaction mode. If your coefficients sum up to 0, you should have a result that is more or less independent of luminance contrast. With the sum equal to 1 you will have the influence of the luminance contrast as strongly as we have had it so far.
Have fun playing! 
Dirk
|
|
Logged |
|
|
|
Laurent Itti
YaBB Moderator
YaBB God
iLab rocks!
Gender:
Posts: 550
|
|
Re: Methods for computing orientations
« Reply #7 on: 04/28/03 at 11:12:48 » |
Quote Modify
|
yeah, sounds most excellent! will be playing with it indeed 
thanks!
-- laurent
|
|
Logged |
|
|
|
donald92135-AT-yahoo.com
YaBB Full Member
I love YaBB 1G - SP1!
Posts: 130
|
|
Re: Methods for computing orientations
« Reply #8 on: 05/08/06 at 01:01:51 » |
Quote Modify
|
If you have some time check out this COOL website:
http://www.google.com
Rest assured this is great entertainment!!!
Also check out the following website I came across:
http://www.usc.edu
It will knock your socks off!
|
|
Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print