Pattern matching robustness

[Klemen]

Hello!

I was hoping someone with more experience in pattern matching could offer some help.

I am acquiring two types of images; depth image and rgb image (both images are aligned - calibrated, so that the position of the corresponding pixels match).

Rgb image is used to find distinct markers on the measured object by using the IMAQ Find Pattern 3 VI (rotation or shift invariant). The tracked markers directly ifluence the 3d reconstruced object, since i am using their position to translate and rotate the 3d object in my global coordinate system. The template for the correlation is created automatically by selectiong a ROI on the rgb image prior to matching.

This is working as expected (the correlation score is high, averaging around 0.98), but i would like to improve the robustness of the process. Because of the matching, the positions of the markers in the image change rapidly, which consequently results as an error on the 3d object (Z-axis error of ~ 3mm). This is manifested as a low amplitude high frequency motion of the 3d object in real time measurements.

Any suggestions for improving this? I am already using Gaussian blur on the rgb image (ksize = 5), which helps to some extent.

Maybe this is the result of the improper image contrast, or autocontrast property of the rgb camera?

Thank you for any help or advice,

K

[CAS_MattF]

I all depends on your source image.  I use a direct digital capture method and 100% reliable with no issues.  If you using a camera then you need to control your environment.  Light level, object position, distance, focus, ect...  If anything changes when you will need to create new reference images.  Final note, NEVER use a compressed image format like JPG, PNG, GIF, ect...  I always use BMP that are not compressed but larger.  You can always compress them for saving after the pattern match is complete.

[Klemen]

Thank you for the reply. I usually use the bmp format (also for saving,since i need to run the same process on the acquired data later).

I belive my biggest issue is in the rgb camera properties - auto contrast and auto white balance... But unfortunatelly i cannot acess this properties.

[Klemen]

Having some problems with my mobile,sory.

To continue the last post:

I do some software correction of brightnes and contrast,which puts my corr. score to 0.99. But i think this small increase does not play any significant part for my problem. The biggest issue is in the detection of the patterns center positions.

I think the best thing would be to try to access the camera properties. Until then i need to assure the most optimal and constant lighting conditions during the measurements.

Any thoughts and/or comments?

Thank you,
K

[Zody]

Hey Klemen,

Ofcourse it is very important having the proper and constant lighting.

Here you can find an example of pattern matching.

http://zone.ni.com/devzone/cda/epd/p/id/5594

Kind regards,

[invancea]

Hello Klemen,

You have mentioned that positions of markers are changing rapidly. Are you saying that patternmatching returns different positions though the actual position of markers does not change much? Could you provide more information about deviation of positions returned by pattern matching? Average pixel deviation?  Is it possible for you to post few images to illustrate the issue?

Thanks,

Antony.

[CAS_MattF]

To see how you image is changing and not matching you can do an IMAQ Subtract to visually see the difference.  This may help you understand what is going on with your inspection.

[Klemen]

Thank you all for your suggestions and answers. Will try them as soon as possible.

Below is the RGB image, where the matches are overlaid (note that the actual matching is of course performed on an u8 image). You can see that the positions are shifting rapidly. This is "worst-case" scenario (almost direct sunlight, changing shadows, etc...). The method is rotation invariant (-90 to 90 deg). Shif invariant produces by my estimation (lower correlation) worst results.

The green ROI's are where the pattern matching occurs (they shift with the positions of the markers). The yellow ROI is used to extract the 3d shape below (the "high-fequency, low amplitude" motion is also visible).

This describes my problem.

I will measure average deviation of pixels and post the results when I can.

Regards,

K

[Klemen]

Upddate on the standard deviation of markers center position (in pixels) based on 1000 samples:

Coord.                         u          v

M0 (top left):             0.03     0.20

M1 (top right):           0.17     0.59

M2 (bottom left):        0.04     0.17    

M3 (bottom right):      0.14     0.29

I think the markers are not the cause of this problem (the accuracy is subpixel) - I think the 3d transformation is the culprit.

Best regards,

K

[invancea]

Klemen,

Thanks for your effort in making all these very nice illustrative gif files. I think size of marker (small?) and lighting variation are causing inaccuracy with Pattern Matching.

Few Suggestions.

1. Have you tried using little bigger Dots? Pattern Matching could provide better stability if templates are reasonably large.

2. I guess your set up is free of perspective distortion, if you feel that camera not perpendicular it is better to use different templates for each marker.

3. You mentioned that shift invariant provides worst resutls. You could use lower Initial step size (~3) in Pattern Matching Advanced options to improve the matching.

4. Just try something other than PM. Assuming that those are circular dots and they are constant for your setup, you can fit circle using Contour Analysis and use the center point as location.

Thanks,

Antony.