Converting VisionImage to System.Drawing.Bitmap/Image in .NET, and vice-versa: my solution!

[Nelval]

First, let me share my rants... I'm really sick of this, I have the last couple of days without finding anything, the support service is not that aware of the .NET functionalities, my comapny paid for the NI Vision license but the support is not even able to tell me how to deal properly with the .NET libraries... the documentation is to me, not clear enough... (most of the time I have to refer to LabVIEW documentation and to compare whether first if the methods are matching and secondly to adapt them correcly to .NET).

I know that NI is mostly based on C & LabVIEW... but please do not forget .NET technologies!

Anyway. the topic is how to convert a VisionImage Object to a Bitmap / Image Object from the System.Drawing assembly. Here is the situation, I have to put several selection on some pictures through the ImageViewer and get from a camera.(here is also about the ImageViewer there is some troubles and would be good to talk deeply about this into another topic, badly coded, and not really not adaptable or customizable either).

Those pictures are used then as some patterns for performing analysis in real situations, the results (given by image processing to see if pictures from real scenes match expectations from the definitions defined previously) have then to be saved and displayed into a huge DataGridView / GridControl (DevExpress Control another third party libray in my project, Im not making advertising either, just for explaining...) so I cant really use the ImageViewer (first, it's really memory consuming to display it even by using a Server Mode (means only loaded ).

Imagine even 100 rows in table showing this component..., when especailly I already have .NET components doing this job pretty well but guess what they use Image / Bitmap Objects from System.Drawing assembly...

Most of people thinks about a I/O solution (basically it means you have your ObjectA, you export / save it to a file and then you load from this file to the other type of ObjectB (let's say) you wanted to convert... Obviously it takes time, poorly efficient (yes it works but we already have Object present in memory why the f***k we have to save them into a file just to load them into another file...).

I also give a try with the Serialization of the Bitmap / Image to Byte[] and Flatten and Unflatten methods, but do not work as well... The reason comes from that serialization is performed with the whole object not only about the pixel values... so it throws exceptions in both directions.

Another proposal given by some others and actually the first test I tried was to convert from Bitmap to VisionImage and vice-versa using a double loops system (or even a single one) by getting pixel and setting pixel... guess what it is incredibly slow too! The reason is quite simple, the getter / setter (in both Object's, Bitmap, Image and even VisionImage) are super slow cause they have to lock the memory to prevent from any memory leak, overflow or unmanaged tricky things... (safe and slow again...). I also tried to use the ArrayToImage methods or ImageToArray methods provided by the VisionImage Object but it all the same old song!

Then this solution to another one, is actually to get the pixels directly from the memoy, there are a couple articles / posts about it. First I have to say that's Im still wondering WHY adding alignment in a picture values would make it faster for image processing (except for a better dealing with border
?)

Why oh the h**s you NI chose to put border + alignments within the format of the ImageVision? To try to have power of 2 and bits shifting tricky things? Please explain, cause I'm really curious about that!

Anyway considering this: 

By the way, I also did some revert engineering to better understand your VisionImage Object (located using NationalInstruments.Vision assembly) guy. Thanks http://ilspy.net/ but most of the Methods are in fact make calls to Imaq Library through the NiVision.dll, purely binary based (guess this one is coded with C or C++, cause performance matters). But I could saw some hidden members that was intentionally hidden! So why...?

(basically they are hidden in any Microsoft Visual Studio but if you are using another non-MS IDE (and that does exist!) like SharpDevelop you can see the sh*t behind), like some pointers, lineWidth).

One way conversion (VisionImage to Bitmap / Image) is pretty straightforward and does not required some wordaround... some of them on the website already found solution it is really functional, it can be found here... (but you have to dig a lot for that...) see the Sources below. But the other is complicated... first you have to find out the StartPtr, secondly you have to Copy the bytes extracted from LockBits / UnlockBits methods from the pointer of Bitmap / Image pointer to a manager array and then copy back to the StartPtr with the right length...

FYI, NI is supposed to provide easy-to-use (also called, "user-friendly) solutions and not pain-in-the-a** but when it's just a little not that NI Spirit (i.e. not LabVIEW related...) seems hard.

Would it be too hard for you guy to make some efforts? I usually do not like to loose my time for complaining, but when someone / company is paying for a product it/he/she has expectations... just do better a .NET job for the coming versions, thanks!

Sources:

http://forums.ni.com/t5/Measurement-Studio-for-NET/How-to-obtain-System-Drawing-Image-from-CWIMAQIma...

http://forums.ni.com/t5/LabVIEW/on-IMAQ-ArraytoImage/m-p/1021897/highlight/true#M455529

http://forums.ni.com/t5/Machine-Vision/Vision-2009-ArrayToImage-Method-difference/m-p/1063821/highli...

https://decibel.ni.com/content/docs/DOC-15159

https://decibel.ni.com/content/docs/DOC-13140

https://decibel.ni.com/content/docs/DOC-15165

http://forums.ni.com/t5/Machine-Vision/converet-IMAQ-to-Bitmap-in-memory-without-saving-it-as-a-file...

http://forums.ni.com/t5/Machine-Vision/How-can-the-ArrayToImage-function-be-speeded-up/td-p/952119

[Nelval]

Seems the thread have been created twice, sorry for that.

Here is my solution:

public static VisionImage ToVisionImage(this Image image)
{
return NIHelperMethods.ConvertImageToVisionImage(image);
}

/// <summary>
/// Convert a VisionImage to an equivalent Image Object.
/// </summary>
/// <param name="visionImage">The VisionImage Object used a source of the conversion.</param>
/// <returns>An equivalent Image Object.</returns>
public static Image ToImage(this VisionImage visionImage)
{
return NIHelperMethods.ConvertVisionImageToImage(visionImage);
}
}

[Nelval]

Sorry again here is the complete version

/// <summary>
/// A set of methods dealing with NI Vision particularities...
/// </summary>
public static class NIHelperMethods
{
/// <summary>
/// Convert a Image / Bitmap Object to an equivalent VisionImage Object.
/// </summary>
/// <param name="image">The Image Object used a source of the conversion.</param>
/// <returns>An equivalent VisionImage Object.</returns>
public static VisionImage ConvertImageToVisionImage(Image image)
{
VisionImage visionImage = new VisionImage(ImageType.Rgb32, 0);
visionImage.SetSize(image.Width, image.Height);

Int32 pixelFormatSize = Image.GetPixelFormatSize(image.PixelFormat) / 8;

// Why???? Why DO I need to add this image.width... magic?

Int32 stride = image.Width * (1 + pixelFormatSize);

Bitmap bitmap = new Bitmap(image).Clone(new Rectangle(0, 0, image.Width, image.Height), PixelFormat.Format32bppRgb);
Rectangle rectangle = new Rectangle(0, 0, image.Width, image.Height);
BitmapData bitmapData = bitmap.LockBits(rectangle, ImageLockMode.ReadOnly, bitmap.PixelFormat);

Byte[] bytes = new Byte[stride * image.Height];
Marshal.Copy(bitmapData.Scan0, bytes, 0, bytes.Length);
bitmap.UnlockBits(bitmapData);
Marshal.Copy(bytes, 0, visionImage.StartPtr, bytes.Length);

return visionImage;
}

/// <summary>
/// Convert a VisionImage to an equivalent Image Object.
/// </summary>
/// <param name="visionImage">The VisionImage Object used a source of the conversion.</param>
/// <returns>An equivalent Image Object.</returns>
public static Image ConvertVisionImageToImage(VisionImage visionImage)
{
ImageType imageType = visionImage.Type;

CommonAlgorithms.Cast(visionImage, visionImage, ImageType.Rgb32);

Bitmap bitmap = new Bitmap(visionImage.Width, visionImage.Height, (Int32)visionImage.LineWidthInBytes, PixelFormat.Format32bppRgb, visionImage.StartPtr);
Bitmap bitmapClone = bitmap.Clone() as Bitmap;
Rectangle rectangle = new Rectangle(0, 0, bitmapClone.Width, bitmapClone.Height);
BitmapData bitmapData = bitmapClone.LockBits(rectangle, ImageLockMode.ReadOnly, bitmapClone.PixelFormat);
bitmapClone.UnlockBits(bitmapData);

CommonAlgorithms.Cast(visionImage, visionImage, imageType);

return bitmapClone;
}
}

/// <summary>
/// A set ofextension methods dealing with NI Vision particularities...
/// </summary>
public static class NIExtensionMethods
{
/// <summary>
/// Convert a Image / Bitmap Object to an equivalent VisionImage Object.
/// </summary>
/// <param name="image">The Image Object used a source of the conversion.</param>
/// <returns>An equivalent VisionImage Object.</returns>
public static VisionImage ToVisionImage(this Image image)
{
return NIHelperMethods.ConvertImageToVisionImage(image);
}

/// <summary>
/// Convert a VisionImage to an equivalent Image Object.
/// </summary>
/// <param name="visionImage">The VisionImage Object used a source of the conversion.</param>
/// <returns>An equivalent Image Object.</returns>
public static Image ToImage(this VisionImage visionImage)
{
return NIHelperMethods.ConvertVisionImageToImage(visionImage);
}
}

[Andrey_Dmitriev]

---

@Nelval wrote:

 First I have to say that's Im still wondering WHY adding alignment in a picture values would make it faster for image processing (except for a better dealing with border

?)

Why oh the h**s you NI chose to put border + alignments within the format of the ImageVision? To try to have power of 2 and bits shifting tricky things? Please explain, cause I'm really curious about that!

---

This is a very basic question. The short answer here - because performance. You will get penalties by access unaligned pointers. In most imageprocessing libraries (such as Intel Perfomance Primitives or OpenCV) the images are aligned - that means each row started from aligned address. On modern CPUs penalties may be not so dramatical as with old processors - but still exists. Second reason - for very high performance algorithms you can use SSE commands for processing and process several pixels with single operation - and these commands will work only if your image is aligned.

Andrey.

[Nelval]

Hm interesting that's probably why my solution is not that perfect... it seems that, when I convert from a bitmap(actually converted from a VisionImage Object, loaded from a file) to a new VisionImageObject it does not work properly (sometimes), there is some missing parts or the pictures rotated and kinda wrongly ordered.

According to the NI document, only the first pixel should be 32 bytes aligned. It would mean that a picture read by the Bitmap object is not necesseraly aligned that's interesting!

I can have the first pixel address, how can I change it to make it 32 bytes aligned... something like below?

// Seems the anything in .NET is 8 bytes, still I dont not understand why my code posted previously could work in some cases.
IntPtr intPtr = Marshal.AllocHGlobal(size +8);

// Alignment
IntPtr intPtrAligned = newIntPtr(32 *(((intPtr as Int64) + 31)/32));

// Use intPtrAligned//...
Marshal.FreeHGlobal(intPtrAligned);

[Nelval · Accepted solution]

Ok it seems that the documentation about the internal structure of NI Vision Image is not that precise.

I finally found out a solution that work for any kind of picture (as a both dimensions are greater than zero), 

/// <summary>
        /// 
        /// </summary>
        /// <param name="image"></param>
        /// <returns></returns>
        public static VisionImage ConvertImageToVisionImage(Image image)
        {
            VisionImage visionImage = new VisionImage(ImageType.Rgb32, 0);
            visionImage.SetSize(image.Width, image.Height);

            Bitmap bitmap = new Bitmap(image);
            Rectangle rectangle = new Rectangle(0, 0, bitmap.Width, bitmap.Height);
            bitmap = bitmap.Clone(rectangle, PixelFormat.Format32bppRgb);

            Int32 pixelFormatSize = Image.GetPixelFormatSize(bitmap.PixelFormat) / 8;
            Int32 strideBitmap = bitmap.Width * pixelFormatSize;
            Int32 strideVisionImage = Convert.ToInt32(visionImage.LineWidthInBytes);
            Int32 alignmentWidth = strideVisionImage - strideBitmap;

            BitmapData bitmapData = bitmap.LockBits(rectangle, ImageLockMode.ReadOnly, bitmap.PixelFormat);
            Byte[] bytes = new Byte[strideBitmap * bitmap.Height];
            Marshal.Copy(bitmapData.Scan0, bytes, 0, bytes.Length);
            bitmap.UnlockBits(bitmapData);

            if (strideBitmap == strideVisionImage)
            {
                Marshal.Copy(bytes, 0, visionImage.StartPtr, bytes.Length);
            }
            else
            {
                Byte[] bytesWithAlignments = new Byte[strideVisionImage * image.Height];

                Int32 offset = 0;

                for (Int32 i = 0; i < bytesWithAlignments.Length; i++)
                {
                    if (((i % strideVisionImage) == 0) && (i > alignmentWidth))
                    {
                        offset += alignmentWidth;
                    }

                    if ((i - offset) < bytes.Length)
                    {
                        bytesWithAlignments[i] = bytes[i - offset];
                    }
                }

                Marshal.Copy(bytesWithAlignments, 0, visionImage.StartPtr, bytesWithAlignments.Length);
            }

            return visionImage;
        }

There is probably a better and more efficient solution that Im gonna dig for soon, then I will publish here, this huge missing part of NI Vision .NET library.

[Andrey_Dmitriev]

You will get memory leak here

Byte[] bytes = new Byte[strideBitmap * bitmap.Height];

and here:

Byte[] bytesWithAlignments = new Byte[strideVisionImage * image.Height];

Or .net garbage collector will delete allocated memory automatically?

[Nelval]

Good point, that actually I already thought about it, but I did some tests, dealing with hundreds of conversions without a fuss.

In case, I would add a GC.Collect(), maybe also giving a preference for the youngest generation of Objects in memory.

In fact, I would have been more worried about the fact of the copyied bytes(cause it;s not a shallow copy), than the managed arrays (that are certainly deleted once the GC is mad at them :p).

[JohnW242424]

While this may not be a 'true solution' and might not work for all cases, here is a quick and simple way to do it that I have used often.

All you need to do is copy your bitmap to the clipbopard,

then call the Algorithms.ClipboardToImage(VisionImage) method.

Here is some C# code as an example.

//assuming you previously created a Bitmap

//called someBitmap, PixelFormat.Format32bppRgb

Clipboard.SetImage(someBitmap);

VisionImage visImg = new VisionImage(ImageType.Rgb32);//select the correct type on it will get set to U8 by default
Algorithms.ClipboardToImage(visImg);

//not sure what languages everyone is using but you should

//cleanup the clipboard and your bitmaps or you can run out of memory

//pretty fast if you are working with big images
Clipboard.Clear();

someBitmap.Dispose();

Thats it!

[Andrey_Dmitriev]

---

@JohnW242424 wrote:

While this may not be a 'true solution' and might not work for all cases, here is a quick and simple way to do it that I have used often.

All you need to do is copy your bitmap to the clipbopard,

then call the Algorithms.ClipboardToImage(VisionImage) method.

---

What happened if I'll hit Print Screen key while the code executes between Clipboard.SetImage and Algorithms.ClipboardToImage calls? (Assumed that I'm fast and lucky enough )