How LabVIEW Stores Variant Data in Memory ?

[Peter_S]

Hello,

The documentation page http://zone.ni.com/reference/en-XX/help/371361P-01/lvconcepts/how_labview_stores_data_in_memory/ write to variant only: LabVIEW stores variants as handles to a LabVIEW internal data structure. Variant data is made up of 4 bytes.

I have binary data from an array of waveform, flattend as string and have to read it outside of Labview (C#, C++) . With the refered documentation I can read: ArrayDimension, t0, dt, data array.

But than come the attributes as variant.

Where can I find the documentation of the variant memory layout?

Peter

[wiebe@CARYA]

---

@Peter_S wrote:

Where can I find the documentation of the variant memory layout?

---

I think you've found all there is...

The easy way out would be to get the attribute info in LV and then pass it to the dll.

Alternatively, this might be possible by calling the GVariantTDR_ functions exported by LabVIEW.exe. But those are not documented, and their prototypes might change over LabVIEW version. The variant type data structure might also change over LV versions and targets.

EDIT: Actually the exported function name is LvVariantGetAttribute, but I don't think the prototype is public. Sometimes those functions require inputs that are simply not available outside the runtime engine...

[rolfk]

Just as Wiebe already explained. The LabVIEW variant memory layout is private and undocumented. It is also subject to change between LabVIEW versions. The only safe way to access them across LabVIEW versions are the exported LvVariant.... APIs but they are undocumented too, and likely subject to changes too, which is one reason not to document them. Once documented they couldn't change them anymore and need to make backwards compatibility hacks.

The flattened format is also not really documented and subject to change between LabVIEW versions but I know for one thing that the flattened format does contain a version number. So with loads of time it could be possible to reverse engineer that part and make it relatively safe to use even across LabVIEW versions. However I don't have the time nor inclination to do it.

[wiebe@CARYA]

NI could provide\develop an API to get attributes if they wanted to (could take years though).

I think the API\function is actually there, but without documented it's useless for all the mentioned reasons.

In LabVIEW, Get Attribute returns data as a variant, or adapts to it's wired input. So an (excising or future) API that does that won't solve much. You'd get the attribute as a variant, but you still need to get it's value.

I do think OP has a valid use case, and there are no easy workarounds.

[Peter_S]

Interesting: "The LabVIEW variant memory layout is private and undocumented."

Then NI should write this at http://zone.ni.com/reference/en-XX/help/371361P-01/lvconcepts/how_labview_stores_data_in_memory/

And the hint is not very helpfull: "LabVIEW stores waveforms like clusters." A waveform include a variant with the attributes.

But is it ok. The waveform components must copied to a cluster and the attributes must  stored in native types (arrays of string/numbers) or  as json string with JKI's lib. In the source https://github.com/JKISoftware/JKI-JSON-Serialization you can see: every variant type is handled as special case.

[rolfk]

---

@Peter_S wrote:

But is it ok. The waveform components must copied to a cluster and the attributes must  stored in native types (arrays of string/numbers) or  as json string with JKI's lib. In the source https://github.com/JKISoftware/JKI-JSON-Serialization you can see: every variant type is handled as special case.

---

That's what you would have to do in C code too, if you wanted to use that API and yes the whole stuff can get highly recursive too, as each variant can contain again attributes that contain variants as values. Not something that most people really would want to do.

[Peter_S]

---

@rolfk wrote:

The LabVIEW variant memory layout is private and undocumented. It is also subject to change between LabVIEW versions. The only safe way to access them across LabVIEW versions are the exported LvVariant....

Than it is not safe to used "Flatten To String.vi" for objects including variants beetween Apps of different Labview Runtime Versions?

Strange that this is not documented in http://zone.ni.com/reference/en-XX/help/371361P-01/glang/flatten_to_string/

For "data string" is written:

"Such a string can be stored in a file or sent over a network. .... Use the Unflatten From String function to convert a data string back to any data type."

[rolfk]

Flattened format is not the same as in memory layout!

When flattening data LabVIEW makes sure to use a standardized format on all LabVIEW platforms. That includes that data is BigEndian and certain datatypes are flattened to a different format than what it uses in native form. For instance the EXT floating point datatype is a 10 byte number on Windows, an 8 byte double float on most other platforms but it is flattened to a 16 byte long double format in flattened form. This is because there used to be a version of LabVIEW for the Sun SPARC CPU and while the CPU had no hardware extended floating point support, Sun did deliver the Solaris OS with a software emulation library that supported 16 byte long double floating point numbers. For all datatypes except Variants the memory format of the various datatypes hasn't really changed much in all those years, except that booleans were changed from 16 bit values to an 8 bit value between LabVIEW 4.x and 5.0 and that did also change the flattened format for booleans.

When adding variants they choose a different approach. While the memory layout can vary between different platforms, the flattened format is standardized.  But it contains a version number too, that indicates in which version it was created. And the Unflatten function can deal with older variant formats but obviously the opposite is not true. So creating a flattened stream in LabVIEW 2009 that contains variants can be read by a LabVIEW 2009 and newer version of LabVIEW with Unflatten from String. But you can not Unflatten a byte stream that was created in a newer version of LabVIEW, when it contains Variants (and by deference also Waveforms, as they can contain Variants too as attributes).

[Peter_S]

Hi Rolf, thank you for the clarification that memory layout is not flattend layout. Then I asked the wrong question.

I'm interested in the layout of "data string" of "Flatten to String.vi".

Is this documented for variant? For all other types it looks like the memory layout.

---

@rolfk wrote: But you can not Unflatten a byte stream that was created in a newer version of LabVIEW, when it contains Variants

---

That means for a streaming network:

• one sender (DAQ System -> Flatten to String -> udp multicast sender)
• many receiver (udp multicast receiver -> Unflatten from String)

The sender app can not upgraded with a newer Labview runtime version until all receiver apps are also upgraded.

Peter

[Peter_S]

I found older post related to "flattened format variant"

https://forums.ni.com/t5/LabVIEW/Why-is-a-flattened-variant-different-in-LabVIEW-2012-and-LabVIEW/td...

https://forums.ni.com/t5/LabVIEW/Format-of-the-variant-in-binary-file/td-p/2114958

The flattened format of variant seems not documented by NI but in the last post is a good sheet.