Display resolution of the min, max and increment controls of the data entry property window should be matched to the data type resolution.

[fr@nk]

I'd like to suggest that the display resolution of the min, max and increment controls of the data entry property window be matched to the data type resolution. 

I'm working a piece of equipment to test an avionics navigation system component. One of the components functions is computation and display of aircraft course and bearing. The units for these measurements are degrees and the resolution is 13 bits or 0.04394531250°. If I add a control to the front panel I can specify display format to allow double precision display (greater than the 13 bits required). If I add other controls to provide step and sweep rates and resolutions I can do that to double resolution as well.

The only parameters that I can’t view to the required 13 bits are associated with data entry. If I want to set the maximum value to 359.9560546875 (full scale minus 1 lsb), the minimum to 0, the increment size to 0.0439453125 (1 lsb), ‘coerce’ to max and min and  ‘coerce to nearest ‘ increment size, the data entry displays automatically limit the displayed resolution. Why? All other parameters are viewable with up to ‘double’ precision if using doubles. The actual values used in the data entry coercion process are also double precision. But if I type 0.0439456125 or  0.0439453125 into the increment size control, 0.0439 is displayed and I get subtle errors with the former (incorrect) value. I can’t even look at the data entry window and see my mistake. I have to guess that I may have entered the wrong value and type it in again and check it carefully before removing focus from the increment entry.

Course/Bearing bit value table

1	180.00000000000
2	90.00000000000
3	45.00000000000
4	22.50000000000
5	11.25000000000
6	5.62500000000
7	2.81250000000
8	1.40625000000
9	0.70312500000
10	0.35156250000
11	0.17578125000
12	0.08789062500
13	0.04394531250
14	0.02197265625

[PhillipBrooks]

Kudoed. I'm not sure how the dialog would determine how many digits to display; maybe using the Display Format settings as a 'hint'?

If you need to use a control with these specific properties in multiple places, you might consider using a strict typdef control.

If your "unit under test" changes at a later point to use 14 bits of precision, you can edit the strict typedef settings once and all your VIs that use the typedef will use the new step size.

[fr@nk]

I don't need a control to display the max/min/increment parameters. I need to be able to see the parameters displayed with the same resolution as the control object the parameters are methods of. Using the display format setting would be the way I would try to implement it. It seems that this setting affects all of the other parametric settings, just not min, max and increment.

Frank

[johnsold]

Either use Frank's suggestion of formatting the values in the dialog according to the format of the control or display everything the programmer enters, at least to the limit for the datatype. This would be analogous to the Adapt to Entered Data mode for numeric constants, except that it would apply to formatting rather than datatype.

Lynn

[viScience]

Funny, I am also working on a Avionics application and in frustration over not being able to set a Numeric Increment of .0078125

(it gets coorced to .0078) I search ni.com and found this idea.

[X.]

@sachsm: You actually CAN set the numeric increment to that value. It just does not display it to the necessary precision.

At least that's how it works in LV 2012.

It is bordering to a bug, imo.

[Manzolli]

Can you use a control with a U16 Representation, Data Entry minimum = 0 and maximum = 8191 then do the math: bearing = control * 360 / 8192 ?

An Integer can easily be converted to a float, but the opposite direction can be problematic.