I'd like to suggest that the display resolution of the min, max and increment controls of the data entry property window be matched to the data type resolution.
I'm working a piece of equipment to test an avionics navigation system component. One of the components functions is computation and display of aircraft course and bearing. The units for these measurements are degrees and the resolution is 13 bits or 0.04394531250°. If I add a control to the front panel I can specify display format to allow double precision display (greater than the 13 bits required). If I add other controls to provide step and sweep rates and resolutions I can do that to double resolution as well.
The only parameters that I can’t view to the required 13 bits are associated with data entry. If I want to set the maximum value to 359.9560546875 (full scale minus 1 lsb), the minimum to 0, the increment size to 0.0439453125 (1 lsb), ‘coerce’ to max and min and ‘coerce to nearest ‘ increment size, the data entry displays automatically limit the displayed resolution. Why? All other parameters are viewable with up to ‘double’ precision if using doubles. The actual values used in the data entry coercion process are also double precision. But if I type 0.0439456125 or 0.0439453125 into the increment size control, 0.0439 is displayed and I get subtle errors with the former (incorrect) value. I can’t even look at the data entry window and see my mistake. I have to guess that I may have entered the wrong value and type it in again and check it carefully before removing focus from the increment entry.
Course/Bearing bit value table
1 |
180.00000000000 |
2 |
90.00000000000 |
3 |
45.00000000000 |
4 |
22.50000000000 |
5 |
11.25000000000 |
6 |
5.62500000000 |
7 |
2.81250000000 |
8 |
1.40625000000 |
9 |
0.70312500000 |
10 |
0.35156250000 |
11 |
0.17578125000 |
12 |
0.08789062500 |
13 |
0.04394531250 |
14 |
0.02197265625 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.