LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Numeric Indicator rounding data to the nearest integer

Solved!
Go to solution

I have a numeric indicator for a 2D array of doubles, and when I play with the Display Format in Properties for the indicator, I can adjust it to show 3 digits of precision. However on the indicator, I end up just getting something like 1548.000 . However, it didn't make sense to me that this number wouldn't have some amount of non zero decimal places, and when I probe the 2d array wire going into the indicator while running, I can see that the number should actually be something like 1548.315.... Is there some setting for rounding in the indicator that I have checked that I can't seem to find? (The indicator is set to U16) Any help would be greatly appreciated.

Colin

0 Kudos
Message 1 of 2
(2,385 Views)
Solution
Accepted by topic author GarbageCan

Of course I answer my own question as soon as I post it. I realized as I wrote down that the indicator is U16 that that was my problem, changing it to the double representation fixed it. Sorry for this needless topic.

0 Kudos
Message 2 of 2
(2,381 Views)