Remove scientific notation in report

[JDeckert]

I'm calculating a number and placing it in a comment box on a report. This works fine, but one of the numbers has decided to display in scientific notation (1.52666667e-3). The problem is that the comment box is not large enough to display the scientific notation and gives no indication of not fitting into the box. Instead I see 1.52666 with no indication of the exponent. I'd much rather see .001527 however I don't want to force a STR format with "d.dddddd" because most numbers don't need this much precision and I'd probably run into errors for numbers such as 12345 which won't fit into the comment box with forced 6 digit precision. If I can force it out of scientific precision, it'll probably be fine.

BTW: is there a certain number range where scientific notation becomes the default? If I knew this range I could then force the format with the STR function to display the way I want. e.g. IF x>-001 and x<.001 then STR(X,"d.dddd")

Either method would probably work for me.

thanks,

James

[freighTrain]

Hi JDeckert,

I'm trying to replicate this and I've tried a number that is smaller than the one you mentioned but it was displaying in decimal format. How are you displaying the number? Are you using AutoAdj at all?

[JDeckert]

I've attached the clip of code.

This code creates a column of Upper Bounds.

thanks for your input. No I haven't tried autoadj.

[freighTrain]

I'm not sure what kind of value is returned from MyVar but does changing the comment box size or text alignment make any difference? Also, was there a reason for setting the comment text so many times?

[JDeckert]

MyVar is a 2 dimensional pointer (Channel Group, Channel) to numeric channels that meet some criteria.

change box size: I had to make the box 3 times the width to display the number correctly. It shows 1.27933333333333E-03*. I'm appending the "*" to show the field is calculated instead of unadultered from the .csv. Maybe the problem isn't that the number is small, but that it's infinitely repeating. The number is calculate from the values .001919 - abs(.001919)/3

Left Justify: no difference from right justify.

Autoadj: shows 1.28E-

I'm making a column of channels on the report, that's why the comment text is being set multiple times.

Humphreyy - thanks for your ideas and help.

[freighTrain · Accepted solution]

Okay - I'm also seeing DIAdem auto formating repeating decimals to scientific notation. I guess what you could do is check the number of digits the value has and if it exceeds a certain number, format it with d.ddd.

So, something along the lines of: @@IIF(Len(valueToCheck)<10, valueToCheck, Str(valueToCheck, "d.ddd"))@@

[JDeckert]

Ok, good idea.

thanks, I'm glad you could reproduce it.

James