Problem with Math.floor

[sbinder]

I've noticed a problem with the Math.floor function in OL 4.2. I know this is an issue in swf8 - haven't tried it in other platforms yet.

You can replicate the issue by setting a variable to (i.e. v) to "7.12". Then look at Math.floor(100 * v). You'll get 711 instead of 712. You can also see the problem if you type Math.floor( 100 * "7.12" ) into the debugger.

Obviously, the problem has to do with the floating point representation of the string "7.12". If you take Math.floor( 100 * 7.12 ) you get the correct answer (712). If you set a varible to 100 * "7.12" and Debug.write the variable, it prints as 7.12. But then if you take Math.floor() of the variable - you get 711. Does anybody have any suggestions as to a workaround? the only thing I can think of is something like (horrifying):
Math.floor( ( 100 * "7.12") + 0.000001 ).

It should work correctly if the input number (i.e. 7.12) is guaranteed to be only a few decimal places long. But it's really, really ugly.

[pal]

What about 'parseFloat(...)'?

[ramesh]

Sbinder,

I tested the above case on the nightly builds and resulte are below,

swf8 - result is 711
swf9 - result is 712
swf10 - result is 712
DHTML - result is 712

In the swf8, if we assign the variable with quotes then its showing the wrong result(above). The below way of assignment provide the correct result on the swf8,

var v=7.12;
Debug.write(Math.floor(100*v));

waiting for the expert answers!

Thanks and Regards,
Ramesh G.

[sbinder]

The problem is that the input value is being read from an edittext component - so using a literal number is not really an option. I showed the input value as a literal string in the example above because it's the easiest way to reproduce the error.

[senshi]

Seems to be an issue with the IEEE 754 (http://en.wikipedia.org/wiki/IEEE_754-1985) representation of floating point numbers in swf8. You could use "Math.round" instead of "Math.floor".

[ramesh]

It looks like parseFloat doesn't solve the problem and Math.round is doing the magic!

Thanks and Regards,
Ramesh G.