Temporary solution is to specify how many decimal numbers you want, using [=format]
[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]
- chris
On Jan 12, 2010, at 2:17, Stuart Tremain wrote:> Why does [math]36.70+0.00+0.00=+0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????The issue you have run in to is common with computer programmi=ng.=A0 I would categorize it as a computer "bug".
h=ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647
(2^63-1) + 1 =3D -(2&63)
Computers have certain amounts of precision, and sometimes subtractinga number from itself may result in a number that is very nearly zero,but not quite.
The solution is to use the format context, as Christophe stated.
|
Temporary solution is to specify how many decimal numbers you want, using [=format]
[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]
- chris
On Jan 12, 2010, at 2:17, Stuart Tremain wrote:> Why does [math]36.70+0.00+0.00=+0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????The issue you have run in to is common with computer programmi=ng.=A0 I would categorize it as a computer "bug".
h=ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647
(2^63-1) + 1 =3D -(2&63)
Computers have certain amounts of precision, and sometimes subtractinga number from itself may result in a number that is very nearly zero,but not quite.
The solution is to use the format context, as Christophe stated.
DOWNLOAD WEBDNA NOW!
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...