Re: [WebDNA] Strange MATH result

This WebDNA talk-list message is from

2010


It keeps the original formatting.
numero = 104389
interpreted = N
texte = --00504502d56759fe8d047cf99342 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 12, 2010 at 5:33 AM, wrote: > Temporary solution is to specify how many decimal numbers you want, using > [format] > > > [format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format] > > - chris > > On Jan 12, 2010, at 2:17, Stuart Tremain wrote: > > > Why does [math]36.70+0.00+0.00+0.00-6.95-29.75[/math] = > 3.5527136788005e-015 ????? > The issue you have run in to is common with computer programming. I would categorize it as a computer "bug". http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems It is similar to other issues that you may run in to such as: 9223372036854775807 + 1 = -9223372036854775807 or 2147483647 + 1 = -2147483647 (2^63-1) + 1 = -(2&63) Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite. The solution is to use the format context, as Christophe stated. --00504502d56759fe8d047cf99342 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 12, 2010 at 5:33 AM, <christophe.billi= ottet@webdna.us> wrote:
Temporary solution is to specify how many decimal numbers you want, using [= format]


[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]

- chris

On Jan 12, 2010, at 2:17, Stuart Tremain wrote:

> Why does [math]36.70+0.00+0.00= +0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????
The issue you have run in to is common with computer programmi= ng.=A0 I would categorize it as a computer "bug".

h= ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647

(2^63-1) + 1 =3D -(2&63)

Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite.


The solution is to use the format context, as Christophe stated.


--00504502d56759fe8d047cf99342-- Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Strange MATH result (Stuart Tremain 2010)
  2. Re: [WebDNA] Strange MATH result (JJ WebDNA 2010)
  3. Re: [WebDNA] Strange MATH result - BUG (Govinda 2010)
  4. Re: [WebDNA] Strange MATH result (christophe.billiottet@webdna.us 2010)
  5. Re: [WebDNA] Strange MATH result (christophe.billiottet@webdna.us 2010)
  6. Re: [WebDNA] Strange MATH result - BUG (Stuart Tremain 2010)
  7. Re: [WebDNA] Strange MATH result - BUG (Govinda 2010)
  8. Re: [WebDNA] Strange MATH result (Stuart Tremain 2010)
  9. Re: [WebDNA] Strange MATH result (Govinda 2010)
  10. Re: [WebDNA] Strange MATH result - BUG (Stuart Tremain 2010)
  11. [WebDNA] Strange MATH result (Stuart Tremain 2010)
--00504502d56759fe8d047cf99342 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 12, 2010 at 5:33 AM, wrote: > Temporary solution is to specify how many decimal numbers you want, using > [format] > > > [format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format] > > - chris > > On Jan 12, 2010, at 2:17, Stuart Tremain wrote: > > > Why does [math]36.70+0.00+0.00+0.00-6.95-29.75[/math] = > 3.5527136788005e-015 ????? > The issue you have run in to is common with computer programming. I would categorize it as a computer "bug". http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems It is similar to other issues that you may run in to such as: 9223372036854775807 + 1 = -9223372036854775807 or 2147483647 + 1 = -2147483647 (2^63-1) + 1 = -(2&63) Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite. The solution is to use the format context, as Christophe stated. --00504502d56759fe8d047cf99342 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 12, 2010 at 5:33 AM, <christophe.billi= ottet@webdna.us> wrote:
Temporary solution is to specify how many decimal numbers you want, using [= format]


[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]

- chris

On Jan 12, 2010, at 2:17, Stuart Tremain wrote:

> Why does [math]36.70+0.00+0.00= +0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????
The issue you have run in to is common with computer programmi= ng.=A0 I would categorize it as a computer "bug".

h= ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647

(2^63-1) + 1 =3D -(2&63)

Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite.


The solution is to use the format context, as Christophe stated.


--00504502d56759fe8d047cf99342-- JJ WebDNA

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Two stores, one server (1998) WebCat2 - [include] tags (1997) select multiple 2 more cents (1997) SKU (1997) REPOST: Upgrage pricing... The future of WebCatalog is coming with 4.0... (2000) Number of Open Databases (1998) [WebDNA] reverse rank=off (2008) SetMIMEHeader Behaviour (2000) Emailer errors (1997) Opinion: [input] should be called [output] ... (1997) practicing safe queries.. (2000) [WriteFile] problems (1997) Email Set-Up? (1997) HTML docs are wrong about listwords ... (2000) Bug when including a db file ... ? (2000) Webcatalog quitting (1997) Malformed Pages (1999) Trouble with Netscape (1998) japanese characters (1997) No messages for a week? (2000)