Re: [WebDNA] Strange MATH result

This WebDNA talk-list message is from

2010


It keeps the original formatting.
numero = 104389
interpreted = N
texte = --00504502d56759fe8d047cf99342 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 12, 2010 at 5:33 AM, wrote: > Temporary solution is to specify how many decimal numbers you want, using > [format] > > > [format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format] > > - chris > > On Jan 12, 2010, at 2:17, Stuart Tremain wrote: > > > Why does [math]36.70+0.00+0.00+0.00-6.95-29.75[/math] = > 3.5527136788005e-015 ????? > The issue you have run in to is common with computer programming. I would categorize it as a computer "bug". http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems It is similar to other issues that you may run in to such as: 9223372036854775807 + 1 = -9223372036854775807 or 2147483647 + 1 = -2147483647 (2^63-1) + 1 = -(2&63) Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite. The solution is to use the format context, as Christophe stated. --00504502d56759fe8d047cf99342 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 12, 2010 at 5:33 AM, <christophe.billi= ottet@webdna.us> wrote:
Temporary solution is to specify how many decimal numbers you want, using [= format]


[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]

- chris

On Jan 12, 2010, at 2:17, Stuart Tremain wrote:

> Why does [math]36.70+0.00+0.00= +0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????
The issue you have run in to is common with computer programmi= ng.=A0 I would categorize it as a computer "bug".

h= ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647

(2^63-1) + 1 =3D -(2&63)

Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite.


The solution is to use the format context, as Christophe stated.


--00504502d56759fe8d047cf99342-- Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Strange MATH result (Stuart Tremain 2010)
  2. Re: [WebDNA] Strange MATH result (JJ WebDNA 2010)
  3. Re: [WebDNA] Strange MATH result - BUG (Govinda 2010)
  4. Re: [WebDNA] Strange MATH result (christophe.billiottet@webdna.us 2010)
  5. Re: [WebDNA] Strange MATH result (christophe.billiottet@webdna.us 2010)
  6. Re: [WebDNA] Strange MATH result - BUG (Stuart Tremain 2010)
  7. Re: [WebDNA] Strange MATH result - BUG (Govinda 2010)
  8. Re: [WebDNA] Strange MATH result (Stuart Tremain 2010)
  9. Re: [WebDNA] Strange MATH result (Govinda 2010)
  10. Re: [WebDNA] Strange MATH result - BUG (Stuart Tremain 2010)
  11. [WebDNA] Strange MATH result (Stuart Tremain 2010)
--00504502d56759fe8d047cf99342 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 12, 2010 at 5:33 AM, wrote: > Temporary solution is to specify how many decimal numbers you want, using > [format] > > > [format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format] > > - chris > > On Jan 12, 2010, at 2:17, Stuart Tremain wrote: > > > Why does [math]36.70+0.00+0.00+0.00-6.95-29.75[/math] = > 3.5527136788005e-015 ????? > The issue you have run in to is common with computer programming. I would categorize it as a computer "bug". http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems It is similar to other issues that you may run in to such as: 9223372036854775807 + 1 = -9223372036854775807 or 2147483647 + 1 = -2147483647 (2^63-1) + 1 = -(2&63) Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite. The solution is to use the format context, as Christophe stated. --00504502d56759fe8d047cf99342 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 12, 2010 at 5:33 AM, <christophe.billi= ottet@webdna.us> wrote:
Temporary solution is to specify how many decimal numbers you want, using [= format]


[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]

- chris

On Jan 12, 2010, at 2:17, Stuart Tremain wrote:

> Why does [math]36.70+0.00+0.00= +0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????
The issue you have run in to is common with computer programmi= ng.=A0 I would categorize it as a computer "bug".

h= ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647

(2^63-1) + 1 =3D -(2&63)

Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite.


The solution is to use the format context, as Christophe stated.


--00504502d56759fe8d047cf99342-- JJ WebDNA

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Notepad problems - SOLVED! (1998) BUG: Switch inside ListFields context (2005) Tag Corruption (2000) [SHOWIF AND/OR] (1997) [WebDNA] Mac OS Intel version? (2008) WebCat2 several catalogs? (1997) Sort Order on a page search (1997) Bad Cookie / Internet Option / Internet Explorer (2004) Re:multiple digests, please stop (1997) Re:quit command on NT (1997) How much is too much? I can never remember the answer. (2002) Multiple prices (1997) Image maps/Webcat (1997) Major problem (1999) [date format] w/in sendmail (1997) Nav. 4 probs with cart - Serious problem (1997) form data submission gets truncated (1997) Two prices in shoppingcart? (1997) convert date and time (2005) WCS Newbie question (1997)