Re: [WebDNA] Strange MATH result

This WebDNA talk-list message is from

2010


It keeps the original formatting.
numero = 104389
interpreted = N
texte = --00504502d56759fe8d047cf99342 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 12, 2010 at 5:33 AM, wrote: > Temporary solution is to specify how many decimal numbers you want, using > [format] > > > [format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format] > > - chris > > On Jan 12, 2010, at 2:17, Stuart Tremain wrote: > > > Why does [math]36.70+0.00+0.00+0.00-6.95-29.75[/math] = > 3.5527136788005e-015 ????? > The issue you have run in to is common with computer programming. I would categorize it as a computer "bug". http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems It is similar to other issues that you may run in to such as: 9223372036854775807 + 1 = -9223372036854775807 or 2147483647 + 1 = -2147483647 (2^63-1) + 1 = -(2&63) Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite. The solution is to use the format context, as Christophe stated. --00504502d56759fe8d047cf99342 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 12, 2010 at 5:33 AM, <christophe.billi= ottet@webdna.us> wrote:
Temporary solution is to specify how many decimal numbers you want, using [= format]


[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]

- chris

On Jan 12, 2010, at 2:17, Stuart Tremain wrote:

> Why does [math]36.70+0.00+0.00= +0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????
The issue you have run in to is common with computer programmi= ng.=A0 I would categorize it as a computer "bug".

h= ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647

(2^63-1) + 1 =3D -(2&63)

Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite.


The solution is to use the format context, as Christophe stated.


--00504502d56759fe8d047cf99342-- Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Strange MATH result (Stuart Tremain 2010)
  2. Re: [WebDNA] Strange MATH result (JJ WebDNA 2010)
  3. Re: [WebDNA] Strange MATH result - BUG (Govinda 2010)
  4. Re: [WebDNA] Strange MATH result (christophe.billiottet@webdna.us 2010)
  5. Re: [WebDNA] Strange MATH result (christophe.billiottet@webdna.us 2010)
  6. Re: [WebDNA] Strange MATH result - BUG (Stuart Tremain 2010)
  7. Re: [WebDNA] Strange MATH result - BUG (Govinda 2010)
  8. Re: [WebDNA] Strange MATH result (Stuart Tremain 2010)
  9. Re: [WebDNA] Strange MATH result (Govinda 2010)
  10. Re: [WebDNA] Strange MATH result - BUG (Stuart Tremain 2010)
  11. [WebDNA] Strange MATH result (Stuart Tremain 2010)
--00504502d56759fe8d047cf99342 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 12, 2010 at 5:33 AM, wrote: > Temporary solution is to specify how many decimal numbers you want, using > [format] > > > [format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format] > > - chris > > On Jan 12, 2010, at 2:17, Stuart Tremain wrote: > > > Why does [math]36.70+0.00+0.00+0.00-6.95-29.75[/math] = > 3.5527136788005e-015 ????? > The issue you have run in to is common with computer programming. I would categorize it as a computer "bug". http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems It is similar to other issues that you may run in to such as: 9223372036854775807 + 1 = -9223372036854775807 or 2147483647 + 1 = -2147483647 (2^63-1) + 1 = -(2&63) Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite. The solution is to use the format context, as Christophe stated. --00504502d56759fe8d047cf99342 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 12, 2010 at 5:33 AM, <christophe.billi= ottet@webdna.us> wrote:
Temporary solution is to specify how many decimal numbers you want, using [= format]


[format 6.2f][math]36.70+0.00+0.00+0.00-6.95-29.75[/math][/format]

- chris

On Jan 12, 2010, at 2:17, Stuart Tremain wrote:

> Why does [math]36.70+0.00+0.00= +0.00-6.95-29.75[/math] =3D 3.5527136788005e-015 ?????
The issue you have run in to is common with computer programmi= ng.=A0 I would categorize it as a computer "bug".

h= ttp://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

It is similar to other issues that you may run in to such as:
9223372036854775807 + 1 =3D -9223372036854775807
or
2147483647 + 1 =3D -2147483647

(2^63-1) + 1 =3D -(2&63)

Computers have certain amounts of precision, and sometimes subtracting a number from itself may result in a number that is very nearly zero, but not quite.


The solution is to use the format context, as Christophe stated.


--00504502d56759fe8d047cf99342-- JJ WebDNA

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Stopping bad HTML propagation ? (1997) [OT] HTML Table Cell Problem (2003) WebCat for mass emailings (1997) 2.1 bugs left over from beta cycle (1998) textA (1998) [item] appears after 'remove last item' ... (1997) Me know logical no need (2002) Email template names (1997) Tracking inbound links for discounts (1997) World Address Info (2003) Cobalt RaQ2 and WebCatalog: who uses it? (2000) Adding Multiple Items (1999) displaying New products (Yikes! it's Fixed!) (1997) emailer error -108 (1997) formvariables exact=F (2004) Cool new site (1996) [WriteFile] problems (1997) PCS Emailer's role ? (1997) [format] problems (1998) W* V4.2 SSL and webcat very slow (2000)