Re: [WebDNA] Math Precision Preshmilson

This WebDNA talk-list message is from

2019


It keeps the original formatting.
numero = 114956
interpreted = N
texte = 2584 My guess is this problem is due to the processor that uses = floating-point arithmetic, which means a rounding error can be = introduced in certain cases when given any fixed number of bits, a = calculation with real numbers will produce quantities that cannot be = exactly represented using that many bits. Therefore the result of a = floating-point calculation must often be rounded in order to fit back = into its finite representation. This rounding error is the = characteristic feature of floating-point computation. - chris > On Oct 31, 2019, at 01:24, Stuart Tremain = wrote: >=20 > That is very strange >=20 > I did a few versions with the same results on fcgi 8.5.1 >=20 > [math]5.4-6+0.0[/math] > [math]5.4-6+0.1[/math] > [math]5.4-6+0.2[/math] > [math]5.4-6+0.3[/math] > [math]5.4-6+0.4[/math] > [math]5.4-6+0.5[/math] =3D failed > [math]5.4-6+0.6[/math] =3D failed > [math]5.4-6+0.7[/math] > [math]5.4-6+0.8[/math] > [math]5.4-6+0.9[/math] >=20 >=20 > [math]5.0-6+0.6[/math] > [math]5.1-6+0.6[/math] > [math]5.2-6+0.6[/math] > [math]5.3-6+0.6[/math] > [math]5.4-6+0.6[/math] =3D failed > [math]5.5-6+0.6[/math] > [math]5.6-6+0.6[/math] > [math]5.7-6+0.6[/math] > [math]5.8-6+0.6[/math] > [math]5.9-6+0.6[/math] --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us . Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Math Precision Preshmilson (christophe.billiottet@webdna.us 2019)
  2. Re: [WebDNA] Math Precision Preshmilson (Stuart Tremain 2019)
  3. [WebDNA] Math Precision Preshmilson (Michael Davis 2019)
2584 My guess is this problem is due to the processor that uses = floating-point arithmetic, which means a rounding error can be = introduced in certain cases when given any fixed number of bits, a = calculation with real numbers will produce quantities that cannot be = exactly represented using that many bits. Therefore the result of a = floating-point calculation must often be rounded in order to fit back = into its finite representation. This rounding error is the = characteristic feature of floating-point computation. - chris > On Oct 31, 2019, at 01:24, Stuart Tremain = wrote: >=20 > That is very strange >=20 > I did a few versions with the same results on fcgi 8.5.1 >=20 > [math]5.4-6+0.0[/math] > [math]5.4-6+0.1[/math] > [math]5.4-6+0.2[/math] > [math]5.4-6+0.3[/math] > [math]5.4-6+0.4[/math] > [math]5.4-6+0.5[/math] =3D failed > [math]5.4-6+0.6[/math] =3D failed > [math]5.4-6+0.7[/math] > [math]5.4-6+0.8[/math] > [math]5.4-6+0.9[/math] >=20 >=20 > [math]5.0-6+0.6[/math] > [math]5.1-6+0.6[/math] > [math]5.2-6+0.6[/math] > [math]5.3-6+0.6[/math] > [math]5.4-6+0.6[/math] =3D failed > [math]5.5-6+0.6[/math] > [math]5.6-6+0.6[/math] > [math]5.7-6+0.6[/math] > [math]5.8-6+0.6[/math] > [math]5.9-6+0.6[/math] --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us . christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

all db's in one folder and protected (1998) WC2f3 (1997) Database Structure? (1998) [OT] friday pics :-) (2005) Applescript, WebDNA, and PDF (1997) wish list (2002) Suffix Mapping (1997) [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (2009) OT-Load balance (2004) [OT] Robust order processing (2003) Navigator 4.01 (1997) Re:2nd WebCatalog2 Feature Request (1996) Interfacing WebMerchant to www.fedex.com (1997) Calculating multiple shipping... (1997) SiteBuilder Problem (2004) [WebDNA] Need someone to port CakePHP auction site to WebDNA (2010) Number of Feilds Passed to [append] (2000) WebCat2b12--[searchstring] bug (1997) frames & carts (1997) Ignoring [OrderFile] context (2005)