Re: Odd [math] behaviour
This WebDNA talk-list message is from 2003
It keeps the original formatting.
numero = 52773
interpreted = N
texte = Since my background is art rather than math ;-)...I'm haveing difficulty understanding this.On my calculator:If I take 3.2/3 I get 1.06 - repeatingIf I take 6000.4-5300.3 I get 700.1How again does [math][/math] interpret this differently?sorry if my brain is thick.(I'll blaim it on the late 80's)DonovanJohn Peacock wrote:[snip]> It's your computer, not WebDNA. Floating point math is not exact; it is > approximate. Certain combinations will not ever work in the way you are > used to "on paper" so you just have to deal with it.> > I see that round() is not a valid function within the [math] (which is > dumb), so you'll have to brew your own:> > [math]floor((6000.4-5300.3+0.005)*100)/100[/math]> > will display what you would expect.> > NOTE: because you are using floating point, even the above may not work > all of the time, because some decimal representations are not possible > to accurately store using floating point numbers (which are stored > internally as a power of 2 not 10).> > HTH> > John> > -- =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o DONOVAN D. BROOKE Eucalyptus Design Developer / Consultant <-Web Development (specializing in eCommerce), -> <-Graphic Design, and Pre-Press Consultation -> ADDRESS:> Donovan Brooke DBA Eucalyptus Design P.O. Box 260153 Madison, WI 53726-0153 PH:> 1.608.592.3567 email:> dbrooke@euca.us Web:> http://www.euca.us =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o-------------------------------------------------------------This message is sent to you because you are subscribed to the mailing list
.To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/
Associated Messages, from the most recent to the oldest:
Since my background is art rather than math ;-)...I'm haveing difficulty understanding this.On my calculator:If I take 3.2/3 I get 1.06 - repeatingIf I take 6000.4-5300.3 I get 700.1How again does [math][/math] interpret this differently?sorry if my brain is thick.(I'll blaim it on the late 80's)DonovanJohn Peacock wrote:[snip]> It's your computer, not WebDNA. Floating point math is not exact; it is > approximate. Certain combinations will not ever work in the way you are > used to "on paper" so you just have to deal with it.> > I see that round() is not a valid function within the [math] (which is > dumb), so you'll have to brew your own:> > [math]floor((6000.4-5300.3+0.005)*100)/100[/math]> > will display what you would expect.> > NOTE: because you are using floating point, even the above may not work > all of the time, because some decimal representations are not possible > to accurately store using floating point numbers (which are stored > internally as a power of 2 not 10).> > HTH> > John> > -- =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o DONOVAN D. BROOKE Eucalyptus Design Developer / Consultant <-Web Development (specializing in eCommerce), -> <-Graphic Design, and Pre-Press Consultation -> ADDRESS:> Donovan Brooke DBA Eucalyptus Design P.O. Box 260153 Madison, WI 53726-0153 PH:> 1.608.592.3567 email:> dbrooke@euca.us Web:> http://www.euca.us =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o-------------------------------------------------------------This message is sent to you because you are subscribed to the mailing list .To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/
Donovan
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Imagemagick and fonts (2004)
carriage returns in data (1997)
I forgot (1998)
carriage returns in data (1997)
[SEARCH] Context for SKU prices not working... (1998)
Re2: frames & carts (1997)
Online reference (1997)
OT : javascript form filling (2000)
The Send Mail Button is the Best Troubleshooter (1998)
Exclude by date - multiple (1997)
WebDNA Quick Reference (Reserved Words) (2000)
Explorer 3.0/ Access Denied! (1997)
RE: Nested [SEARCH] and [REPLACE] problem. (1997)
[include file=filename.inc&strip=t] (2002)
[username][password] not showing up! HELP! (1999)
WC1.6 to WC2 date formatting (1997)
Authorizenet answers and more questions (2002)
Problems with [Applescript] (1997)
RAM variables (1997)
WebCat2 beta 11 - new prefs ... (1997)