Re: Odd [math] behaviour

This WebDNA talk-list message is from

2003


It keeps the original formatting.
numero = 52773
interpreted = N
texte = Since my background is art rather than math ;-)... I'm haveing difficulty understanding this. On my calculator: If I take 3.2/3 I get 1.06 - repeating If I take 6000.4-5300.3 I get 700.1 How again does [math][/math] interpret this differently? sorry if my brain is thick. (I'll blaim it on the late 80's) Donovan John Peacock wrote: [snip] > It's your computer, not WebDNA. Floating point math is not exact; it is > approximate. Certain combinations will not ever work in the way you are > used to "on paper" so you just have to deal with it. > > I see that round() is not a valid function within the [math] (which is > dumb), so you'll have to brew your own: > > [math]floor((6000.4-5300.3+0.005)*100)/100[/math] > > will display what you would expect. > > NOTE: because you are using floating point, even the above may not work > all of the time, because some decimal representations are not possible > to accurately store using floating point numbers (which are stored > internally as a power of 2 not 10). > > HTH > > John > > -- =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o DONOVAN D. BROOKE Eucalyptus Design Developer / Consultant <-Web Development (specializing in eCommerce), -> <-Graphic Design, and Pre-Press Consultation -> ADDRESS:> Donovan Brooke DBA Eucalyptus Design P.O. Box 260153 Madison, WI 53726-0153 PH:> 1.608.592.3567 email:> dbrooke@euca.us Web:> http://www.euca.us =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Associated Messages, from the most recent to the oldest:

    
  1. Re: Odd [math] behaviour ( Tim Robinson 2003)
  2. Re: Odd [math] behaviour ( John Peacock 2003)
  3. Re: Odd [math] behaviour ( Paul Willis 2003)
  4. Re: Odd [math] behaviour ( Donovan 2003)
  5. Re: Odd [math] behaviour ( John Peacock 2003)
  6. Re: Odd [math] behaviour ( "WebDna @" 2003)
  7. Re: Odd [math] behaviour ( John Peacock 2003)
  8. Re: Odd [math] behaviour ( Donovan 2003)
  9. Re: Odd [math] behaviour ( "WebDna @" 2003)
  10. Re: Odd [math] behaviour ( Pat Holliday 2003)
  11. Re: Odd [math] behaviour ( "WebDna @" 2003)
  12. Re: Odd [math] behaviour ( John Peacock 2003)
  13. Odd [math] behaviour ( Pat Holliday 2003)
Since my background is art rather than math ;-)... I'm haveing difficulty understanding this. On my calculator: If I take 3.2/3 I get 1.06 - repeating If I take 6000.4-5300.3 I get 700.1 How again does [math][/math] interpret this differently? sorry if my brain is thick. (I'll blaim it on the late 80's) Donovan John Peacock wrote: [snip] > It's your computer, not WebDNA. Floating point math is not exact; it is > approximate. Certain combinations will not ever work in the way you are > used to "on paper" so you just have to deal with it. > > I see that round() is not a valid function within the [math] (which is > dumb), so you'll have to brew your own: > > [math]floor((6000.4-5300.3+0.005)*100)/100[/math] > > will display what you would expect. > > NOTE: because you are using floating point, even the above may not work > all of the time, because some decimal representations are not possible > to accurately store using floating point numbers (which are stored > internally as a power of 2 not 10). > > HTH > > John > > -- =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o DONOVAN D. BROOKE Eucalyptus Design Developer / Consultant <-Web Development (specializing in eCommerce), -> <-Graphic Design, and Pre-Press Consultation -> ADDRESS:> Donovan Brooke DBA Eucalyptus Design P.O. Box 260153 Madison, WI 53726-0153 PH:> 1.608.592.3567 email:> dbrooke@euca.us Web:> http://www.euca.us =o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o=o ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Donovan

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Imagemagick and fonts (2004) carriage returns in data (1997) I forgot (1998) carriage returns in data (1997) [SEARCH] Context for SKU prices not working... (1998) Re2: frames & carts (1997) Online reference (1997) OT : javascript form filling (2000) The Send Mail Button is the Best Troubleshooter (1998) Exclude by date - multiple (1997) WebDNA Quick Reference (Reserved Words) (2000) Explorer 3.0/ Access Denied! (1997) RE: Nested [SEARCH] and [REPLACE] problem. (1997) [include file=filename.inc&strip=t] (2002) [username][password] not showing up! HELP! (1999) WC1.6 to WC2 date formatting (1997) Authorizenet answers and more questions (2002) Problems with [Applescript] (1997) RAM variables (1997) WebCat2 beta 11 - new prefs ... (1997)