Re: Math Bug
This WebDNA talk-list message is from 1999
It keeps the original formatting.
numero = 25466
interpreted = N
texte = This is rather likely to be due the nature of floating point math. Internal to the computer, all floating point numbers are stored as binary (i.e. based on power of 2) numbers, and certain fixed point decimal numbers _cannot_ be represented exactly. It will tend to happen more with larger decimal places or with larger numbers. The truly amazing thing is that the effect will not be exactly the same for Mac vs PC vs Unix, since the exact internal binary representation varies by CPU. Some CPU's will use BCD (binary coded decimal) under some circumstances, which do not have the problem with floating point notation errors.The only way to completely deal with this problem is to use strictly integer math, with an implied number of decimal places. For example, the number stored would be the integer 7992424, but it would be displayed 79924.24. A nice side effect of this is that all math functions on integers are 4-12 times faster than the same function on floating point values.Unfortunately, WebCat is not a strongly typed language, which means there is no way to make a particular value be stored as an integer vs a floting point number. You could try and multiply all of your field values by 100 (assumming you are using dollars/cents) so that there are no decimals. Then you could add /100 to all displayed values, which would restore the decimal points. Even this is not 100%, because of the floating point notation problem.If you are lucky, WebCat's math library will treat the numbers as integers, but there is no way for anyone to know (except Grant). It would be nice if we could give WebCat a field defination which would include what type of field it is. Currency could be an internal integer, external fixed point variable. Maybe that can get squeezed into version 5.0. how about a .hdr file with two lines, the second of which contains datatype definitions in the form currency, floatingpoint, string, integer.I wrote a module in Perl called Math::Currency, which deals with this very problem. I store the values as huge integers, and at the point of display, I actually treat it as a string and insert a decimal for display. I plan on expanding it to support fixed point math of arbitrary decimal places, in my copius free time :( Sadly, WebCat does not support inline Perl calls yet ;)Hope this explains everything...John Peacock ____________________Reply Separator____________________ Subject: Math Bug Author:
(WebCatalog Talk) Date: 12/13/99 4:19 PMI've run across a math bug when summarizing a large number of records using this technique: [some search context] [math][founditems][commission]+[/founditems]0[/math] [/search]Once the search hits about the 130-135th record the total which should look like: 79924.24 instead gets formatted as 79924.2399999999. I've zapped gremlins, reloaded the database, changed the startat value, verified that all values are integers (all are formatted x.xx ), even modified the contents of the records with different values and still the computed value gets formatted as .xx99999999.As a further test I created a test db and wrote out the commission values from the problem db to the test db and the results are the same - although the problem begins at the 125th record - *does not* appear in the 126th record but comes back at the 127th record. What the heck is going on here?Running 3.04PI w/Web* 4.0 on a G4 w/OS 8.6 Thanks Marty Schmid------------------------------------------------------------- Brought to you by CommuniGate Pro - The Buzz Word Compliant Messaging Server. To end your Mail problems go to .This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to
Associated Messages, from the most recent to the oldest:
This is rather likely to be due the nature of floating point math. Internal to the computer, all floating point numbers are stored as binary (i.e. based on power of 2) numbers, and certain fixed point decimal numbers _cannot_ be represented exactly. It will tend to happen more with larger decimal places or with larger numbers. The truly amazing thing is that the effect will not be exactly the same for Mac vs PC vs Unix, since the exact internal binary representation varies by CPU. Some CPU's will use BCD (binary coded decimal) under some circumstances, which do not have the problem with floating point notation errors.The only way to completely deal with this problem is to use strictly integer math, with an implied number of decimal places. For example, the number stored would be the integer 7992424, but it would be displayed 79924.24. A nice side effect of this is that all math functions on integers are 4-12 times faster than the same function on floating point values.Unfortunately, WebCat is not a strongly typed language, which means there is no way to make a particular value be stored as an integer vs a floting point number. You could try and multiply all of your field values by 100 (assumming you are using dollars/cents) so that there are no decimals. Then you could add /100 to all displayed values, which would restore the decimal points. Even this is not 100%, because of the floating point notation problem.If you are lucky, WebCat's math library will treat the numbers as integers, but there is no way for anyone to know (except Grant). It would be nice if we could give WebCat a field defination which would include what type of field it is. Currency could be an internal integer, external fixed point variable. Maybe that can get squeezed into version 5.0. how about a .hdr file with two lines, the second of which contains datatype definitions in the form currency, floatingpoint, string, integer.I wrote a module in Perl called Math::Currency, which deals with this very problem. I store the values as huge integers, and at the point of display, I actually treat it as a string and insert a decimal for display. I plan on expanding it to support fixed point math of arbitrary decimal places, in my copius free time :( Sadly, WebCat does not support inline Perl calls yet ;)Hope this explains everything...John Peacock ____________________Reply Separator____________________ Subject: Math Bug Author: (WebCatalog Talk) Date: 12/13/99 4:19 PMI've run across a math bug when summarizing a large number of records using this technique: [some search context] [math][founditems][commission]+[/founditems]0[/math] [/search]Once the search hits about the 130-135th record the total which should look like: 79924.24 instead gets formatted as 79924.2399999999. I've zapped gremlins, reloaded the database, changed the startat value, verified that all values are integers (all are formatted x.xx ), even modified the contents of the records with different values and still the computed value gets formatted as .xx99999999.As a further test I created a test db and wrote out the commission values from the problem db to the test db and the results are the same - although the problem begins at the 125th record - *does not* appear in the 126th record but comes back at the 127th record. What the heck is going on here?Running 3.04PI w/Web* 4.0 on a G4 w/OS 8.6 Thanks Marty Schmid------------------------------------------------------------- Brought to you by CommuniGate Pro - The Buzz Word Compliant Messaging Server. To end your Mail problems go to .This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to
jpeacock@univpress.com
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Search/sort in URL Was: GuestBook example (1997)
Conditional searching & displaying (1997)
How true is this? (1999)
too many nested [xxx] (1997)
OT: Poll Please (2002)
Hard Questions ? (1997)
RE: Signal Raised (1997)
Re:Off Topic: Frames Killer? (1998)
Emailer (WebCat2) (1997)
WebCatalog 402rc2 is now available (2001)
default error on NT? (1997)
New NT beta available (1997)
Re:Need help... (1997)
Authenticate (1999)
WebCatalog 2.0 & WebDNA docs in HTML ... (1997)
Webcat 2.0.1b1 bug with IE 3.01/4.0p1 (1997)
Weird problems with [SHOWIF]s (1997)
WebCat2 Append problem (B14Macacgi) (1997)
Desperatly seeking (2000)
[CART] (1997)