Re: [WebDNA] An unknown error occured // Deadlock avoided
This WebDNA talk-list message is from 2011
It keeps the original formatting.
numero = 106920
interpreted = N
texte = Chris, yes, you are right; That makes sense to word it more like that = ("speed improvement"... "give control back to the admin"). I only said = "bug" because we have sometimes talked about the issue with the word = "bug" on this list in recent past... and because the whole point of the = admin pref seems to imply giving control to the admin.. and when it did = not (prior to v.7) then the pref is false advertizing.. and then "bug" = seems to fit. Anyway, Tommy, be sure you check what you have set for that pref. If = set to "auto-commit" to disk, then the workaround I mentioned will not = help anything (because then webdna will still be writing to disk on = every operation). Note I have solved crashing problems more than once where legacy code = had an [append] inside a loop ([loop], or [founditems], etc.) that = looped more than a (couple +/-) thousand times... by switching to the = [replace... &append=3DT] workaround. Credit goes to Matt Perosi for = pointing the issue out to me in the first place.=20 Tommy, but do heed Chris' point that if you turn off auto-commit and you = also do not add code to commit to disk once in a while, say once an = hour, or once per day, or something.. then your data is just floating in = RAM and in case of crash, then POOF!.. it's all gone! On 2011-07-11, at 10:31 AM, christophe.billiottet@webdna.us wrote: > Govinda, interesting workaround using [replace]. Note the [append] tag = systematically writing to disk it is not a bug, it had been implemented = this way by SmithMicro, probably as a safety measure. If you check = "Only commit databases to disk when instructed" in the WebDNA = administration and you have a crash, data not written to disk WILL be = lost...=20 >=20 > We just thought in version 7.0 that we could re-enable administrator = control on this point. >=20 > - chris >=20 >=20 >=20 > On Jul 11, 2011, at 11:05, Govinda wrote: >=20 >> Tommy >>=20 >> If you are using any version of webdna less than version 7, then = please note that there is a webdna bug that causes [append] tags to = write to disk, even if the admin pref. has been set to not do that. =20 >> Normally (unless the admin pref has been set to auto-commit) webdna = just writes any new record to the *RAM copy* of the db.. and your disk = copy does not get updated to match the RAM copy until you do a = [closedatabase] or [commitdatabase]. But because of the bug, before = webdna version 7, [append] writes to the disk copy constantly.. meaning = that appends are much slower than they should be... =20 >>=20 >> The workaround for this, which I recommend you implement (in case you = do keep the below 'topbanner.inc' code you posted), is to use a = [replace...] with the '&append=3DT' param (instead of the [append]). In = the [replace...] params, you just search for a record which you *know* = will not exist - to force the append operation... and since [replace..] = does not have the same bug, your operation (essentially an append, same = as before) stays in RAM - nice and fast. >>=20 >> Some thing like this: >>=20 >> replace this: >> [append db=3D[path]../../database/counter.db]DB_DATO=3D[date = %Y/%m/%d]&DB_ANTAL=3D1[/append] >>=20 >> ...with this: >> [Replace = db=3D[path]../../database/counter.db&eqDB_DATOdatarq=3DdontFind2ForceAppen= dHaveAniceDay]DB_DATO=3D[date %Y/%m/%d]&DB_ANTAL=3D1[/Replace] >>=20 >>=20 >> ...and do the same kind of thing for the [append] to 'online.db' . >>=20 >>=20 >> I have no idea if implementing this will help your situation.. but it = might relieve some pressure .. and if your issue is a phantom one = (meaning not consistent on every firing of the said problem code (if we = even know where that problem code is)), then just relieving some = pressure could give you months or years of wiggle room to address the = real deeper issue(s). >>=20 >> - Govinda >> -------------- >> Old WebDNA talklist archives: >> http://dev.webdna.us/TalkListArchive/index.tpl?db=3Dwebdna-talk >>=20 >>=20 >> On 2011-07-11, at 4:03 AM,
= wrote: >>=20 >>> [snip] >>=20 >>> Only two places is the online.db file used. >>>=20 >>> One is just a search with nothing else done to it. The other appends = or >>> replaces in online.db I am guessing this is where the problem is. = Despite >>> this appearing to be a windows problem, I hope someone will look at = this >>> and perhaps comment if it is even sane to do something like this. >>>=20 >>> topbanner.inc >>> [search db=3D[path]../../database/online.db&eqDB_IPdatarq=3D[IPAddress= ]] >>> [showIf [numFound]=3D0] >>> [append >>> = db=3D[path]../../database/online.db]DB_IP=3D[IPAddress]&DB_DATO=3D[DATE]&D= B_TID=3D[time]&DB_URL=3D[url][ThisURL]?a=3Db[FormVariables][hideIf >>> = [name]=3Dtemplate]&[name]=3D[value][/hideIf][/FormVariables][/url]&DB_KUND= EID=3D[kunde]&DB_BROWSER=3D[url][browser][/url]&DB_URL_REF=3D[url][REFERRE= R][/url][/append] >>> [search db=3D[path]../../database/counter.db&eqDB_DATOdata=3D[date= %Y/%m/%d]] >>> [showIf [numFound]<1] >>> [append db=3D[path]../../database/counter.db]DB_DATO=3D[da= te >>> %Y/%m/%d]&DB_ANTAL=3D1[/append] >>> [/showIf] >>> [showIf [numFound]>0] >>>=20 >>> = [replacefoundItems]DB_ANTAL=3D[math][DB_ANTAL]+1[/math][/replacefoundItems= ] >>> [/showIf] >>> [/search] >>> [/showIf] >>> [showIf [numFound]=3D1] >>> [replace >>> = db=3D[path]../../database/online.db&eqDB_IPdatarq=3D[IPAddress]]DB_DATO=3D= [DATE]&DB_TID=3D[time]&DB_URL=3D[url][ThisURL]?a=3Db[FormVariables][hideIf= >>> = [name]=3Dtemplate]&[name]=3D[value][/hideIf][/FormVariables][/url]&DB_KUND= EID=3D[kunde][/replace] >>> [/showIf] >>> [/search] >>=20 >>> [snip] >>=20 >>=20 >>=20 >> --------------------------------------------------------- >> This message is sent to you because you are subscribed to >> the mailing list . >> To unsubscribe, E-mail to: >> archives: http://mail.webdna.us/list/talk@webdna.us >> Bug Reporting: support@webdna.us >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us - Govinda -------------- Old WebDNA talklist archives: http://dev.webdna.us/TalkListArchive/index.tpl?db=3Dwebdna-talk
Associated Messages, from the most recent to the oldest:
Chris, yes, you are right; That makes sense to word it more like that = ("speed improvement"... "give control back to the admin"). I only said = "bug" because we have sometimes talked about the issue with the word = "bug" on this list in recent past... and because the whole point of the = admin pref seems to imply giving control to the admin.. and when it did = not (prior to v.7) then the pref is false advertizing.. and then "bug" = seems to fit. Anyway, Tommy, be sure you check what you have set for that pref. If = set to "auto-commit" to disk, then the workaround I mentioned will not = help anything (because then webdna will still be writing to disk on = every operation). Note I have solved crashing problems more than once where legacy code = had an [append] inside a loop ([loop], or [founditems], etc.) that = looped more than a (couple +/-) thousand times... by switching to the = [replace... &append=3DT] workaround. Credit goes to Matt Perosi for = pointing the issue out to me in the first place.=20 Tommy, but do heed Chris' point that if you turn off auto-commit and you = also do not add code to commit to disk once in a while, say once an = hour, or once per day, or something.. then your data is just floating in = RAM and in case of crash, then POOF!.. it's all gone! On 2011-07-11, at 10:31 AM, christophe.billiottet@webdna.us wrote: > Govinda, interesting workaround using [replace]. Note the [append] tag = systematically writing to disk it is not a bug, it had been implemented = this way by SmithMicro, probably as a safety measure. If you check = "Only commit databases to disk when instructed" in the WebDNA = administration and you have a crash, data not written to disk WILL be = lost...=20 >=20 > We just thought in version 7.0 that we could re-enable administrator = control on this point. >=20 > - chris >=20 >=20 >=20 > On Jul 11, 2011, at 11:05, Govinda wrote: >=20 >> Tommy >>=20 >> If you are using any version of webdna less than version 7, then = please note that there is a webdna bug that causes [append] tags to = write to disk, even if the admin pref. has been set to not do that. =20 >> Normally (unless the admin pref has been set to auto-commit) webdna = just writes any new record to the *RAM copy* of the db.. and your disk = copy does not get updated to match the RAM copy until you do a = [closedatabase] or [commitdatabase]. But because of the bug, before = webdna version 7, [append] writes to the disk copy constantly.. meaning = that appends are much slower than they should be... =20 >>=20 >> The workaround for this, which I recommend you implement (in case you = do keep the below 'topbanner.inc' code you posted), is to use a = [replace...] with the '&append=3DT' param (instead of the [append]). In = the [replace...] params, you just search for a record which you *know* = will not exist - to force the append operation... and since [replace..] = does not have the same bug, your operation (essentially an append, same = as before) stays in RAM - nice and fast. >>=20 >> Some thing like this: >>=20 >> replace this: >> [append db=3D[path]../../database/counter.db]DB_DATO=3D[date = %Y/%m/%d]&DB_ANTAL=3D1[/append] >>=20 >> ...with this: >> [Replace = db=3D[path]../../database/counter.db&eqDB_DATOdatarq=3DdontFind2ForceAppen= dHaveAniceDay]DB_DATO=3D[date %Y/%m/%d]&DB_ANTAL=3D1[/Replace] >>=20 >>=20 >> ...and do the same kind of thing for the [append] to 'online.db' . >>=20 >>=20 >> I have no idea if implementing this will help your situation.. but it = might relieve some pressure .. and if your issue is a phantom one = (meaning not consistent on every firing of the said problem code (if we = even know where that problem code is)), then just relieving some = pressure could give you months or years of wiggle room to address the = real deeper issue(s). >>=20 >> - Govinda >> -------------- >> Old WebDNA talklist archives: >> http://dev.webdna.us/TalkListArchive/index.tpl?db=3Dwebdna-talk >>=20 >>=20 >> On 2011-07-11, at 4:03 AM, = wrote: >>=20 >>> [snip] >>=20 >>> Only two places is the online.db file used. >>>=20 >>> One is just a search with nothing else done to it. The other appends = or >>> replaces in online.db I am guessing this is where the problem is. = Despite >>> this appearing to be a windows problem, I hope someone will look at = this >>> and perhaps comment if it is even sane to do something like this. >>>=20 >>> topbanner.inc >>> [search db=3D[path]../../database/online.db&eqDB_IPdatarq=3D[IPAddress= ]] >>> [showIf [numFound]=3D0] >>> [append >>> = db=3D[path]../../database/online.db]DB_IP=3D[ipaddress]&DB_DATO=3D[date]&D= B_TID=3D[time]&DB_URL=3D[url][thisurl]?a=3Db[formvariables][hideIf >>> = [name]=3Dtemplate]&[name]=3D[value][/hideIf][/FormVariables][/url]&DB_KUND= EID=3D[kunde]&DB_BROWSER=3D[url][browser][/url]&DB_URL_REF=3D[url][REFERRE= R][/url][/append] >>> [search db=3D[path]../../database/counter.db&eqDB_DATOdata=3D[date= %Y/%m/%d]] >>> [showIf [numFound]<1] >>> [append db=3D[path]../../database/counter.db]DB_DATO=3D[da= te >>> %Y/%m/%d]&DB_ANTAL=3D1[/append] >>> [/showIf] >>> [showIf [numFound]>0] >>>=20 >>> = [replacefounditems]DB_ANTAL=3D[math][DB_ANTAL]+1[/math][/replacefoundItems= ] >>> [/showIf] >>> [/search] >>> [/showIf] >>> [showIf [numFound]=3D1] >>> [replace >>> = db=3D[path]../../database/online.db&eqDB_IPdatarq=3D[ipaddress]]DB_DATO=3D= [date]&DB_TID=3D[time]&DB_URL=3D[url][thisurl]?a=3Db[formvariables][hideIf= >>> = [name]=3Dtemplate]&[name]=3D[value][/hideIf][/FormVariables][/url]&DB_KUND= EID=3D[kunde][/replace] >>> [/showIf] >>> [/search] >>=20 >>> [snip] >>=20 >>=20 >>=20 >> --------------------------------------------------------- >> This message is sent to you because you are subscribed to >> the mailing list . >> To unsubscribe, E-mail to: >> archives: http://mail.webdna.us/list/talk@webdna.us >> Bug Reporting: support@webdna.us >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us - Govinda -------------- Old WebDNA talklist archives: http://dev.webdna.us/TalkListArchive/index.tpl?db=3Dwebdna-talk
Govinda
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
ImageMagick Commands? (2003)
quantity minimum problem (1997)
Trouble with formula.db (1997)
GuestBook example (1997)
Numbers and Numbers and Me (1999)
Pirated WebCat? My 2 cents worth.. (2003)
Error, 101 a DNS problem ? (1997)
international time (1997)
Emailer setup (1997)
emailer on Windows Beta 18 (1997)
easiest way to check for a non-alphanumeric character? (2000)
Is there a max number of fields? (1998)
NT or Mac (1997)
[replaceChars] would be nice ... (1997)
Re[2]: Re[3]: Problem with new formvariables (2000)
[WebDNA] Problem with Sendmail (2012)
WebCatalog2 Feature Feedback (1996)
EIMS Problems (1997)
New Plug-in and Type 11 errors (1997)
[WebDNA] Securing form input on WEBDNA (2018)