Re: Multithreading of [replace]
This WebDNA talk-list message is from 1999
It keeps the original formatting.
numero = 23103
interpreted = N
texte = >>Given that I have written a database to do logging of visitors to the website, >>so that I can do my own analysis of who did what, I am running into a problem >>with the sheer number of write commands being issued to this database > >>[search db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >>[showif [numfound]=0] >>[Append db=carts.db] >>cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >>[/append] >>[/showif] >>[showif [numfound]=1] >>[replace db=carts.db&eqCARTdata=[cart]] >>lastpage=[thisurl] >>[/replace] >>[/showif] >>[/search] > >>So, Grant, here is my question: Is there anything that can be optimized that >>will alleviate the speed hit this code causes under load? > > Multithreading is somewhat slowed here, because carts.db is locked up > during the replace, so no other threads can even read it (other incoming > hits get 'stuck' at the first search until the replace is done). What OS > is this? For multithreading NT would be fastest, MacOS plugin next, and > finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making > it smaller smooths out multithreading but can introduce extra overhead as > well, so it needs tuning.Mac ACGI, which is a big part of the problem. My NT server is on the way, Now I just have the fun of porting/converting the site for NT.> Date searching can be very slow compared to plain text, because WebCatalog > must interpret those dates and calculate lots of stuff. I would re-order > your date text into YYYY/MM/DD order so that it is text-sortable *without* > needing to specify DATEtype=date. I would also set allCase=T so searches > and replaces are case-sensitive (a bit faster).The only reason the date sort is there is to try to get the database read backwards for speed. My logic was a little messed up there, the date stuff is completely irrelevant, and has been removed.> Finally, you can get rid of that whole ShowIf thing: unless I misunderstand > your intention, we already put a feature into replace that will append if > not found: [replace db=xx&append=T] will append a new record if none is > found.There is additional information (first page viewed, first time, refferer) that only apply if there is no preexisting cart, after that, I don't want to over write the info.> I hope this helps! I think doing all of these things could net you from > 50% to 500% speed increase.The WebCatalog not bombing, or popping up serial numbers, or not running out of memory, thus causing customers not to be able to check out would be a pretty big benefit too.I plan on putting the latest beta of the ACGI on the webserver while I am here over the weekend to see if it helps. Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today.
Associated Messages, from the most recent to the oldest:
>>Given that I have written a database to do logging of visitors to the website, >>so that I can do my own analysis of who did what, I am running into a problem >>with the sheer number of write commands being issued to this database > >>[search db=carts.db&eqCARTdata=
[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >>[showif [numfound]=0] >>[Append db=carts.db] >>cart=
[cart]&date=
[date]&firstpage=
[thisurl]&lastpage=
[thisurl] >>[/append] >>[/showif] >>[showif [numfound]=1] >>[replace db=carts.db&eqCARTdata=
[cart]] >>lastpage=
[thisurl] >>[/replace] >>[/showif] >>[/search] > >>So, Grant, here is my question: Is there anything that can be optimized that >>will alleviate the speed hit this code causes under load? > > Multithreading is somewhat slowed here, because carts.db is locked up > during the replace, so no other threads can even read it (other incoming > hits get 'stuck' at the first search until the replace is done). What OS > is this? For multithreading NT would be fastest, MacOS plugin next, and > finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making > it smaller smooths out multithreading but can introduce extra overhead as > well, so it needs tuning.Mac ACGI, which is a big part of the problem. My NT server is on the way, Now I just have the fun of porting/converting the site for NT.> Date searching can be very slow compared to plain text, because WebCatalog > must interpret those dates and calculate lots of stuff. I would re-order > your date text into YYYY/MM/DD order so that it is text-sortable *without* > needing to specify DATEtype=date. I would also set allCase=T so searches > and replaces are case-sensitive (a bit faster).The only reason the date sort is there is to try to get the database read backwards for speed. My logic was a little messed up there, the date stuff is completely irrelevant, and has been removed.> Finally, you can get rid of that whole ShowIf thing: unless I misunderstand > your intention, we already put a feature into replace that will append if > not found: [replace db=xx&append=T] will append a new record if none is > found.There is additional information (first page viewed, first time, refferer) that only apply if there is no preexisting cart, after that, I don't want to over write the info.> I hope this helps! I think doing all of these things could net you from > 50% to 500% speed increase.The WebCatalog not bombing, or popping up serial numbers, or not running out of memory, thus causing customers not to be able to check out would be a pretty big benefit too.I plan on putting the latest beta of the ACGI on the webserver while I am here over the weekend to see if it helps. Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today.
Brian B. Burton
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Great product and great job ! (1997)
[WebDNA] v7 problem? (2012)
PCS Emailer's role ? (1997)
Expert tech support -- a fast and simple solution (long) ... (2000)
WebCat2 - [format thousands] (1997)
[WebDNA] OSX Lion (2012)
proper way to check if a container is empty? (2000)
Looking for a Manual (1997)
Calendar Solutions (1998)
Locking up with WebCatalog... (1997)
Stumpted Again (1997)
Kaaaaahhhhhhhnnnnnnn! (1997)
[format 40s]text[/format] doesn't work (1997)
How about this? (1998)
GuestBook example (1997)
Target wit redirect (1998)
using showpage and showcart commands (1996)
WebCatalog in the search engines (2001)
text size limitation (1997)
select multiple 2 more cents (1997)