Re: Multithreading of [replace]
This WebDNA talk-list message is from 1999
It keeps the original formatting.
numero = 23103
interpreted = N
texte = >>Given that I have written a database to do logging of visitors to the website, >>so that I can do my own analysis of who did what, I am running into a problem >>with the sheer number of write commands being issued to this database > >>[search db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >>[showif [numfound]=0] >>[Append db=carts.db] >>cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >>[/append] >>[/showif] >>[showif [numfound]=1] >>[replace db=carts.db&eqCARTdata=[cart]] >>lastpage=[thisurl] >>[/replace] >>[/showif] >>[/search] > >>So, Grant, here is my question: Is there anything that can be optimized that >>will alleviate the speed hit this code causes under load? > > Multithreading is somewhat slowed here, because carts.db is locked up > during the replace, so no other threads can even read it (other incoming > hits get 'stuck' at the first search until the replace is done). What OS > is this? For multithreading NT would be fastest, MacOS plugin next, and > finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making > it smaller smooths out multithreading but can introduce extra overhead as > well, so it needs tuning.Mac ACGI, which is a big part of the problem. My NT server is on the way, Now I just have the fun of porting/converting the site for NT.> Date searching can be very slow compared to plain text, because WebCatalog > must interpret those dates and calculate lots of stuff. I would re-order > your date text into YYYY/MM/DD order so that it is text-sortable *without* > needing to specify DATEtype=date. I would also set allCase=T so searches > and replaces are case-sensitive (a bit faster).The only reason the date sort is there is to try to get the database read backwards for speed. My logic was a little messed up there, the date stuff is completely irrelevant, and has been removed.> Finally, you can get rid of that whole ShowIf thing: unless I misunderstand > your intention, we already put a feature into replace that will append if > not found: [replace db=xx&append=T] will append a new record if none is > found.There is additional information (first page viewed, first time, refferer) that only apply if there is no preexisting cart, after that, I don't want to over write the info.> I hope this helps! I think doing all of these things could net you from > 50% to 500% speed increase.The WebCatalog not bombing, or popping up serial numbers, or not running out of memory, thus causing customers not to be able to check out would be a pretty big benefit too.I plan on putting the latest beta of the ACGI on the webserver while I am here over the weekend to see if it helps. Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today.
Associated Messages, from the most recent to the oldest:
>>Given that I have written a database to do logging of visitors to the website, >>so that I can do my own analysis of who did what, I am running into a problem >>with the sheer number of write commands being issued to this database > >>[search db=carts.db&eqCARTdata=
[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >>[showif [numfound]=0] >>[Append db=carts.db] >>cart=
[cart]&date=
[date]&firstpage=
[thisurl]&lastpage=
[thisurl] >>[/append] >>[/showif] >>[showif [numfound]=1] >>[replace db=carts.db&eqCARTdata=
[cart]] >>lastpage=
[thisurl] >>[/replace] >>[/showif] >>[/search] > >>So, Grant, here is my question: Is there anything that can be optimized that >>will alleviate the speed hit this code causes under load? > > Multithreading is somewhat slowed here, because carts.db is locked up > during the replace, so no other threads can even read it (other incoming > hits get 'stuck' at the first search until the replace is done). What OS > is this? For multithreading NT would be fastest, MacOS plugin next, and > finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making > it smaller smooths out multithreading but can introduce extra overhead as > well, so it needs tuning.Mac ACGI, which is a big part of the problem. My NT server is on the way, Now I just have the fun of porting/converting the site for NT.> Date searching can be very slow compared to plain text, because WebCatalog > must interpret those dates and calculate lots of stuff. I would re-order > your date text into YYYY/MM/DD order so that it is text-sortable *without* > needing to specify DATEtype=date. I would also set allCase=T so searches > and replaces are case-sensitive (a bit faster).The only reason the date sort is there is to try to get the database read backwards for speed. My logic was a little messed up there, the date stuff is completely irrelevant, and has been removed.> Finally, you can get rid of that whole ShowIf thing: unless I misunderstand > your intention, we already put a feature into replace that will append if > not found: [replace db=xx&append=T] will append a new record if none is > found.There is additional information (first page viewed, first time, refferer) that only apply if there is no preexisting cart, after that, I don't want to over write the info.> I hope this helps! I think doing all of these things could net you from > 50% to 500% speed increase.The WebCatalog not bombing, or popping up serial numbers, or not running out of memory, thus causing customers not to be able to check out would be a pretty big benefit too.I plan on putting the latest beta of the ACGI on the webserver while I am here over the weekend to see if it helps. Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today.
Brian B. Burton
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Append File help needed (2000)
frustrated with search (2001)
Problems problems problems (1997)
[protect admin] (1997)
WebCat2b14MacPlugIn - [include] doesn't hide the search string (1997)
Just a thought (1998)
Sorting when searching with grouped fields (2002)
my price won't move (1997)
Undeliverable Mail (1997)
Stopping bad HTML propagation ? (1997)
Big Databases (1997)
How can I Add several Items into the cart at once? (1997)
encrypt but with normal characters for result? (2005)
Listserver problem (1997)
A simple email page? (1999)
[WebDNA] Date and Time as numbers (2009)
[subtotal] and others (1997)
WebDelivery downloads alias, not original ? (1997)
ooops...WebCatalog [FoundItems] Problem - LONG - (1997)
Updating a database once per day - An example (1998)