Re: Multithreading of [replace]

This WebDNA talk-list message is from

1999


It keeps the original formatting.
numero = 23103
interpreted = N
texte = >>Given that I have written a database to do logging of visitors to the website, >>so that I can do my own analysis of who did what, I am running into a problem >>with the sheer number of write commands being issued to this database > >>[search db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >>[showif [numfound]=0] >>[Append db=carts.db] >>cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >>[/append] >>[/showif] >>[showif [numfound]=1] >>[replace db=carts.db&eqCARTdata=[cart]] >>lastpage=[thisurl] >>[/replace] >>[/showif] >>[/search] > >>So, Grant, here is my question: Is there anything that can be optimized that >>will alleviate the speed hit this code causes under load? > > Multithreading is somewhat slowed here, because carts.db is locked up > during the replace, so no other threads can even read it (other incoming > hits get 'stuck' at the first search until the replace is done). What OS > is this? For multithreading NT would be fastest, MacOS plugin next, and > finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making > it smaller smooths out multithreading but can introduce extra overhead as > well, so it needs tuning.Mac ACGI, which is a big part of the problem. My NT server is on the way, Now I just have the fun of porting/converting the site for NT.> Date searching can be very slow compared to plain text, because WebCatalog > must interpret those dates and calculate lots of stuff. I would re-order > your date text into YYYY/MM/DD order so that it is text-sortable *without* > needing to specify DATEtype=date. I would also set allCase=T so searches > and replaces are case-sensitive (a bit faster).The only reason the date sort is there is to try to get the database read backwards for speed. My logic was a little messed up there, the date stuff is completely irrelevant, and has been removed.> Finally, you can get rid of that whole ShowIf thing: unless I misunderstand > your intention, we already put a feature into replace that will append if > not found: [replace db=xx&append=T] will append a new record if none is > found.There is additional information (first page viewed, first time, refferer) that only apply if there is no preexisting cart, after that, I don't want to over write the info.> I hope this helps! I think doing all of these things could net you from > 50% to 500% speed increase.The WebCatalog not bombing, or popping up serial numbers, or not running out of memory, thus causing customers not to be able to check out would be a pretty big benefit too.I plan on putting the latest beta of the ACGI on the webserver while I am here over the weekend to see if it helps. Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today. Associated Messages, from the most recent to the oldest:

    
  1. Re: Multithreading of [replace] (Kenneth Grome 1999)
  2. Re: Multithreading of [replace] (Christer Olsson 1999)
  3. Re: Multithreading of [replace] (Kenneth Grome 1999)
  4. Re: Multithreading of [replace] (Brian B. Burton 1999)
  5. Re: Multithreading of [replace] (Grant Hulbert 1999)
  6. RE: Multithreading of [replace] (Olin Lagon 1999)
  7. Re: Multithreading of [replace] (Brian B. Burton 1999)
  8. RE: Multithreading of [replace] (Olin Lagon 1999)
  9. Multithreading of [replace] (Brian B. Burton 1999)
>>Given that I have written a database to do logging of visitors to the website, >>so that I can do my own analysis of who did what, I am running into a problem >>with the sheer number of write commands being issued to this database > >>[search db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >>[showif [numfound]=0] >>[Append db=carts.db] >>cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >>[/append] >>[/showif] >>[showif [numfound]=1] >>[replace db=carts.db&eqCARTdata=[cart]] >>lastpage=[thisurl] >>[/replace] >>[/showif] >>[/search] > >>So, Grant, here is my question: Is there anything that can be optimized that >>will alleviate the speed hit this code causes under load? > > Multithreading is somewhat slowed here, because carts.db is locked up > during the replace, so no other threads can even read it (other incoming > hits get 'stuck' at the first search until the replace is done). What OS > is this? For multithreading NT would be fastest, MacOS plugin next, and > finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making > it smaller smooths out multithreading but can introduce extra overhead as > well, so it needs tuning.Mac ACGI, which is a big part of the problem. My NT server is on the way, Now I just have the fun of porting/converting the site for NT.> Date searching can be very slow compared to plain text, because WebCatalog > must interpret those dates and calculate lots of stuff. I would re-order > your date text into YYYY/MM/DD order so that it is text-sortable *without* > needing to specify DATEtype=date. I would also set allCase=T so searches > and replaces are case-sensitive (a bit faster).The only reason the date sort is there is to try to get the database read backwards for speed. My logic was a little messed up there, the date stuff is completely irrelevant, and has been removed.> Finally, you can get rid of that whole ShowIf thing: unless I misunderstand > your intention, we already put a feature into replace that will append if > not found: [replace db=xx&append=T] will append a new record if none is > found.There is additional information (first page viewed, first time, refferer) that only apply if there is no preexisting cart, after that, I don't want to over write the info.> I hope this helps! I think doing all of these things could net you from > 50% to 500% speed increase.The WebCatalog not bombing, or popping up serial numbers, or not running out of memory, thus causing customers not to be able to check out would be a pretty big benefit too.I plan on putting the latest beta of the ACGI on the webserver while I am here over the weekend to see if it helps. Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today. Brian B. Burton

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

WebCat2 - storing unformatted date data? (1997) [WebDNA] table comma field delimiter bug? (2012) WebDNA on Intel Mac? (2006) Webcat no longer supported? (2006) Setting up shop (1997) Single Link browsing (1997) WebCatalog NT beta 18 problem (1997) Running _every_ page through WebCat-error.html (1997) WebDNA Solutions ... (1997) Emails stuck in emails folder (2003) Multipart Form - Ascertain File Type (2007) REDIRECT MULTIPLE RESULTS (2000) Setting up shop (1997) WebDNA 6.0 & MAC OS X Server 10.4.10 (2007) URGENT! ACGI Stopped!!!! (1997) Creating main- and sub-category search (1997) [search] in formulas.db (1997) quick question. (2007) Locking up with WebCatalog... (1997) WebDNA on Windows questions (2007)