Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether?

This WebDNA talk-list message is from

2009


It keeps the original formatting.
numero = 103039
interpreted = N
texte = Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [LOOKUP] instead of [SEARCH] > whenever possible, using "Only commit databases to disk when > instructed" and manually [COMMITDATABASE] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [COMMITDATABASE] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  2. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  3. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  4. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  5. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  6. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  7. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  8. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  9. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  10. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Patrick McCormick 2009)
  11. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  12. [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [lookup] instead of [search] > whenever possible, using "Only commit databases to disk when > instructed" and manually [commitdatabase] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [commitdatabase] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

NT [delete] (1998) Still having math problems (1997) mass mailing (1998) WebCat2 beta 11 - new prefs ... (1997) Generating unique SKU from [cart] (1997) One .hdr, multiple .db's? (2003) Webcat/Webmerchant (1998) Various Problems (1997) [OT] Mom's gift (2006) Part Html part WebDNA (1997) Replacing entry (1998) [showif] with ! (2000) Appending Information from one db to another (1998) Error -108 (1997) japanese characters (1997) PCS Frames (1997) Not really WebCat (1997) Euro WebDNA Conference (2004) [accountNum] and [math] (1997) Upgrading old WebCat Database Files (1997)