Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether?

This WebDNA talk-list message is from

2009


It keeps the original formatting.
numero = 103039
interpreted = N
texte = Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [LOOKUP] instead of [SEARCH] > whenever possible, using "Only commit databases to disk when > instructed" and manually [COMMITDATABASE] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [COMMITDATABASE] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  2. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  3. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  4. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  5. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  6. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  7. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  8. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  9. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  10. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Patrick McCormick 2009)
  11. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  12. [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [lookup] instead of [search] > whenever possible, using "Only commit databases to disk when > instructed" and manually [commitdatabase] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [commitdatabase] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Change Subtotal (2000) Using Plug-In while running 1.6.1 (1997) Date problems (1997) [OT] Colored source code in IE? (2007) File Upload (1997) Comments in db? (1997) Here we go again... (2006) TCP Connect (1999) Multiple catalog databases and showcart (1997) How fast is your server? (2002) Bad card db - *mislabled post* (1999) Plugin or CGI or both (1997) WebCat on Intel Core 2 macs anyone? (2007) Header values are not accepted (1998) Emailer (1997) HTML encoding in URLs (1997) 5.0 Pricing (2003) Emailer again (1997) WebCat2_Mac RETURNs in .db (1997) WebCat2b14MacPlugIn - [include] doesn't hide the search string (1997)