Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether?

This WebDNA talk-list message is from

2009


It keeps the original formatting.
numero = 103039
interpreted = N
texte = Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [LOOKUP] instead of [SEARCH] > whenever possible, using "Only commit databases to disk when > instructed" and manually [COMMITDATABASE] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [COMMITDATABASE] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  2. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  3. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  4. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  5. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  6. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  7. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  8. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  9. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  10. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Patrick McCormick 2009)
  11. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  12. [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [lookup] instead of [search] > whenever possible, using "Only commit databases to disk when > instructed" and manually [commitdatabase] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [commitdatabase] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

WebCat/CyberStudio Compatibility (1998) ASCII Code 127 (2004) WebCat2b12 CGI Mac -- Problems propagating the cart through frames...still (1997) WebCatalog NT beta 18 problem (1997) Size limit for tmpl editor ? (1997) snicker, snicker (1997) [DOS] (1999) Re:my First Ship table (1998) Templates for Customer Database? (1997) WriteFile Formatting (2000) Protect and Serve (1999) Items XX to XX shown (1997) WebCatalog for guestbook ? (1997) Multiple catalog databases and showcart (1997) GoLive CyberStudio 1.0.1 (1997) Sorting problem (1997) Anyone using Editize (inline html editor) w/ WebDNA (2003) Using Applescript to process WebCatalog functions (1998) Generating Report Totals (1997) FW: Shipping calculations (1997)