Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether?

This WebDNA talk-list message is from

2009


It keeps the original formatting.
numero = 103039
interpreted = N
texte = Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [LOOKUP] instead of [SEARCH] > whenever possible, using "Only commit databases to disk when > instructed" and manually [COMMITDATABASE] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [COMMITDATABASE] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  2. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  3. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  4. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  5. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Palle Bo Nielsen 2009)
  6. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  7. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  8. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (christophe.billiottet@webdna.us 2009)
  9. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  10. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Patrick McCormick 2009)
  11. Re: [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
  12. [WebDNA] 3-5 GB of native WebDNA db in RAM?, else MySQL w/WebDNA [SQL] tags, else skip WebDNA altogether? (Govinda 2009)
Just wanted to add how i split very large bases: imagine a base with this kind of data customer name: customer address: customer tel: item title: item description: auction number: price: Splitting the large base in 3 parts: one "customers" base, one "items" base and one extra "bridge" base that lists the owner reference number associated with the item reference number. The [search] tag is used on the "bridge" base and [lookup] tags will recover the proper data from "customers" and "items". Since [lookup] is extremely fast compared to [search], the global result is significantly faster than a single base. Of course, as Patrick wrote, this does does require common sense and strategy, depending on your data and how to use it. - chris On Jul 28, 2009, at 7:38, christophe.billiottet@webdna.us wrote: > My 2 cents: i already worked with projects including 450,000 records > (full descriptions of antics for an auction house) but even this was > far smaller than 1GB (a little less than 200MB if i remember). > Anyway, what i did was split the database in chunks, each items with > its unique reference number, using [lookup] instead of [search] > whenever possible, using "Only commit databases to disk when > instructed" and manually [commitdatabase] to disk and i must say > WebDNA performs remarkably well: fast, stable, it runs like this on > a 4GB RAM G5 Mac with a 15,000rpms SCSI disk for 4 years now. Since > this customer usually forgets to reboot, i do it myself from time to > time, every 4 or 5 months. > Just remember that writing 1GB to a disk, even a fast one, takes > several tens of seconds. If you get many entries to your database, > it is far better to operate in RAM and carefully commit in order to > do not queue several [commitdatabase] operations. > > - chris > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/ christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Exclamation point (1997) newbie question about zip code search (2003) IP Address (2001) Cold Fusion (1998) WC2b12: Yes, Formulas.db is for real (1997) ODBC problems between webcatalog and filemaker pro (2001) browser reliability on large file upload (2004) Running _every_ page through WebCat ? (1997) Date/Time format problems (1997) Thanks Grant (1997) WebCat2 - Getting to the browser's username/password data (1997) Range in [showif] (2000) WebCatalog Hosting (1996) Database Helper missing? (2000) Re:E-Mailer (WebCatb15acgiMac) (1997) Pager notification of completed orders (1998) Database Not Found Problem (2004) message forum question (2005) RE: protect tag on NT (1997) [WebDNA] unix-Linux Intel Apache Module version 8.0.2 (2016)