Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service

This WebDNA talk-list message is from

2013


It keeps the original formatting.
numero = 110761
interpreted = N
texte = Palle, I ran into a similar problem with a product database during the last=20 year. My server already had a quad processor with 16GB of RAM. My biggest performance issues happen when I'm doing an import. My product database was already split into 3 tables. My solution was to=20 reconstruct the fields in those 3 tables into 5 tables. I carefully=20 looked at all the information in the tables and moved all of the least=20 used fields to the extra tables. Then I reprogrammed all the searches=20 so the extra 2 tables are rarely opened. I then created another table with the most important few searchable=20 fields. This acts as my index. I rebuild it nightly. Users hitting the=20 site are always hitting this main index when doing broad product=20 searches. I'm only using a single index at the moment, but I know the=20 next step is to have a different index for different product categories. Even though it took a while to accomplish this the results were very=20 rewarding. -Matt On 10/2/2013 3:41 AM, Palle Bo Nielsen wrote: > Hi all, > > Looking for advise to optimise a webpage using WebDNA and large DB's. > > I have built a forum entirely in WebDNA and the DB's are getting big. T= he main DB has the following characteristics=85 > > - Columns: 11 > - Rows: 847.121 > - Size: 264 MB > > If I display the different generic topics within the Forum and the most= recent Post within each category etc. then the code is executing quite f= ast (acceptable performance). > > But if I want to add the value of posts within each category or within = each thread then the performance is slowing the too much. > > I have a category DB which eventually show 25 topics on the main page o= f the forum, then I do a search into the main DB and search for all posts= with this categories ID and show the numFound. This is repeated for ever= y unique Topic. > > > > << Searching for the Topics >> > [search db=3D/forum/db/db1.db&eqdb1_publishdata=3D1&asdb1_prioritysort=3D= 1&db1_prioritytype=3Dnum&AllReqd=3DT][founditems] > > << Searching for the numFound Value - This One is Pulling a Lot of Perf= ormance even though it's simple, but there is a lot of data to search thr= ough >> > [search db=3D/forum/db/db4.db&eqdb4_publishdata=3D1&eqdb4_db1data=3D[db= 1_sku]&AllReqd=3DT][numFound][founditems][/founditems][/search] > > > > I would appreciate any good advise or ideas on how to accelerate the pe= rformance. Some kind of indexing to avoid live searching would be great, = but how? > > /Palle > > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us > > --=20 Matthew A Perosi Corporate Consultant Mobile Marketing Expert Senior Web Developer SEO Analyst & Educator matt@psiprime.com Psi Prime, Inc. 323 Union Blvd. Totowa, NJ 07512 Direct: 888.872.0274 Fax: 888.488.5924 http://www.perosi.com Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Donovan Brooke 2013)
  2. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Palle Bo Nielsen 2013)
  3. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Palle Bo Nielsen 2013)
  4. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Lawrence Banahan 2013)
  5. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Palle Bo Nielsen 2013)
  6. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Lawrence Banahan 2013)
  7. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Palle Bo Nielsen 2013)
  8. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (matt@psiprime.com 2013)
  9. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Lawrence Banahan 2013)
  10. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (christophe.billiottet@webdna.us 2013)
  11. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Palle Bo Nielsen 2013)
  12. Re: [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Lawrence Banahan 2013)
  13. [WebDNA] Acceleratin Search / Index Performance in a Formum Service (Palle Bo Nielsen 2013)
Palle, I ran into a similar problem with a product database during the last=20 year. My server already had a quad processor with 16GB of RAM. My biggest performance issues happen when I'm doing an import. My product database was already split into 3 tables. My solution was to=20 reconstruct the fields in those 3 tables into 5 tables. I carefully=20 looked at all the information in the tables and moved all of the least=20 used fields to the extra tables. Then I reprogrammed all the searches=20 so the extra 2 tables are rarely opened. I then created another table with the most important few searchable=20 fields. This acts as my index. I rebuild it nightly. Users hitting the=20 site are always hitting this main index when doing broad product=20 searches. I'm only using a single index at the moment, but I know the=20 next step is to have a different index for different product categories. Even though it took a while to accomplish this the results were very=20 rewarding. -Matt On 10/2/2013 3:41 AM, Palle Bo Nielsen wrote: > Hi all, > > Looking for advise to optimise a webpage using WebDNA and large DB's. > > I have built a forum entirely in WebDNA and the DB's are getting big. T= he main DB has the following characteristics=85 > > - Columns: 11 > - Rows: 847.121 > - Size: 264 MB > > If I display the different generic topics within the Forum and the most= recent Post within each category etc. then the code is executing quite f= ast (acceptable performance). > > But if I want to add the value of posts within each category or within = each thread then the performance is slowing the too much. > > I have a category DB which eventually show 25 topics on the main page o= f the forum, then I do a search into the main DB and search for all posts= with this categories ID and show the numFound. This is repeated for ever= y unique Topic. > > > > << Searching for the Topics >> > [search db=3D/forum/db/db1.db&eqdb1_publishdata=3D1&asdb1_prioritysort=3D= 1&db1_prioritytype=3Dnum&AllReqd=3DT][founditems] > > << Searching for the numFound Value - This One is Pulling a Lot of Perf= ormance even though it's simple, but there is a lot of data to search thr= ough >> > [search db=3D/forum/db/db4.db&eqdb4_publishdata=3D1&eqdb4_db1data=3D[db= 1_sku]&AllReqd=3DT][numFound][founditems][/founditems][/search] > > > > I would appreciate any good advise or ideas on how to accelerate the pe= rformance. Some kind of indexing to avoid live searching would be great, = but how? > > /Palle > > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us > > --=20 Matthew A Perosi Corporate Consultant Mobile Marketing Expert Senior Web Developer SEO Analyst & Educator matt@psiprime.com Psi Prime, Inc. 323 Union Blvd. Totowa, NJ 07512 Direct: 888.872.0274 Fax: 888.488.5924 http://www.perosi.com matt@psiprime.com

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

[WebDNA] Hosting Solution (2013) OT: Variables in printed URL (2007) 2.1b3 --> way slow (1997) [WebDNA] grep question (2009) Queertrons? (1997) The IBC root beer has arrived! (1997) Click-through/Referral tracking solution needed (2000) RE: [WebDNA] macOS 10.13.3 and WebDNA Mac Server 8.5.1 - hick-up (2019) Some Questions (1997) a new bug? (1999) graphing (2004) suffix mapping for NT? (1997) Re:[off] Promotions Co? (1997) &FieldNametype=time unnecessary? (2000) Username for Admin Group (1997) Order file numbers different than the cart number. why? (2000) Templates on Unix & CGI on Mac? (1997) [OT] HTML, HTML docs (2001) Emailer Set Up (1997) [WebDNA] Strange MATH result - BUG (2010)