Re: WebCatalog Eating 200% of the CPU
This WebDNA talk-list message is from 2002
It keeps the original formatting.
numero = 44846
interpreted = N
texte = >Scott, > >I disagree here a bit - If I can bring down another persons server that is >running code that is 'clean' then surely that sets us all up for DOS >attacks?I have had several occasions where our server (a dual 1Ghz) was brought to it's knees by a semi-aggressive spider requesting one or two search pages/second over a several hour period. If I don't take action and temporarily block the ip in apache or my firewall, it (unfortunately) sometimes turns into a DOS attack... It shouldn't be that easy to take down a dual gig box, but it appears to be...>If the engine running in multithreaded mode starts so many threads that it >goes into a zombie state and does not respond to any requests why have the >preference?When I have seen this problem (below), the WebCatalog process is using a large (>50) number of threads. But, I'm pretty sure that ThreadLimit was not set at the time...>Were not talking about bringing the engine to a crawl - when I simulate this >the engine stops responding totally. The machine it is running on gets >unresponsive and a lot of the time the only way to fix it is hard reset. > >AlainI'm with Alain on this one. I have seen this exact behavior several times since upgrading to 4.5. Webcat stops responding to requests and sucks up all the cpu on my dual 1Ghz G4, WebDNAMonitor doesn't notice the problem, and eventually apache maxes out it's connections (150 for us). Only a manual WebCatalogCtl restart (preceded by apachectl restart if you wait for apache to max out too) will jostle webcat back into action.I have examined the webcat debug files leading up to these events, and have noticed that webcat stops logging to the debug file several MINUTES before the requests stop getting returned by webcat to the browser. I know this because I have 4 different monitoring apps (2 Pagesentry's, 1 Whistleblower, and a local crontab script with curl) testing the server every minute or two. I was going to report this problem earlier, but I didn't want to rock the boat since I seem to have fixed this particular problem. I now have my ThreadLimit set to 10, which appears to limit the max threads that webcat will use at 18 according to top (go figure). I don't think I have seen this problem since setting the ThreadLimit pref... It's been a long two months and my mind is fading...WebDNAMonitor also restarts or detects a crashed WebCatalog process one or two times a day on this same server. This seems to confirm the earlier thread about webcat running amuk. No unique pages being hit, just things that are always being accessed like /index.html seem to cause a crash one time in 30,000 when the load is fairly high. I'm running 10.2.1 client, for the record.To respond to the earlier request for a Thread watcher: I would theorize that this feature would probably only be useful if webcat had enough cycles to respond to such a request. In the case where it has a boatload of threads running (i.e. the time you would want to see what's going on the most) is probably the exact time when you're least likely to get a reply to your request for that info... Otherwise, I think it would be very cool to instantaneously view what webcat is actively chewing on. This information can be deduced to some extent by the debug log + web server log, but is not very much fun to do over more than a couple second period...Any help on the crashing/zombie issues would be appreciated.-Dale------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Associated Messages, from the most recent to the oldest:
>Scott, > >I disagree here a bit - If I can bring down another persons server that is >running code that is 'clean' then surely that sets us all up for DOS >attacks?I have had several occasions where our server (a dual 1Ghz) was brought to it's knees by a semi-aggressive spider requesting one or two search pages/second over a several hour period. If I don't take action and temporarily block the ip in apache or my firewall, it (unfortunately) sometimes turns into a DOS attack... It shouldn't be that easy to take down a dual gig box, but it appears to be...>If the engine running in multithreaded mode starts so many threads that it >goes into a zombie state and does not respond to any requests why have the >preference?When I have seen this problem (below), the WebCatalog process is using a large (>50) number of threads. But, I'm pretty sure that ThreadLimit was not set at the time...>Were not talking about bringing the engine to a crawl - when I simulate this >the engine stops responding totally. The machine it is running on gets >unresponsive and a lot of the time the only way to fix it is hard reset. > >AlainI'm with Alain on this one. I have seen this exact behavior several times since upgrading to 4.5. Webcat stops responding to requests and sucks up all the cpu on my dual 1Ghz G4, WebDNAMonitor doesn't notice the problem, and eventually apache maxes out it's connections (150 for us). Only a manual WebCatalogCtl restart (preceded by apachectl restart if you wait for apache to max out too) will jostle webcat back into action.I have examined the webcat debug files leading up to these events, and have noticed that webcat stops logging to the debug file several MINUTES before the requests stop getting returned by webcat to the browser. I know this because I have 4 different monitoring apps (2 Pagesentry's, 1 Whistleblower, and a local crontab script with curl) testing the server every minute or two. I was going to report this problem earlier, but I didn't want to rock the boat since I seem to have fixed this particular problem. I now have my ThreadLimit set to 10, which appears to limit the max threads that webcat will use at 18 according to top (go figure). I don't think I have seen this problem since setting the ThreadLimit pref... It's been a long two months and my mind is fading...WebDNAMonitor also restarts or detects a crashed WebCatalog process one or two times a day on this same server. This seems to confirm the earlier thread about webcat running amuk. No unique pages being hit, just things that are always being accessed like /index.html seem to cause a crash one time in 30,000 when the load is fairly high. I'm running 10.2.1 client, for the record.To respond to the earlier request for a Thread watcher: I would theorize that this feature would probably only be useful if webcat had enough cycles to respond to such a request. In the case where it has a boatload of threads running (i.e. the time you would want to see what's going on the most) is probably the exact time when you're least likely to get a reply to your request for that info... Otherwise, I think it would be very cool to instantaneously view what webcat is actively chewing on. This information can be deduced to some extent by the debug log + web server log, but is not very much fun to do over more than a couple second period...Any help on the crashing/zombie issues would be appreciated.-Dale------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Dale LaFountain
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Forms Search Questions (1997)
Displaying raw values (1998)
[ShowIf] and empty fields (1997)
Outsourcing partnership ... (2005)
OT- AS/400 and Macs (2003)
WCS Newbie question (1997)
[cart] clarification... (1997)
Dummy Credit Card Number for debug? (1997)
Looking For WebCatalog Migration Assistance (2003)
shownext problems (1998)
Re:2nd WebCatalog2 Feature Request (1996)
Db design question (relational vs. flat?) (1998)
Passing [quantity] (2000)
Help! WebCat2 bug (1997)
[WebDNA] Emerging from the Shell (was: unique words) (2009)
success - serial numbers and webmerch (1997)
OT - royalty free images (2002)
Cart date/time (2006)
WebCatalog stalls (1998)
WebCat and image maps (1997)