Re: [WebDNA] Spambots, message boards and such
This WebDNA talk-list message is from 2008
It keeps the original formatting.
numero = 100193
interpreted = N
texte = http://oss.oetiker.ch/mrtg/ is monitoring tool. Jym Duane wrote: > answer - for us anyway... > > we have found two things that kill our server. > they are not regular intervals. (but i could see how they could be) > > spambots as described and true dos attacks. (these could come back on > a schedule) > > and > > searchbots that find large patches of endless links like in a calendar > section of a site, > where the code that we wrote does not filter out a malformed request. > (these could come back on a schedule) > > for instance if the bots leave of some of the variables of they search > string that would narrow a search down from all results (thousands) to > the correct result (a few) so the runaway effect is lots of requests > hitting a database in a row for lots of data and large html results also. > the long term solution here is fix code to hide requests if string is > short any variables. (lots to fix) > > meanwhile we have a system that monitors load and restarts webcat and > apachi when the 5 min average exceeds 200% (two processors). > also for hard attacks blocking those ip blocks etc. > > so i can imagine code on all of the same companies serves in the same > data center being scanned regularly by the same bad bot code as it > comes though to get access to the sites. these overloads in our case > could slow server to a crawl and in extreme cases before counter > measures would stop webdna. > > ps - we need to restart both webcat and apachi and that clears some, > then we restart again in two minutes and repeat until load is back > down as it clears requests that are stuck. > > this is not really a webdna issue it is code that needs to be better > written and bots that need to be better written (no control over that) > > our system of monitor and restart also seams to to have beaten back > the attacks/bots so some may have given up. > > we only suffer one big attack a month now on average, for a while - > about the time these other folks say it started we suffered daily or > weekly issues. > > also the processors can run for a long time in the higher load that we > chose to restart from, but in some cases i could see where the extra > load would overheat and lock up servers also. so it is important to > monitor from the load aspect. > > looking at load also enable you to see when the attack or bot starts > so you can find requests that starts it. > > > > jym > > > > Frank Nordberg wrote: >> Frank Nordberg wrote: >> >> > Just a long shot: are there or have there ever been any message board >> > (s) or anything remotely resembling message board(s) on any ip >> > address or URL associated with that server? >> >> >> Bob Atchison wrote: >>> We had one on one of the servers and we moved it off. It had no >>> effect, webDNA would still stop running. It was not written in >>> webDNA. We had other servers with no forums or message boards that >>> still had/have the problem. >> >> That's not the problem in this case then. (I didn't really believe it >> was but thought it would be worth checking.) >> >> However, I think it's important to know that just taking down a >> message board suffering from spam attacks doesn't necessarily solve >> the problem. The spam bots will keep trying to post and if they don't >> get through they just try harder, effectively launching a dos attack >> on the poor server. This is also true if capchats or other safety >> measures are added to a board that's already under attack. >> The trick is to post blank documents at all the URLs associated >> with the infected message board. Aparently serving an empty page is >> easier on the server than a 404 and the difference in load may well >> be the difference between life and death for a poor, overworked >> computer. >> >> >> >> Frank Nordberg >> http://www.musicaviva.com >> http://stores.ebay.com/Nordbergs-Music-Store?refid=store >> >> >> >> > -- Jym Duane - CTO - Purpose Media Creating Your Success Story Marketing : Television - Internet - Print Phone: (877) 443-1323 Email: jym@purposemedia.com Web: www.purposemedia.com Oregon Office - www.GuideToOregon.com PO Box 1725, Jacksonville, OR 97530 California Office - www.OrangeCounty.net PO Box 2025, Capistrano Beach, CA 92624
Associated Messages, from the most recent to the oldest:
http://oss.oetiker.ch/mrtg/ is monitoring tool. Jym Duane wrote: > answer - for us anyway... > > we have found two things that kill our server. > they are not regular intervals. (but i could see how they could be) > > spambots as described and true dos attacks. (these could come back on > a schedule) > > and > > searchbots that find large patches of endless links like in a calendar > section of a site, > where the code that we wrote does not filter out a malformed request. > (these could come back on a schedule) > > for instance if the bots leave of some of the variables of they search > string that would narrow a search down from all results (thousands) to > the correct result (a few) so the runaway effect is lots of requests > hitting a database in a row for lots of data and large html results also. > the long term solution here is fix code to hide requests if string is > short any variables. (lots to fix) > > meanwhile we have a system that monitors load and restarts webcat and > apachi when the 5 min average exceeds 200% (two processors). > also for hard attacks blocking those ip blocks etc. > > so i can imagine code on all of the same companies serves in the same > data center being scanned regularly by the same bad bot code as it > comes though to get access to the sites. these overloads in our case > could slow server to a crawl and in extreme cases before counter > measures would stop webdna. > > ps - we need to restart both webcat and apachi and that clears some, > then we restart again in two minutes and repeat until load is back > down as it clears requests that are stuck. > > this is not really a webdna issue it is code that needs to be better > written and bots that need to be better written (no control over that) > > our system of monitor and restart also seams to to have beaten back > the attacks/bots so some may have given up. > > we only suffer one big attack a month now on average, for a while - > about the time these other folks say it started we suffered daily or > weekly issues. > > also the processors can run for a long time in the higher load that we > chose to restart from, but in some cases i could see where the extra > load would overheat and lock up servers also. so it is important to > monitor from the load aspect. > > looking at load also enable you to see when the attack or bot starts > so you can find requests that starts it. > > > > jym > > > > Frank Nordberg wrote: >> Frank Nordberg wrote: >> >> > Just a long shot: are there or have there ever been any message board >> > (s) or anything remotely resembling message board(s) on any ip >> > address or URL associated with that server? >> >> >> Bob Atchison wrote: >>> We had one on one of the servers and we moved it off. It had no >>> effect, webDNA would still stop running. It was not written in >>> webDNA. We had other servers with no forums or message boards that >>> still had/have the problem. >> >> That's not the problem in this case then. (I didn't really believe it >> was but thought it would be worth checking.) >> >> However, I think it's important to know that just taking down a >> message board suffering from spam attacks doesn't necessarily solve >> the problem. The spam bots will keep trying to post and if they don't >> get through they just try harder, effectively launching a dos attack >> on the poor server. This is also true if capchats or other safety >> measures are added to a board that's already under attack. >> The trick is to post blank documents at all the URLs associated >> with the infected message board. Aparently serving an empty page is >> easier on the server than a 404 and the difference in load may well >> be the difference between life and death for a poor, overworked >> computer. >> >> >> >> Frank Nordberg >> http://www.musicaviva.com >> http://stores.ebay.com/Nordbergs-Music-Store?refid=store >> >> >> >> > -- Jym Duane - CTO - Purpose Media Creating Your Success Story Marketing : Television - Internet - Print Phone: (877) 443-1323 Email: jym@purposemedia.com Web: www.purposemedia.com Oregon Office - www.GuideToOregon.com PO Box 1725, Jacksonville, OR 97530 California Office - www.OrangeCounty.net PO Box 2025, Capistrano Beach, CA 92624
Jym Duane
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
$Quit, $CloseDatabase corrections (1997)
What am I missing (1997)
Likelihood of a duplicate (2005)
Forms & Tables (1998)
Can you do this??? and other stuff (1997)
WebCat editing, SiteGuard & SiteEdit (1997)
Waitfor file (2000)
Now you see it now you donīt (1997)
!@#$$@@# formulas database, tax and freight. (2002)
Location of Browser Info.txt file (1997)
How to Display text in empty fields (1997)
Database Strategy - more... (1998)
Same DB Same Time (2004)
PSC recommends what date format yr 2000??? (1997)
Generating unique SKU from [cart] - FIXED! (1997)
unique ascending numbers (2003)
AJAX with WebDNA (2006)
[shownext] and descending order (1997)
Truncated numbers (2000)
Help! WebCat2 bug (1997)