Forum slowdowns due to excessive bot traffic

PaulStoffregen

Well-known member
Looks like we're again being targeted by overly aggressive bots.

If you experience the forum responding unusually slow or not at all, please make a note of the time and reply here when it's working again. A list of date, time, timezone of slowdowns really helps me to investigate.

Since the last time bots took the forum down, I increased the memcache table sizes and added efficient server-level blocking for some of the worst offending netblocks. But I've also been gradually building a list of banned netblocks that run VPN services spammers use to create accounts. It's a difficult trade-off, because blocking the source of spammers creating accounts puts more load on PHP and mysql, which gives us less ability to withstand the bots that just try to rapidly scrape the whole site.
 
Today I learned Apache web server configurable log format has an option to log the number of microseconds every request took to fulfill. I've added it to this forum server's config. Hopefully it'll make future automatic analyzing for performance impact possible.
 
Back
Top