Comments on nginx: How To Set A Connection Limit For Search Engine Bots Gone Wild (Especially Bingbot)
As a server administrator you might know this problem: you have done everything to optimize your server and it's working really well, and along comes a stupid search engine bot (like Bingbot) and hits all your vhosts at the same time with hundreds of connections, making your server load go up. Of course, you don't want to completely block the bot (unless you don't care about that particular search engine), so you can use robots.txt and/or nginx to control connections of a search engine bot to your server.
1 Comment(s)
Add comment
Comments
By: Anonymous
thanks! Excelent. r.