Block Yandex, AhrefsBot, linkdexbot at server level?

Discussion in 'Server Operation' started by Tastiger, Jul 11, 2017.

  1. Tastiger

    Tastiger Member

    I have had a lot of bandwidth usage lately and checking the apache log it appears as if Yandex, AhrefsBot, linkdexbot are hitting my sites like there is no tomorrow and it also seems as if the latter 2 are managing to bypass the .htaccess files on my sites.

    Is there another option to block these bots other than .htzccess and robots.txt?
     
  2. sjau

    sjau Local Meanie Moderator

    Tastiger likes this.
  3. Tastiger

    Tastiger Member

    many thanks
     
  4. sjau

    sjau Local Meanie Moderator

    haven't tested it myself but the instructions there look fine.
     
  5. Tastiger

    Tastiger Member

    working on it now, but wondering why a bot should be able access the Apache HTTP Server Version 2.4 manual when it isn't in the web directory:-
    sure enough went to:- http://scm-rpg.com.au/manual/zh-cn/mod/module-dict.html
    and there it is ?????
     
  6. Tastiger

    Tastiger Member

    I'll be honest and admit that I really don't understand that page at all can someone set out the steps for me in a simpler way using (and I'm guessing here) /etc/apache2/sites-enabled (vhost)

    At the moment I have at the top of my .htaccess files just under RewriteEngine On
     
    Last edited: Jul 12, 2017

Share This Page