ISPConfig and BandWidth control (mod_cband)

Discussion in 'Tips/Tricks/Mods' started by ecorona, Feb 3, 2006.

  1. ecorona

    ecorona New Member

    Last edited: Feb 3, 2006
  2. falko

    falko Super Moderator ISPConfig Developer

    Thanks for this one! :)
     
  3. ecorona

    ecorona New Member

    Create a form and automate it!

    I think one i can understand how ispconfig forms work i can automate the process and create a new section in the site creation to manage it.

    If some one does it please trhow me a line.
     
  4. danf.1979

    danf.1979 ISPConfig Developer ISPConfig Developer

    That would be great!
     
  5. juampi

    juampi New Member

    Great Job!
    I will test it!

    JP
     
  6. nenad

    nenad ISPConfig Developer ISPConfig Developer

    Please note that the pkg mentioned in how-to is not valid anymore.

    Current release is: libapache2-mod-cband_0.9.7.4-0bpo1_i386.deb
    You can get it here:
    wget http://www.backports.org/debian/pool/main/m/mod-cband/libapache2-mod-cband_0.9.7.4-0bpo1_i386.deb
    -------------------------------------------------------------------------------------------------------
    Please note that:
    /root/ispconfig/isp/conf/host.conf.master

    mentioned in your manual is:

    /root/ispconfig/isp/conf/vhost.conf.master
    -----------------------------------------------------
    Do you know why this is not working:

    Code:
    <CBandClass googlebot_class>
    CBandClassDst 66.249.64/24
    CBandClassDst 66.249.65/24
    CBandClassDst 66.249.79/24
    </CBandClass>
    
    CBandClassRemoteSpeed googlebot_class 20kb/s 2 3
    I receive error #NOT SUPPORTED#
    --------------------------------------------------------
    BTW what is exact purpose of "scoreboard" ? (file?)
    Is it logfile? In which format? How can I read it?
     
    Last edited: Sep 19, 2006
  7. falko

    falko Super Moderator ISPConfig Developer

    It probably means that mod_cband is not enabled in your Apache configuration.
     
  8. nenad

    nenad ISPConfig Developer ISPConfig Developer

    Actually it is:

    take a look here:

    http://www.cvecara-neven.com/cband-status?refresh=5

    besides it accept these directives:

    CBandPeriod 4W
    CBandScoreboard /var/run/apache2/cvecara-neven.com.scoreboard

    --------------------------------------------------------
    BTW what is exact purpose of "scoreboard" ? (file?)
    Is it logfile? In which format? How can I read it?
     
    Last edited: Sep 19, 2006
  9. nenad

    nenad ISPConfig Developer ISPConfig Developer

    Is that error #NOT SUPPORTED# Apache or ISPConfig or mod_cband specific error? Who generates info about that error?
     
  10. falko

    falko Super Moderator ISPConfig Developer

    ISPConfig checks your directives with the
    Code:
    httpd -t
    command, and if that doesn't say "Syntax OK", ISPConfig comments out the directives. So it's Apache that's complaining.

    Do you have a ServerName directive before your mod_cband directive? That's important, otherwise it doesn't work.
     
  11. nenad

    nenad ISPConfig Developer ISPConfig Developer

    I guess that I have:

    ----------------------------------------------
    File /root/ispconfig/isp/conf/vhost.conf.master

    {FP_RESOURCE_CONFIG}
    {FP_ACCESS_CONFIG}
    ###################################
    #
    # ISPConfig vHost Configuration File
    # Version 1.0
    #
    ###################################
    #
    <!-- BEGIN DYNAMIC BLOCK: namevhost -->
    {SERVERIP}
    <!-- END DYNAMIC BLOCK: namevhost -->
    #
    #
    <!-- BEGIN DYNAMIC BLOCK: vhost -->

    ######################################
    # Vhost: {SERVERNAME}
    ######################################
    #
    #
    <VirtualHost {IP}>
    {SUEXEC}
    ServerName {SERVERNAME}
    {HTTPD_INCLUDE}
    ServerAdmin {SERVERADMIN}
    DocumentRoot {DOCUMENTROOT}
    {SERVERALIAS}
    {DIRECTORYINDEX}
    {CGI}
    ErrorLog {WEB_ERROR_LOG}
    {PHP}
    {SSI}
    {WAP}
    {ERRORALIAS}
    {ERROR}
    AliasMatch ^/~([^/]+)(/(.*))? {HTTPD_ROOT}/{WEB}/user/$1/web/$3
    AliasMatch ^/users/([^/]+)(/(.*))? {HTTPD_ROOT}/{WEB}/user/$1/web/$3
    {REWRITE_RULE}
    {FRONTPAGE}
    </VirtualHost>
    #
    {SSL}
    #
    #
    <!-- END DYNAMIC BLOCK: vhost -->
    --------------------------------------------------------
    Othervise this wouldn't work

    http://www.cvecara-neven.com/cband-status-me?refresh=5

    or it would?

    -----------------------------------------------------
    Instructions taken from http://www.e-corona.org/blog,245

    ------------------------------------------------------------
    If you get no errors at this point we are ready to change ISPConfig to be able to use mod_cband Apache directives.

    mod_cband directives must be placed after we set the vhost name ServerName and ISPConfig places our Apache Directives before it so it wont work.

    To solve this we have to edit root/ispconfig/isp/conf/vhost.conf.master which is the template to generate vhosts in /etc/apache2/vhost/Vhosts_ispconfig.conf

    Move the first line {HTTPD_INCLUDE} below ServerName {SERVERNAME} so Apache Directives from ISPConfig will be placed after ServerName and mod_cband will work.

    We are done here so let's take a look in to a few mod_cband Apache directives.
     
    Last edited: Sep 20, 2006
  12. falko

    falko Super Moderator ISPConfig Developer

    What's in your Vhosts_ispconfig.conf (the real one, not the template)?
     
  13. nenad

    nenad ISPConfig Developer ISPConfig Developer

    ###################################
    #
    # ISPConfig vHost Configuration File
    # Version 1.0
    #
    ###################################
    #
    NameVirtualHost 192.168.123.201:80
    <VirtualHost 192.168.123.201:80>
    ServerName localhost
    ServerAdmin root@localhost
    DocumentRoot /var/www/sharedip
    </VirtualHost>
    #
    #
    ######################################
    # Vhost: www.web-hosting.co.yu:80
    ######################################
    #
    #
    <VirtualHost 192.168.123.201:80>
    SuexecUserGroup web1_nenad web1
    ServerName www.web-hosting.co.yu:80
    ServerAdmin webmaster@web-hosting.co.yu
    DocumentRoot /var/www/web1/web
    ServerAlias web-hosting.co.yu
    DirectoryIndex index.html index.htm index.php index.php5 index.php4 index.php3 index.shtml index.cgi index.pl index.jsp Default.htm default.htm
    ScriptAlias /cgi-bin/ /var/www/web1/cgi-bin/
    AddHandler cgi-script .cgi
    AddHandler cgi-script .pl
    ErrorLog /var/www/web1/log/error.log
    AddType application/x-httpd-php .php .php3 .php4 .php5
    php_admin_flag safe_mode Off
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType application/vnd.wap.wmlscriptc .wmlsc .wsc
    AddType text/vnd.wap.wml .wml
    AddType text/vnd.wap.wmlscript .ws .wmlscript
    AddType image/vnd.wap.wbmp .wbmp
    Alias /error/ "/var/www/web1/web/error/"
    ErrorDocument 400 /error/invalidSyntax.html
    ErrorDocument 401 /error/authorizationRequired.html
    ErrorDocument 403 /error/forbidden.html
    ErrorDocument 404 /error/fileNotFound.html
    ErrorDocument 405 /error/methodNotAllowed.html
    ErrorDocument 500 /error/internalServerError.html
    ErrorDocument 503 /error/overloaded.html
    AliasMatch ^/~([^/]+)(/(.*))? /var/www/web1/user/$1/web/$3
    AliasMatch ^/users/([^/]+)(/(.*))? /var/www/web1/user/$1/web/$3
    </VirtualHost>
    #
    #
    #
    ######################################
    # Vhost: www.cvecara-neven.com:80
    ######################################
    #
    #
    <VirtualHost 192.168.123.201:80>
    SuexecUserGroup web4_nenad web4
    ServerName www.cvecara-neven.com:80
    CBandPeriod 4W
    CBandScoreboard /var/run/apache2/cvecara-neven.com.scoreboard
    ServerAdmin webmaster@cvecara-neven.com
    DocumentRoot /var/www/web4/web
    ServerAlias cvecara-neven.com
    DirectoryIndex index.html index.htm index.php index.php5 index.php4 index.php3 index.shtml index.cgi index.pl index.jsp Default.htm default.htm
    ScriptAlias /cgi-bin/ /var/www/web4/cgi-bin/
    AddHandler cgi-script .cgi
    AddHandler cgi-script .pl
    ErrorLog /var/www/web4/log/error.log
    AddType application/x-httpd-php .php .php3 .php4 .php5
    php_admin_flag safe_mode Off
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType application/vnd.wap.wmlscriptc .wmlsc .wsc
    AddType text/vnd.wap.wml .wml
    AddType text/vnd.wap.wmlscript .ws .wmlscript
    AddType image/vnd.wap.wbmp .wbmp
    Alias /error/ "/var/www/web4/web/error/"
    ErrorDocument 400 /error/invalidSyntax.html
    ErrorDocument 401 /error/authorizationRequired.html
    ErrorDocument 403 /error/forbidden.html
    ErrorDocument 404 /error/fileNotFound.html
    ErrorDocument 405 /error/methodNotAllowed.html
    ErrorDocument 500 /error/internalServerError.html
    ErrorDocument 503 /error/overloaded.html
    AliasMatch ^/~([^/]+)(/(.*))? /var/www/web4/user/$1/web/$3
    AliasMatch ^/users/([^/]+)(/(.*))? /var/www/web4/user/$1/web/$3
    </VirtualHost>
    #
    #
    #
    ######################################
    # Vhost: www.web-hosting-solutions.biz:80
    ######################################
    #
    #
    <VirtualHost 192.168.123.201:80>
    SuexecUserGroup web8_nenad web8
    ServerName www.web-hosting-solutions.biz:80
    ServerAdmin webmaster@web-hosting-solutions.biz
    DocumentRoot /var/www/web8/web
    ServerAlias web-hosting-solutions.biz
    DirectoryIndex index.html index.htm index.php index.php5 index.php4 index.php3 index.shtml index.cgi index.pl index.jsp Default.htm default.htm
    ScriptAlias /cgi-bin/ /var/www/web8/cgi-bin/
    AddHandler cgi-script .cgi
    AddHandler cgi-script .pl
    ErrorLog /var/www/web8/log/error.log
    AddType application/x-httpd-php .php .php3 .php4 .php5
    php_admin_flag safe_mode Off
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType application/vnd.wap.wmlscriptc .wmlsc .wsc
    AddType text/vnd.wap.wml .wml
    AddType text/vnd.wap.wmlscript .ws .wmlscript
    AddType image/vnd.wap.wbmp .wbmp
    Alias /error/ "/var/www/web8/web/error/"
    ErrorDocument 400 /error/invalidSyntax.html
    ErrorDocument 401 /error/authorizationRequired.html
    ErrorDocument 403 /error/forbidden.html
    ErrorDocument 404 /error/fileNotFound.html
    ErrorDocument 405 /error/methodNotAllowed.html
    ErrorDocument 500 /error/internalServerError.html
    ErrorDocument 503 /error/overloaded.html
    AliasMatch ^/~([^/]+)(/(.*))? /var/www/web8/user/$1/web/$3
    AliasMatch ^/users/([^/]+)(/(.*))? /var/www/web8/user/$1/web/$3
    </VirtualHost>
    #
    <IfModule mod_ssl.c>
    <VirtualHost 192.168.123.201:443>
    SuexecUserGroup web8_nenad web8
    ServerName www.web-hosting-solutions.biz:443
    ServerAdmin webmaster@web-hosting-solutions.biz
    DocumentRoot /var/www/web8/web
    ServerAlias web-hosting-solutions.biz
    DirectoryIndex index.html index.htm index.php index.php5 index.php4 index.php3 index.shtml index.cgi index.pl index.jsp Default.htm default.htm
    ScriptAlias /cgi-bin/ /var/www/web8/cgi-bin/
    AddHandler cgi-script .cgi
    AddHandler cgi-script .pl
    ErrorLog /var/www/web8/log/error.log
    AddType application/x-httpd-php .php .php3 .php4 .php5
    php_admin_flag safe_mode Off
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType application/vnd.wap.wmlscriptc .wmlsc .wsc
    AddType text/vnd.wap.wml .wml
    AddType text/vnd.wap.wmlscript .ws .wmlscript
    AddType image/vnd.wap.wbmp .wbmp
    SSLEngine on
    SSLCertificateFile /var/www/web8/ssl/www.web-hosting-solutions.biz.crt
    SSLCertificateKeyFile /var/www/web8/ssl/www.web-hosting-solutions.biz.key
    Alias /error/ "/var/www/web8/web/error/"
    ErrorDocument 400 /error/invalidSyntax.html
    ErrorDocument 401 /error/authorizationRequired.html
    ErrorDocument 403 /error/forbidden.html
    ErrorDocument 404 /error/fileNotFound.html
    ErrorDocument 405 /error/methodNotAllowed.html
    ErrorDocument 500 /error/internalServerError.html
    ErrorDocument 503 /error/overloaded.html
    AliasMatch ^/~([^/]+)(/(.*))? /var/www/web8/user/$1/web/$3
    AliasMatch ^/users/([^/]+)(/(.*))? /var/www/web8/user/$1/web/$3
    SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    </VirtualHost>
    </IfModule>
    #
    #
     
  14. falko

    falko Super Moderator ISPConfig Developer

    That's ok, so mod_cband should work... :confused:
     
  15. nenad

    nenad ISPConfig Developer ISPConfig Developer

    Should I use your install instructions
    http://www.howtoforge.com/mod_cband_apache2_bandwidth_quota_throttling
    (build from source code)

    Or should I use:
    http://www.e-corona.org/blog,245
    (debian precompiled pkg)

    Right now I installed mod_cban with both tutorials.
    Mybie should I un-install both, and then try only one?

    How to deinstall this version:
    http://www.howtoforge.com/mod_cband_apache2_bandwidth_quota_throttling

    And how to deinstall this version:
    http://www.e-corona.org/blog,245
    -------------------------------------------------------------------------------------
    Anyway this is accepted by apache:

    CBandClassRemoteSpeed googlebot_class 20kb/s 2 3

    But this is not accepted:

    <CBandClass googlebot_class>
    CBandClassDst 66.249.64/24
    CBandClassDst 66.249.65/24
    CBandClassDst 66.249.79/24
    </CBandClass>
     
    Last edited: Sep 22, 2006
  16. nenad

    nenad ISPConfig Developer ISPConfig Developer

    Debbuging :)

    This:

    Code:
    <CBandClass googlebot_class>
    CBandClassDst 66.249.64/24
    CBandClassDst 66.249.65/24
    CBandClassDst 66.249.79/24
    </CBandClass>
    Is not accepted, so if I break that into only this

    Code:
    CBandClassDst 66.249.64/24
    CBandClassDst 66.249.65/24
    CBandClassDst 66.249.79/24
    Than it is accepted.

    Therefore my conclusion is that

    Code:
    <CBandClass googlebot_class>
    causes all of of the troubles.
    I do believe that mod_cband should interpret this in proper manner.
    I am wondering at this moment is the support for this command included/properly compiled into

    /usr/lib/apache2/modules/mod_cband.so ?
    ------------------------------------------------------------------
    With all RESPECT to the ecorona and his manual this example from his tutorial:

    Do nothing (the bold /red line) if CBandClass is NOT defined... :(

    Correct me if I am wrong.
     
    Last edited: Sep 22, 2006
  17. nenad

    nenad ISPConfig Developer ISPConfig Developer

    After I deinstalled every single trace of mod_cband, I installed it once agin according to Falko's tutorial, and guess what?

    CBandClass command is not working again thus I do belive that there is a bug in mod_cband.
    I will try to write to author of mod_cband.

    However it might be posible that I made somewhere some mistake...?

    Any toughts about this?

    Is there anyone who can make this work, or with working example for
    mod_cband and CBandClass command ?
     
  18. nenad

    nenad ISPConfig Developer ISPConfig Developer

    BTW I noticed this:

    What if I don't want this? "webmaster"@web-hosting.co.yu

    I think that there should be an option in ISPConfig to define email of server admin, actually domain admin..?
     
  19. nenad

    nenad ISPConfig Developer ISPConfig Developer

    So, maybie I found what is the problem.
    First I stopped ISPConfig, then I put manually

    Code:
    <CBandClass googlebot_class>
    CBandClassDst 66.249.64/24
    CBandClassDst 66.249.65/24
    CBandClassDst 66.249.79/24
    </CBandClass> 
    And then I checked apache syntax to see what is really happening and here is result:

    server201:~# /etc/init.d/ispconfig_server stop
    Shutting down ISPConfig system...
    /root/ispconfig/httpd/bin/apachectl stop: httpd stopped
    ISPConfig system stopped!

    server201:~# apache2 -t
    [Sat Sep 23 03:05:05 2006] [warn] module cache_module is already loaded, skipping
    Syntax error on line 71 of /etc/apache2/vhosts/Vhosts_ispconfig.conf:
    <CBandClass> cannot occur within <VirtualHost> section
     
  20. nenad

    nenad ISPConfig Developer ISPConfig Developer

    And BINGO!!!

    I added CBandClass manually:

    Code:
    ######################################
    # Vhost: www.cvecara-neven.com:80
    ######################################
    #
    #
    
    <CBandClass googlebot_class>
    CBandClassDst 66.249.64/24
    CBandClassDst 66.249.65/24
    CBandClassDst 66.249.79/24
    CBandClassDst 66.249.66.107/24
    </CBandClass>
    
    <VirtualHost 192.168.123.201:80>
    
    as you can see just before <VirtualHost 192.168.123.201:80>

    and here is result:

    [​IMG]

    Nasty GoogleBot who is visiting my small website (600+ pictures & 200+ text pages) every single day all day long, and generates over 300 MB of traffic is finally limited :)

    Don't take me wrong, I love Google and I need his spiders crawl my website but not so fast and preferably not to enerate so much traffic.

    So, it seems that "ISPConfig & mod_cband for Dummies (as I am)" is therfore finished...

    For now :) :cool:
     

Share This Page