Slow cms websites -need help

Discussion in 'Server Operation' started by Nousfeed, Nov 11, 2020.

  1. Nousfeed

    Nousfeed New Member

    So I have a server with 8 cores and 32GB or ram with SSD storage, static websites are very fast. but anything with a db is very slow. The wordpress sites have a 4sec wait time when running, the majority of them are low traffic. CPU and ram usage is quite low.

    I have ispconfig default installation from here https://www.howtoforge.com/perfect-server-debian-10-buster-apache-bind-dovecot-ispconfig-3-1/
    The Sites use to load under 2 seconds, now some of them take up to 10secs, mainly because of this wait time.
    I was thinking its something to do with the database settings, they may need tuning.
    I have run mysqltuner.pl
    Code:
     >>  MySQLTuner 1.7.20 - Major Hayden <[email protected]>
     >>  Bug reports, feature requests, and downloads at http://mysqltuner.pl/
     >>  Run with '--help' for additional options and output filtering
    
    [--] Skipped version check for MySQLTuner script
    Please enter your MySQL administrative login: root
    Please enter your MySQL administrative password: [OK] Currently running supported MySQL version 10.3.22-MariaDB-0+deb10u1
    [OK] Operating on 64-bit architecture
    
    -------- Log file Recommendations ------------------------------------------------------------------
    [!!] Log file /var/log/mysql/error.log doesn't exist
    
    -------- Storage Engine Statistics -----------------------------------------------------------------
    [--] Status: +Aria +CSV +InnoDB +MEMORY +MRG_MyISAM +MyISAM +PERFORMANCE_SCHEMA +SEQUENCE
    [--] Data in MyISAM tables: 31.6M (Tables: 132)
    [--] Data in InnoDB tables: 1.1G (Tables: 3049)
    [OK] Total fragmented tables: 0
    
    -------- Analysis Performance Metrics --------------------------------------------------------------
    [--] innodb_stats_on_metadata: OFF
    [OK] No stat updates during querying INFORMATION_SCHEMA.
    
    -------- Security Recommendations ------------------------------------------------------------------
    [OK] There are no anonymous accounts for any database users
    [!!] User '[email protected]' has no password set.
    [!!] There is no basic password file list!
    
    -------- CVE Security Recommendations --------------------------------------------------------------
    [--] Skipped due to --cvefile option undefined
    
    -------- Performance Metrics -----------------------------------------------------------------------
    [--] Up for: 1d 19h 58m 29s (14M q [91.510 qps], 238K conn, TX: 60G, RX: 27G)
    [--] Reads / Writes: 92% / 8%
    [--] Binary logging is disabled
    [--] Physical Memory     : 30.7G
    [--] Max MySQL memory    : 19.7G
    [--] Other process memory: 0B
    [--] Total buffers: 432.0M global + 130.9M per thread (151 max threads)
    [--] P_S Max memory usage: 0B
    [--] Galera GCache Max memory usage: 0B
    [OK] Maximum reached memory usage: 2.6G (8.47% of installed RAM)
    [OK] Maximum possible memory usage: 19.7G (64.35% of installed RAM)
    [OK] Overall possible memory usage with other process is compatible with memory available
    [OK] Slow queries: 0% (0/14M)
    [OK] Highest usage of available connections: 11% (17/151)
    [OK] Aborted connections: 0.20%  (477/238241)
    [!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
    [!!] Query cache may be disabled by default due to mutex contention.
    [OK] Query cache efficiency: 37.6% (7M cached / 19M selects)
    [!!] Query cache prunes per day: 770109
    [OK] Sorts requiring temporary tables: 0% (3 temp sorts / 1M sorts)
    [!!] Joins performed without indexes: 2061
    [!!] Temporary tables created on disk: 75% (573K on disk / 755K total)
    [OK] Thread cache hit rate: 99% (17 created / 238K connections)
    [!!] Table cache hit rate: 0% (2K open / 1M opened)
    [!!] table_definition_cache(400) is lower than number of tables(3340)
    [OK] Open file limit used: 5% (216/4K)
    [OK] Table locks acquired immediately: 99% (763K immediate / 763K locks)
    
    -------- Performance schema ------------------------------------------------------------------------
    [--] Performance schema is disabled.
    [--] Memory used by P_S: 0B
    [--] Sys schema isn't installed.
    
    -------- ThreadPool Metrics ------------------------------------------------------------------------
    [--] ThreadPool stat is enabled.
    [--] Thread Pool Size: 8 thread(s).
    [--] Using default value is good enough for your version (10.3.22-MariaDB-0+deb10u1)
    
    -------- MyISAM Metrics ----------------------------------------------------------------------------
    [!!] Key buffer used: 18.9% (25M used / 134M cache)
    [OK] Key buffer size / total MyISAM indexes: 128.0M/7.1M
    [OK] Read Key buffer hit rate: 99.2% (6M cached / 55K reads)
    [OK] Write Key buffer hit rate: 97.7% (24K cached / 23K writes)
    
    -------- InnoDB Metrics ----------------------------------------------------------------------------
    [--] InnoDB is enabled.
    [--] InnoDB Thread Concurrency: 0
    [OK] InnoDB File per table is activated
    [!!] InnoDB buffer pool / data size: 128.0M/1.1G
    [!!] Ratio InnoDB log file size / InnoDB Buffer pool size (75 %): 48.0M * 2/128.0M should be equal to 25%
    [OK] InnoDB buffer pool instances: 1
    [--] Number of InnoDB Buffer Pool Chunk : 1 for 1 Buffer Pool Instance(s)
    [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances
    [OK] InnoDB Read buffer efficiency: 99.46% (421585931 hits/ 423862279 total)
    [OK] InnoDB Write log efficiency: 98.18% (54942103 hits/ 55962819 total)
    [OK] InnoDB log waits: 0.00% (0 waits / 1020716 writes)
    
    -------- AriaDB Metrics ----------------------------------------------------------------------------
    [--] AriaDB is enabled.
    [OK] Aria pagecache size / total Aria indexes: 128.0M/1B
    [!!] Aria pagecache hit rate: 94.6% (9M cached / 537K reads)
    
    -------- TokuDB Metrics ----------------------------------------------------------------------------
    [--] TokuDB is disabled.
    
    -------- XtraDB Metrics ----------------------------------------------------------------------------
    [--] XtraDB is disabled.
    
    -------- Galera Metrics ----------------------------------------------------------------------------
    [--] Galera is disabled.
    
    -------- Replication Metrics -----------------------------------------------------------------------
    [--] Galera Synchronous replication: NO
    [--] No replication slave(s) for this server.
    [--] Binlog format: MIXED
    [--] XA support enabled: ON
    [--] Semi synchronous replication Master: OFF
    [--] Semi synchronous replication Slave: OFF
    [--] This is a standalone server
    
    -------- Recommendations ---------------------------------------------------------------------------
    General recommendations:
        Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
        We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
                 See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
                 (specially the conclusions at the bottom of the page).
        When making adjustments, make tmp_table_size/max_heap_table_size equal
        Reduce your SELECT DISTINCT queries which have no LIMIT clause
        Increase table_open_cache gradually to avoid file descriptor limits
        Read this before increasing table_open_cache over 64: https://bit.ly/2Fulv7r
        Read this before increasing for MariaDB https://mariadb.com/kb/en/library/optimizing-table_open_cache/
        This is MyISAM only table_cache scalability problem, InnoDB not affected.
        See more details here: https://bugs.mysql.com/bug.php?id=49177
        This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
        Beware that open_files_limit (4190) variable
        should be greater than table_open_cache (2000)
        Performance schema should be activated for better diagnostics
        Consider installing Sys schema from https://github.com/mysql/mysql-sys for MySQL
        Consider installing Sys schema from https://github.com/FromDual/mariadb-sys for MariaDB
        Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU
    Variables to adjust:
        query_cache_size (=0)
        query_cache_type (=0)
        query_cache_size (> 16M)
        join_buffer_size (> 256.0K, or always use indexes with JOINs)
        tmp_table_size (> 16M)
        max_heap_table_size (> 16M)
        table_open_cache (> 2000)
        table_definition_cache(400) > 3340 or -1 (autosizing if supported)
        performance_schema = ON enable PFS
        innodb_buffer_pool_size (>= 1.1G) if possible.
        innodb_log_file_size should be (=16M) if possible, so InnoDB total log files size equals to 25% of buffer pool size.
    Where do I make these adjustments, my my.cnf file only has the lines including the cnf of 2 directories.
    Is it safe to edit the my.cnf file directly or do i need to find the corresponding files in the included directories, I looked at them and a lot of the options will need to be added.
    All info I can find just mentions my.cnf

    If you don't think its the database any idea of what it could be. I would really appreciate any help on this. ​
     
  2. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    You can try my.cnf, or there is often a directory like /etc/mysql/conf.d/ it similar that includes any files found, which is ready to ensure you is loaded last be naming it 99local.cnf or such.
     
  3. Nousfeed

    Nousfeed New Member

    Thanks for the help, I have a conf.d folder and mariadb.conf.d folder. So if i create a file named 99local.cnf and add any changed settings in that, it will take priority over any settings in any other file?

    I have also installed a plugin that caches database queries on one of the websites i was testing , it reduced the wait time from 4secs down to 500ms. So I'm pretty sure I'm looking in the right direction.
     
  4. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    I have my custom conf in a file in /etc/mysql/mariadb.conf.d, so that's fine.
     
  5. Nousfeed

    Nousfeed New Member

    Thanks for answering my question.
    So what I have planned is create a file called 99-local.cnf with the contents.
    Code:
    # Custom config for fine tuning
    #
    query_cache_type =0
    query_cache_size = 16M
    join_buffer_size = 256.0K
    tmp_table_size = 16M
    max_heap_table_size = 16M
    table_open_cache = 2000
    table_definition_cache =4000
    performance_schema = ON enable PFS
    innodb_buffer_pool_size =16G
    innodb_log_file_size =16M
    What I'm unsure on now is some of these settings, I'm thinking only doing 1-2 settings at a time starting with table_open_cache and table definition_cache. What im confused with is the
    Code:
    innodb_buffer_pool_size =16G
    innodb_log_file_size =16M
    mysqltuner.pl recommends two different things, for innodb_buffer_pool_size 75% of available memory and innodb_log_file_size should be 25% of buffer_pool_size but it also says it should be 16M

    Can anyone make some recommendations on these settings or some more info would be greatly appreciated.
     
  6. Nousfeed

    Nousfeed New Member

    So I made just the one change
    Code:
    table_definition_cache =4000
    It has made quite a big difference by itself. server wait time went from 3500-4000ms to 500-1200ms. Ill leave it for a day and make some of the other changes. Thanks for the help provided so far.
     
    Th0m and Jesse Norell like this.
  7. nhybgtvfr

    nhybgtvfr Active Member

    you don't say what the cms is.

    there are some things you can do generally, like use php-fpm, switch apache to use mpm_event etc which can improve performance and reduce load. obviously not going to make any difference to mysql stuff though, but the mysqltuner stuff you're doing should help there.

    check your cms settings, it could be keeping a log of visitors. also wordpress itself and some plugins can create loads of transient records, this could be adding gigabytes to the size of the sited database and really slow it down.
    so for eg wordpress, check it's crons are configured and working properly, it should be clearing out old transient records.
    check and limit what the cms is keeping track of and logging in the database.
    check there aren't pages where unrestricted comments are allowed and is being abused, you could have a site where a page has thousands of comments being loaded every time someone visits that page.
    lots of page revisions, spam comments etc. it all add up and slows down the site.
    run regular maintenance and cleanups of the cms databases. eg, for wordpress:

    https://torquemag.io/2020/05/wordpress-database-maintenance/
    https://wp-rocket.me/blog/make-wordpress-database-clean-whistle/
     
    Jesse Norell likes this.

Share This Page