yet another apache2 tuning thread

[erosbk]

Sorry, I did'nt researched if it was or not deprecated. You get the idea, if you know that is deprecated, use what you propose instead!

If you can experiment a little, it could be ca good idea to change the values in every vhost. For sure you will have cgi process for less time in memory. Debian will continue using cache, but you can limit swap. Tunning a web server is an every day works :P

[Ovidiu]

seems web9 was the culprit. I changed the proposed timeout to 300 from originally 3600 and it seems to work.

still very high usage of resources but at least I know who and why is using them: http://zice.ro/munin/serverkompetenz...net/index.html

every time this client sends his newsletter to about 10.000 recipients (all verified, no spam) the server kinda bogged down. looks better right now with the timeout set to 300 (at least that is the only thing I changed and it works) - not sure if that is the cause for it working better now.

still, back to the newsletter, I have set it to send out 50 mails / minute and its using SMTP.

the server is set up according to this how-to: http://www.howtoforge.com/perfect-se...er-ispconfig-3

the only sifference is that I didn't start from scratch but with the strato minimal installation which means the whole thing is running on a software raid 1 - possibly this is the bottleneck?

any good links for checking/tuning software raids or isn't there much to tune?

###edit###
forgot to add that the newsletter is a wordpress plugin and relies on a certain wordpress function that mimics cron, basically it depends on having visitors to trigger so if nobody visits the site, the newsleter doesn't get sent as its never triggered... just for your info.

[till]

I guess the only part where you can tune the system is to check and rewrite the php code of the neslwtter addon to scale better for large newsletters. Normally such a software should use not many resources, but if it e.g. tries to load all subscriber data into an large array at once or similar mistakes, then it can bring down a server.

[Ovidiu]

IdleTimeout 300 saved me :-)

no more idle php-cgi processes sitting around.
now I am onto debugging the HD I/O and latency, found the problem too: mysql.

the million dollar question is how can I set this as a default parameter for all new sites that will be created and keep that change persistent even if I update ispcfg3?

when moving to this server it seems I took over almost all relevant settings except that mysql was running with some default configuration and nothing was cached so sending 10.000 was killing the HD because of all those uncached DB accesses.
finetuning mysql right now to balance things out so that I am neither using too much RAM for the cache nor have too many uncached HD accesses.

[erosbk]

mkdir /scripts/
cd /scripts
wget http://www.day32.com/MySQL/tuning-primer.sh
wget http://mysqltuner.pl/mysqltuner.pl
chmod 700 tuning-primer.sh mysqltuner.pl

perl /scripts/mysqltuner.pl
/scripts/tuning-primer.sh

you can play with this for mysql.

You have to find the template used by ispconfig to create every vhost (I don't know where to change it, but I am sure that you will find it :P). As I remember, with each update of ispconfig, ALL FILES are replaced so... you can make a script to use sed and reemplace the lines after each update, or just modify them manually after each update...

Till and Falko I think will have a better solution but... I think that 1 hour of idle life is not the best for a "generic installation". Maybe it could be the best for a server with 8-12 gb ram, but it is just an opinion (they know exactly why them set that timer :P)

[Ovidiu]

thanks, I know those. add this one to the list: http://hackmysql.com/mysqlreport its just that since this server actually has little traffic I have to wait for 1-2 days after each mysql config change I make :-)

lets see if Till or Falko add some idea as to where to permanently make that change.

[erosbk]

I think (99.9% sure :P) that you must change the time in this file:

/usr/local/ispconfig/server/conf/vhost.conf.master

But wait for Falko and Till post xD

Edit: line 158

[Ovidiu]

not 100% solved :-(

what does this tell you? http://zice.ro/munin/serverkompetenz...net/index.html

server got stuck last night, when I came to work today, all I saw with pstree were about 50 apache2 processes and about the same amount of php-cgi procs.

Can't spot anything unusual in the graphs, can you?

[erosbk]

Yes, your system is swaping unnecessarily.

You can try this:
echo 10 > /proc/sys/vm/swappiness

But I recommend make this permanent using (you can always rollback this to 60, make backup before of this file):

echo "vm/swappiness=10" >> /etc/sysctl.conf

Restart your system, and watch results.

All my servers are configured with that value, but I didn't have the oportunity yet to increase the load of the server to see if it behave as I want.

In your system, the kernel is using all the free memory as cache (it is good), but, because you are using almost 2gb of memory, Debian by default is prepared to begin swap early (60 as default value, read the explanations pls)... so... lowering it to 10, your system should stop swapping until (this is JUST A GUESS) 80-90% of your memory is in use, and not begin at 30-40% as it is happening right now...

You have your system, swap area and data in the same disks... all of them are fighting for (writte) the use of the disk and it slows down everything (swap mainly). In a future will be a good idea to separate system/swap from user data files.

Every time a user access a site, ir creates a cgi process in memory... if it is not used for 300 seconds, it will die... if it is used, it will remain in memory... if you let them stay idle in memory for 3600 secs, and a user access it until 3600 secs area reached, your process will live forever in memory... with 300 you free memory quickly, lowering the memory consumption BUT, if you have request to every site you are hosting, one request in less than 300 secs, well... you will increase the memory consumption, no matter if you set 300 or 3000000000 seconds... changing this setting only fixed one of your "problems" (free resources if them are not in using so... with this you lower your memory consumption and minimize your OS swap necessity). You main problem I think, is swapping...

Regards.

[Ovidiu]

hmm... I had set swappiness to 10 already when you fist proposed it so not sure whats going on here :-(

there might be some vhosts left without the timeout set from 3600 to 300...
talking about separating files from swap, yeah, I know but msot entry level root servers come with two equal sized HDs so it is meant to be used in a software raid...
besides, seeing the low usage of this server....