Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > Linux Forums > Server Operation

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #41  
Old 13th May 2011, 15:52
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Sorry, I did'nt researched if it was or not deprecated. You get the idea, if you know that is deprecated, use what you propose instead!

If you can experiment a little, it could be ca good idea to change the values in every vhost. For sure you will have cgi process for less time in memory. Debian will continue using cache, but you can limit swap. Tunning a web server is an every day works :P
Reply With Quote
Sponsored Links
  #42  
Old 16th May 2011, 19:28
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,238
Thanks: 70
Thanked 21 Times in 17 Posts
Default

seems web9 was the culprit. I changed the proposed timeout to 300 from originally 3600 and it seems to work.

still very high usage of resources but at least I know who and why is using them: http://zice.ro/munin/serverkompetenz...net/index.html

every time this client sends his newsletter to about 10.000 recipients (all verified, no spam) the server kinda bogged down. looks better right now with the timeout set to 300 (at least that is the only thing I changed and it works) - not sure if that is the cause for it working better now.

still, back to the newsletter, I have set it to send out 50 mails / minute and its using SMTP.

the server is set up according to this how-to: http://www.howtoforge.com/perfect-se...er-ispconfig-3

the only sifference is that I didn't start from scratch but with the strato minimal installation which means the whole thing is running on a software raid 1 - possibly this is the bottleneck?

any good links for checking/tuning software raids or isn't there much to tune?

###edit###
forgot to add that the newsletter is a wordpress plugin and relies on a certain wordpress function that mimics cron, basically it depends on having visitors to trigger so if nobody visits the site, the newsleter doesn't get sent as its never triggered... just for your info.

Last edited by Ovidiu; 16th May 2011 at 21:16.
Reply With Quote
  #43  
Old 16th May 2011, 20:05
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,586
Thanks: 792
Thanked 4,983 Times in 3,903 Posts
Default

I guess the only part where you can tune the system is to check and rewrite the php code of the neslwtter addon to scale better for large newsletters. Normally such a software should use not many resources, but if it e.g. tries to load all subscriber data into an large array at once or similar mistakes, then it can bring down a server.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #44  
Old 19th May 2011, 20:34
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,238
Thanks: 70
Thanked 21 Times in 17 Posts
Default

IdleTimeout 300 saved me :-)

no more idle php-cgi processes sitting around.
now I am onto debugging the HD I/O and latency, found the problem too: mysql.

the million dollar question is how can I set this as a default parameter for all new sites that will be created and keep that change persistent even if I update ispcfg3?

when moving to this server it seems I took over almost all relevant settings except that mysql was running with some default configuration and nothing was cached so sending 10.000 was killing the HD because of all those uncached DB accesses.
finetuning mysql right now to balance things out so that I am neither using too much RAM for the cache nor have too many uncached HD accesses.
Reply With Quote
  #45  
Old 19th May 2011, 20:44
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

mkdir /scripts/
cd /scripts
wget http://www.day32.com/MySQL/tuning-primer.sh
wget http://mysqltuner.pl/mysqltuner.pl
chmod 700 tuning-primer.sh mysqltuner.pl

perl /scripts/mysqltuner.pl
/scripts/tuning-primer.sh

you can play with this for mysql.

You have to find the template used by ispconfig to create every vhost (I don't know where to change it, but I am sure that you will find it :P). As I remember, with each update of ispconfig, ALL FILES are replaced so... you can make a script to use sed and reemplace the lines after each update, or just modify them manually after each update...

Till and Falko I think will have a better solution but... I think that 1 hour of idle life is not the best for a "generic installation". Maybe it could be the best for a server with 8-12 gb ram, but it is just an opinion (they know exactly why them set that timer :P)
Reply With Quote
  #46  
Old 19th May 2011, 20:52
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,238
Thanks: 70
Thanked 21 Times in 17 Posts
Default

thanks, I know those. add this one to the list: http://hackmysql.com/mysqlreport its just that since this server actually has little traffic I have to wait for 1-2 days after each mysql config change I make :-)

lets see if Till or Falko add some idea as to where to permanently make that change.
Reply With Quote
  #47  
Old 19th May 2011, 22:48
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

I think (99.9% sure :P) that you must change the time in this file:

/usr/local/ispconfig/server/conf/vhost.conf.master

But wait for Falko and Till post xD

Edit: line 158
Reply With Quote
  #48  
Old 24th May 2011, 09:16
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,238
Thanks: 70
Thanked 21 Times in 17 Posts
Default

not 100% solved :-(

what does this tell you? http://zice.ro/munin/serverkompetenz...net/index.html

server got stuck last night, when I came to work today, all I saw with pstree were about 50 apache2 processes and about the same amount of php-cgi procs.

Can't spot anything unusual in the graphs, can you?
Reply With Quote
  #49  
Old 24th May 2011, 17:56
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Yes, your system is swaping unnecessarily.

You can try this:
echo 10 > /proc/sys/vm/swappiness

But I recommend make this permanent using (you can always rollback this to 60, make backup before of this file):

echo "vm/swappiness=10" >> /etc/sysctl.conf

Restart your system, and watch results.

All my servers are configured with that value, but I didn't have the oportunity yet to increase the load of the server to see if it behave as I want.

In your system, the kernel is using all the free memory as cache (it is good), but, because you are using almost 2gb of memory, Debian by default is prepared to begin swap early (60 as default value, read the explanations pls)... so... lowering it to 10, your system should stop swapping until (this is JUST A GUESS) 80-90% of your memory is in use, and not begin at 30-40% as it is happening right now...

You have your system, swap area and data in the same disks... all of them are fighting for (writte) the use of the disk and it slows down everything (swap mainly). In a future will be a good idea to separate system/swap from user data files.

Every time a user access a site, ir creates a cgi process in memory... if it is not used for 300 seconds, it will die... if it is used, it will remain in memory... if you let them stay idle in memory for 3600 secs, and a user access it until 3600 secs area reached, your process will live forever in memory... with 300 you free memory quickly, lowering the memory consumption BUT, if you have request to every site you are hosting, one request in less than 300 secs, well... you will increase the memory consumption, no matter if you set 300 or 3000000000 seconds... changing this setting only fixed one of your "problems" (free resources if them are not in using so... with this you lower your memory consumption and minimize your OS swap necessity). You main problem I think, is swapping...

Regards.

Last edited by erosbk; 24th May 2011 at 18:01.
Reply With Quote
  #50  
Old 24th May 2011, 21:15
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,238
Thanks: 70
Thanked 21 Times in 17 Posts
 
Default

hmm... I had set swappiness to 10 already when you fist proposed it so not sure whats going on here :-(

there might be some vhosts left without the timeout set from 3600 to 300...
talking about separating files from swap, yeah, I know but msot entry level root servers come with two equal sized HDs so it is meant to be used in a software raid...
besides, seeing the low usage of this server....

Last edited by Ovidiu; 24th May 2011 at 21:17.
Reply With Quote
Reply

Bookmarks

Tags
apache2, server, tuning

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ISPConfig 3.0.1.4 causes Apache to have too many open files gkovacs Installation/Configuration 29 27th February 2013 08:59
Whole server went down wxman General 25 28th September 2012 18:32
about: tuning apache2 Ovidiu HOWTO-Related Questions 26 2nd February 2011 12:28
ISPConfig 3 Security mnzava Installation/Configuration 8 8th March 2010 11:00
Swap usage issue MrM General 23 7th October 2009 10:52


All times are GMT +2. The time now is 07:40.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.