HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials

HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials (http://www.howtoforge.com/forums/index.php)
-   Server Operation (http://www.howtoforge.com/forums/forumdisplay.php?f=5)
-   -   Thrashing issues with mpm-itk (http://www.howtoforge.com/forums/showthread.php?t=62140)

mzandstra 7th June 2013 19:25

Thrashing issues with mpm-itk
 
Hi,

I'm using a 2 VPSes (1 apache, 1 mysql) to run ~30 php/mysql sites. It's a shared-user environment, so I went with mpm-itk. My problem now is that apache seems to be ignoring my ServerLimit and MaxClients settings and ending up in swap, which causes all the sites to crash.

Relevant chunk of httpd.conf:

Code:

<IfModule mpm_itk_module>
        ServerLimit            30
        StartServers            5
        MinSpareServers          3
        MaxSpareServers        7
        MaxClients              30
        MaxRequestsPerChild  1000
        AssignUserId apache apache
</IfModule>

I've confirmed that mpm-itk is being used, and this is CentOS 5.5 with 1 GB RAM. I wrote a shell script / cron job to check apache's memory usage every 5 minutes, here's some of the output:

Code:

06-07-2013 14:55:02 - 76 MB Free with 52 httpd processes of average size 20895
06-07-2013 15:00:01 - 319 MB Free with 62 httpd processes of average size 18776
06-07-2013 15:05:02 - 137 MB Free with 58 httpd processes of average size 20388
06-07-2013 15:10:02 - 94 MB Free with 55 httpd processes of average size 22005
06-07-2013 15:15:03 - 91 MB Free with 55 httpd processes of average size 20211
06-07-2013 15:20:02 - 121 MB Free with 62 httpd processes of average size 18422
06-07-2013 15:25:01 - 859 MB Free with 10 httpd processes of average size 12281
06-07-2013 15:30:01 - 845 MB Free with 10 httpd processes of average size 11435
06-07-2013 15:35:01 - 852 MB Free with 10 httpd processes of average size 10672

As you can see, apache is going way over the limit of 30 threads I set. Anyone know why this is? I installed mpm-itk from the CentOS repo, not as a patch.

Thanks,
-michael

mzandstra 17th June 2013 23:06

*Shameless Bump*
 
*Tap* *Tap* Is this thing on?

mzandstra 21st June 2013 17:16

Would Apache reading the mpm-itk config first be the issue? I mean, perhaps the prefork settings under
Code:

<IfModule prefork.c>
StartServers      8
MinSpareServers    5
MaxSpareServers  20
ServerLimit      18
MaxClients      18
MaxRequestsPerChild  4000
</IfModule>

are overwriting my itk settings because they are read later?

Server crashed again this morning... You can see a large chunk of missing output here from when the server was down:

Code:

06-21-2013 12:46:16 - 65 MB Free with 76 httpd processes of average size 18155
06-21-2013 13:00:56 - 60 MB Free with 104 httpd processes of average size 13890
06-21-2013 13:00:56 - 60 MB Free with 103 httpd processes of average size 13885
06-21-2013 13:50:07 - 77 MB Free with 101 httpd processes of average size 11852
06-21-2013 13:50:07 - 77 MB Free with 102 httpd processes of average size 11736
06-21-2013 13:01:09 - 74 MB Free with 93 httpd processes of average size 12537
06-21-2013 13:50:10 - 621 MB Free with 105 httpd processes of average size 9546

My other thought is that perhaps httpd isn't properly killing the itk processes after 4000 requests like it should be... Thoughts?

mzandstra 25th June 2013 21:53

I was able to get a piece of the main httpd error log right as this issue was occuring

Code:

[Tue Jun 25 13:43:08 2013] [error] server reached MaxClients setting, consider raising the MaxClients setting
[Tue Jun 25 13:43:10 2013] [error] child died with signal 9
[Tue Jun 25 13:43:47 2013] [error] child died with signal 9
[Tue Jun 25 13:43:49 2013] [error] child died with signal 9
[Tue Jun 25 13:43:55 2013] [error] child died with signal 9
[Tue Jun 25 13:43:59 2013] [error] child died with signal 9
[Tue Jun 25 13:44:03 2013] [error] child died with signal 9
[Tue Jun 25 13:44:05 2013] [error] child died with signal 9
[Tue Jun 25 13:44:07 2013] [error] child died with signal 9
[Tue Jun 25 13:44:08 2013] [error] child died with signal 9
[Tue Jun 25 13:44:09 2013] [error] child died with signal 9
[Tue Jun 25 13:44:10 2013] [error] child died with signal 9
[Tue Jun 25 13:44:12 2013] [error] child died with signal 9
[Tue Jun 25 13:44:13 2013] [error] child died with signal 9
[Tue Jun 25 13:44:13 2013] [error] child died with signal 9
[Tue Jun 25 13:44:15 2013] [error] child died with signal 9
[Tue Jun 25 13:45:18 2013] [error] child died with signal 9
[Tue Jun 25 13:45:19 2013] [error] child died with signal 9
[Tue Jun 25 13:45:29 2013] [error] child died with signal 9

From my memory monitor:

Code:

06-25-2013 13:44:02 - 63 MB Free with 42 httpd processes of average size 24894
My question is, why did httpd wait for those 42 processes to spawn before killing them off? It should never have let that many spawn in the first place.

mzandstra 8th July 2013 15:53

Still waiting for a response here. Has nobody had this problem before? It's causing major issues for me.


All times are GMT +2. The time now is 01:41.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.