Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 3 > General

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 11th April 2011, 20:18
crypted crypted is offline
Senior Member
 
Join Date: Dec 2006
Location: Oklahoma, USA
Posts: 429
Thanks: 3
Thanked 14 Times in 6 Posts
Default Apache2 FCGID locking up the box?

My box continually locks up with loads of over 190.

In the Apache error log, I see this for several hours prior to the lockup:

Code:
[Mon Apr 11 13:43:28 2011] [warn] mod_fcgid: process 12274 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:34 2011] [warn] mod_fcgid: process 12304 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:34 2011] [warn] mod_fcgid: process 12297 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:34 2011] [warn] mod_fcgid: process 12295 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:34 2011] [warn] mod_fcgid: process 12299 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:34 2011] [warn] mod_fcgid: process 12296 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:34 2011] [warn] mod_fcgid: process 12274 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:40 2011] [warn] mod_fcgid: process 12304 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:40 2011] [warn] mod_fcgid: process 12297 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:40 2011] [warn] mod_fcgid: process 12295 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:40 2011] [warn] mod_fcgid: process 12299 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:40 2011] [warn] mod_fcgid: process 12296 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:40 2011] [warn] mod_fcgid: process 12274 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:46 2011] [warn] mod_fcgid: process 12304 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:46 2011] [warn] mod_fcgid: process 12297 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:46 2011] [warn] mod_fcgid: process 12295 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:46 2011] [warn] mod_fcgid: process 12299 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:46 2011] [warn] mod_fcgid: process 12296 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:46 2011] [warn] mod_fcgid: process 12274 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:52 2011] [warn] mod_fcgid: process 12304 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:52 2011] [warn] mod_fcgid: process 12297 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:43:52 2011] [warn] mod_fcgid: process 12295 graceful kill fail, sending SIGKILL
.....
[Mon Apr 11 13:53:51 2011] [warn] mod_fcgid: process 13068 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:53:51 2011] [warn] mod_fcgid: process 13066 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:54:44 2011] [warn] mod_fcgid: process 13081 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:55:06 2011] [warn] mod_fcgid: process 13109 graceful kill fail, sending SIGKILL    
[Mon Apr 11 13:55:20 2011] [warn] mod_fcgid: process 13109 graceful kill fail, sending SIGKILL   
[Mon Apr 11 13:55:26 2011] [warn] mod_fcgid: process 13112 graceful kill fail, sending SIGKILL
[Mon Apr 11 13:55:26 2011] [warn] mod_fcgid: process 13109 graceful kill fail, sending SIGKILL
In the end, I have to have the ISP hard-reboot the server so that it will be back online and accessible.

Any ideas? This has been destroying my productivity since switching to ISPC3 last year...
__________________
ISPC3 on Debian! It's great!
Reply With Quote
Sponsored Links
  #2  
Old 12th April 2011, 10:24
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 36,032
Thanks: 826
Thanked 5,381 Times in 4,228 Posts
Default

Therae are no known problems with ispconfig and fcgid in the current versions. Which ISPConfig version do you use, what are the hardware specifications of the server etc. And please post the fcgi starter file of a affected website.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #3  
Old 12th April 2011, 14:23
crypted crypted is offline
Senior Member
 
Join Date: Dec 2006
Location: Oklahoma, USA
Posts: 429
Thanks: 3
Thanked 14 Times in 6 Posts
Default

I'm running Debian Squeeze x64 (happened on Lenny x64 also) Linux 2.6.32-5-amd64 on x86_64.

It's a Intel(R) Pentium(R) D CPU 3.00GHz 2 cores, 4 GB RAM, and 2x 300GB SATA2 hdds.

I've got ISPC 3.0.3.2 installed.

Here are several FCGI starter scripts from almost all of the sites that end up with the crazy spawning of processes and lag crashing.
Code:
cat /var/www/php-fcgi-scripts/web11/.php-fcgi-starter 
#!/bin/sh
PHPRC="/var/www/conf/web11"
export PHPRC
PHP_DOCUMENT_ROOT="/var/www/clients/client1/web11"
export PHP_DOCUMENT_ROOT
# The variable PHP_FCGI_CHILDREN is onyl useful for lighty or nginx as apache 
# mod_fcgi will control the number of childs themself and never use the additional processes.
# PHP_FCGI_CHILDREN=1
# export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /usr/bin/php-cgi \
 -d open_basedir="/var/www/clients/client1/web11/web:/var/www/clients/client1/web11/tmp:/var/www/derekgordon.com/web:/srv/www/derekgordon.com/web:/usr/share/php5:/tmp:/usr/share/phpmyadmin" \
-d upload_tmp_dir=/var/www/clients/client1/web11/tmp \
-d session.save_path=/var/www/clients/client1/web11/tmp \
 $1
Code:
my:/home/dingle# cat /var/www/php-fcgi-scripts/web5/.php-fcgi-starter 
#!/bin/sh
PHPRC="/var/www/conf/web5"
export PHPRC
PHP_DOCUMENT_ROOT="/var/www/clients/client1/web5"
export PHP_DOCUMENT_ROOT
# The variable PHP_FCGI_CHILDREN is onyl useful for lighty or nginx as apache 
# mod_fcgi will control the number of childs themself and never use the additional processes.
# PHP_FCGI_CHILDREN=1
# export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /usr/bin/php-cgi \
 -d open_basedir="/var/www/clients/client1/web5/web:/var/www/clients/client1/web5/tmp:/var/www/areyouliberal.com/web:/srv/www/areyouliberal.com/web:/usr/share/php5:/tmp:/usr/share/phpmyadmin" \
-d upload_tmp_dir=/var/www/clients/client1/web5/tmp \
-d session.save_path=/var/www/clients/client1/web5/tmp \
export LD_PRELOAD = amon.so
 $1
Code:
my:/home/dingle# cat /var/www/php-fcgi-scripts/web8/.php-fcgi-starter 
#!/bin/sh
PHPRC="/var/www/conf/web8"
export PHPRC
PHP_DOCUMENT_ROOT="/var/www/clients/client1/web8"
export PHP_DOCUMENT_ROOT
# The variable PHP_FCGI_CHILDREN is onyl useful for lighty or nginx as apache 
# mod_fcgi will control the number of childs themself and never use the additional processes.
# PHP_FCGI_CHILDREN=1
# export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /usr/bin/php-cgi \
 -d open_basedir="/var/www/clients/client1/web8/web:/var/www/clients/client1/web8/tmp:/var/www/inter5.org/web:/srv/www/inter5.org/web:/usr/share/php5:/tmp:/usr/share/phpmyadmin" \
-d upload_tmp_dir=/var/www/clients/client1/web8/tmp \
-d session.save_path=/var/www/clients/client1/web8/tmp \
export LD_PRELOAD = amon.so
 $1
Code:
my:/home/dingle# cat /var/www/php-fcgi-scripts/web35/.php-fcgi-starter 
#!/bin/sh
PHPRC="/var/www/conf/web35"
export PHPRC
PHP_DOCUMENT_ROOT="/var/www/clients/client1/web35"
export PHP_DOCUMENT_ROOT
# The variable PHP_FCGI_CHILDREN is onyl useful for lighty or nginx as apache 
# mod_fcgi will control the number of childs themself and never use the additional processes.
# PHP_FCGI_CHILDREN=1
# export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /usr/bin/php-cgi \
 -d open_basedir="/var/www/clients/client1/web35/web:/var/www/clients/client1/web35/tmp:/var/www/masonicleaks.com/web:/srv/www/masonicleaks.com/web:/usr/share/php5:/tmp:/usr/share/phpmyadmin" \
-d upload_tmp_dir=/var/www/clients/client1/web35/tmp \
-d session.save_path=/var/www/clients/client1/web35/tmp \
 $1my:/home/dingle#
__________________
ISPC3 on Debian! It's great!
Reply With Quote
  #4  
Old 12th April 2011, 14:56
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 36,032
Thanks: 826
Thanked 5,381 Times in 4,228 Posts
Default

Does these lines are really like this:

-d session.save_path=/var/www/clients/client1/web5/tmp \
export LD_PRELOAD = amon.so
$1

it has to be:

-d session.save_path=/var/www/clients/client1/web5/tmp \
$1

The other parts of the file look fine. You should check your apache configuration file and limit the number of apache instances that get spawned.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #5  
Old 12th April 2011, 15:05
crypted crypted is offline
Senior Member
 
Join Date: Dec 2006
Location: Oklahoma, USA
Posts: 429
Thanks: 3
Thanked 14 Times in 6 Posts
Default

I was playing around with amon earlier. So, those entries weren't always there.

When you say the max instances, are you talking about the FCGI part in the vhosts files?

If so, what would you recommend the settings to be?
__________________
ISPC3 on Debian! It's great!
Reply With Quote
  #6  
Old 12th April 2011, 15:24
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 36,032
Thanks: 826
Thanked 5,381 Times in 4,228 Posts
Default

Quote:
When you say the max instances, are you talking about the FCGI part in the vhosts files?
I'am talking about the mpm settings in the apache2.conf file.

But you can take a look in the vhost file as well, be default the values in the vhost file should be:

# IdleTimeout n (3600 seconds)
# An idle fastcgi application will be terminated after IdleTimeout seconds.
IdleTimeout 3600

# ProcessLifeTime n (7200 seconds)
# A fastcgi application will be terminated if lifetime expired, even no error is detected.
ProcessLifeTime 7200

# MaxProcessCount n (1000)
# The max count of total fastcgi process count.
# MaxProcessCount 1000

# DefaultMinClassProcessCount n (3)
# The minimum number of fastcgi application instances for any one fastcgi application.
# Idle fastcgi will not be killed if their count is less than n
# Set this to 0, and tweak IdleTimeout
DefaultMinClassProcessCount 0

# DefaultMaxClassProcessCount n (100)
# The maximum number of fastcgi application instances allowed to run for
# particular one fastcgi application.
DefaultMaxClassProcessCount 100

# IPCConnectTimeout n (3 seconds)
# The connect timeout to a fastcgi application.
IPCConnectTimeout 8

# IPCCommTimeout n (20 seconds)
# The communication timeout to a fastcgi application. Please increase this
# value if your CGI have a slow initialization or slow respond.
IPCCommTimeout 360

# BusyTimeout n (300 seconds)
# A fastcgi application will be terminated if handing a single request
# longer than busy timeout.
BusyTimeout 300


you can e.g. tune the MaxProcessCount and DefaultMaxClassProcessCount parameters.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #7  
Old 12th April 2011, 16:16
crypted crypted is offline
Senior Member
 
Join Date: Dec 2006
Location: Oklahoma, USA
Posts: 429
Thanks: 3
Thanked 14 Times in 6 Posts
 
Default

Right, and in apache2 I see the following with mpm involved:

Code:
IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10           
    MaxClients          150
    MaxRequestsPerChild   0
</IfModule>
Code:
<IfModule mpm_worker_module>
    StartServers          2
    MaxClients          150
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadsPerChild      25
    MaxRequestsPerChild   0
</IfModule>
what would you recommend for changing? Would max clients of 50 be too low? With a MaxSpareThreads of 45?
__________________
ISPC3 on Debian! It's great!
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ISPConfig 3.0.1.4 causes Apache to have too many open files gkovacs Installation/Configuration 29 27th February 2013 08:59
Whole server went down wxman General 25 28th September 2012 18:32
SMS Gateway Ubuntu server 9.10 Playsms Smstools3. letsharehowto HOWTO-Related Questions 13 1st September 2012 20:32
yet another apache2 tuning thread Ovidiu Server Operation 65 5th July 2011 15:07
ISPConfig 3 Security mnzava Installation/Configuration 8 8th March 2010 11:00


All times are GMT +2. The time now is 19:54.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.