Server load / top results

[unsichtbare]

My server seems to have unusually high load for a webserver hosting about 7 sites with almost no traffic. Can anyone explain why Apache runs at 100% (or close) most of the time?

Here are the specs: Ubuntu 7.0.4 (The Perfect etup) ISP Config, 1GB RAM, 2-2.4ghz proc.

results of top:

top - 21:17:51 up 1:57, 1 user, load average: 1.53, 1.45, 1.41
Tasks: 92 total, 4 running, 88 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.7%us, 0.3%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1035704k total, 665192k used, 370512k free, 25296k buffers
Swap: 1759076k total, 0k used, 1759076k free, 432816k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4862 www-data 25 0 55552 25m 4832 R 100 2.6 16:54.68 apache2
6523 www-data 25 0 55528 26m 4520 R 100 2.6 4:30.78 apache2
6176 root 15 0 8028 2452 1996 R 0 0.2 0:02.41 sshd
1 root 18 0 2908 1844 524 S 0 0.2 0:05.64 init
2 root RT 0 0 0 0 S 0 0.0 0:00.11 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:00.01 watchdog/0
5 root RT 0 0 0 0 S 0 0.0 0:00.22 migration/1
6 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1
7 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/1
8 root 10 -5 0 0 0 S 0 0.0 0:00.02 events/0
9 root 10 -5 0 0 0 S 0 0.0 0:00.06 events/1
10 root 10 -5 0 0 0 S 0 0.0 0:00.16 khelper
11 root 10 -5 0 0 0 S 0 0.0 0:00.01 kthread
35 root 10 -5 0 0 0 S 0 0.0 0:00.10 kblockd/0
36 root 11 -5 0 0 0 S 0 0.0 0:04.59 kblockd/1
37 root 16 -5 0 0 0 S 0 0.0 0:00.00 kacpid
38 root 16 -5 0 0 0 S 0 0.0 0:00.00 kacpi_notify
94 root 10 -5 0 0 0 S 0 0.0 0:00.03 kseriod
115 root 16 0 0 0 0 S 0 0.0 0:00.00 pdflush
116 root 15 0 0 0 0 S 0 0.0 0:03.10 pdflush
117 root 11 -5 0 0 0 S 0 0.0 0:00.00 kswapd0
118 root 11 -5 0 0 0 S 0 0.0 0:00.00 aio/0
119 root 11 -5 0 0 0 S 0 0.0 0:00.00 aio/1
749 root 15 0 0 0 0 S 0 0.0 0:00.01 kirqd
1866 root 16 -5 0 0 0 S 0 0.0 0:00.00 ata/0
1868 root 16 -5 0 0 0 S 0 0.0 0:00.00 ata/1
1870 root 16 -5 0 0 0 S 0 0.0 0:00.00 ata_aux
1940 root 15 -5 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
2093 root 10 -5 0 0 0 S 0 0.0 0:05.43 kjournald
2224 root 21 -4 2296 616 356 S 0 0.1 0:03.02 udevd
3002 root 14 -5 0 0 0 S 0 0.0 0:00.00 kpsmoused
3603 root 18 0 1648 508 440 S 0 0.0 0:00.02 getty
3604 root 18 0 1652 512 440 S 0 0.0 0:00.03 getty
3606 root 18 0 1652 512 440 S 0 0.0 0:00.03 getty
3607 root 18 0 1652 508 440 S 0 0.0 0:00.04 getty
3610 root 18 0 1652 508 440 S 0 0.0 0:00.03 getty

-John

[falko]

Are there any errors in Apache's error log?
Do you use eaccelerator for your PHP? http://www.eaccelerator.net/

[wpwood3]

Post your http.conf LESS the comments.

Be sure to check your setting for KeepAliveTimeout in http.conf
Setting this to 1 can make a big difference in reducing server load.

These settings also affect server load:

<IfModule prefork.c>
StartServers
MinSpareServers
MaxSpareServers
ServerLimit
MaxClients
MaxRequestsPerChild
</IfModule>

Be sure to restart Apache if you make any changes to http.conf.

Also, as Falko suggested, a good php accelerator can make a big difference if your website contains lots pf php. I use XCache and find it to be better than eAccelerator but YMMV.

[unsichtbare]

Here is a day from my Apache error log

[Sat Jan 05 08:05:35 2008] [error] [client 78.108.51.68] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /w00tw00t.at.ISC.SANS.DFind
[Sat Jan 05 18:48:17 2008] [error] [client 199.227.134.114] Invalid method in request \x80L\x01\x03
[Sat Jan 05 19:19:10 2008] [notice] caught SIGTERM, shutting down
[Sat Jan 05 19:21:13 2008] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
PHP Warning: Module 'json' already loaded in Unknown on line 0
[Sat Jan 05 19:21:17 2008] [notice] Apache/2.2.3 (Ubuntu) PHP/5.2.1 mod_ssl/2.2.3 OpenSSL/0.9.8c configured -- resuming normal operations
[Sat Jan 05 19:21:30 2008] [notice] caught SIGTERM, shutting down
[Sat Jan 05 19:21:40 2008] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
PHP Warning: Module 'json' already loaded in Unknown on line 0
[Sat Jan 05 19:21:41 2008] [notice] Apache/2.2.3 (Ubuntu) PHP/5.2.1 mod_ssl/2.2.3 OpenSSL/0.9.8c configured -- resuming normal operations

also:

PHP Warning: Module 'json' already loaded in Unknown on line 0

I have installed Xcache, but performance was never the issue. I am concerned that Apache seems to always want 100% of one or more proc's.

<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0
</IfModule>

-J

[wpwood3]

Originally Posted by unsichtbare

PHP Warning: Module 'json' already loaded in Unknown on line 0

That means that this module is being loaded twice. You need to edit /etc/php.ini and comment out the line extension=json.so like this:

 
;extension=json.so

Then restart Apache.
This should eliminate the error but it's unlikely to be the cause of your real problem.

This error may mean your SSL certificate is corrupt:

 
[Sat Jan 05 18:48:17 2008] [error] [client 199.227.134.114] Invalid method in request \x80L\x01\x03

Here's a thread that may help fix that:
http://www.howtoforge.com/forums/showthread.php?t=121

Your real problem is likely a runaway php process due to a bad script.

One other thought. If you use MySQL then you should check and optimize the tables in your databases. phpMyAdmin works great for that.

[unsichtbare]

Would it be feasable to turn PHP off one site at a time to determine where the problem is? All of the sites are migrated from other servers, so it is very possible that there are problemo of which I am unaware.

-J

[wpwood3]

Originally Posted by unsichtbare

Would it be feasable to turn PHP off one site at a time to determine where the problem is?

Yes
That's easy to do in ISPConfig. Go to ISP Manager and click on each site. The Basis tab has an option to turn PHP Scripts on and off.
Be sure to restart Apache after the change.