I need help to debug a slow website

[vtapas]

Hi,I am running a website behind a reverse proxy in following environment.

 Server A --------------------Server B (site running eduCommons on Plone)
(running apache reverse        (back end server)
 proxy)

Server A and B are on same physical machine.
B is a guest Operating System on A.

Operating System on Server B is CentOS 5.5
Operating System on Server A is Ubuntu 10.04
eduCommons is a content management system.
You check about it here.


recorded the website loading time in the Chrome developer tools time line section and
based on what I observed

following page elements take more than 100 ms on the website
jquery checks
113 ms http://192.168.1.5:8080/eduCommons/p...achekey0517.js

244ms http://192.168.1.5:8080/eduCommons/p...achekey1443.js
247ms http://192.168.1.5:8080/eduCommons/p...chekey5402.css

like this there are some more css and page elements which have taken
100ms or 200ms in their GET request from server.

on this link
http://www.chicagostyleseo.com/2010/...-site-is-slow/
they mentioned what Google has defined as a slow website following
words are mentioned.

"Google has made it clear that any process that takes more than 100
milliseconds (1/10 of a second) is too long"

So that means the website is slow.

In fact I have asked a few people to access the website on internet and they too reported the same.

In a step by step way then to debug the problem I did following

1) The site http://site5.abc.com was loading slowly
(Some page elements showed 411ms in Google Developer tools snapshot here)
2) From my laptop checked http://192.168.1.5:8080/eduCommons

( got a few page elements 211 ms and one was 770 ms a snapshot for the 700 ms attached here)

3) Then to check if the site is loading slow on CentOS server itself I did


ssh -L 8080:localhost:8080 root@192.168.1.5

4) Then from my laptop when above ssh connection on step 3 was alive

http://localhost:8080/eduCommons
and saw a page element loading at 411 ms.


If I am not wrong then by step 4 it is confirming that site is slow on server itself.


I enabled %D in /etc/httpd/conf/httpd.conf

LogFormat "%a %D %>s %B" resp
CustomLog /var/log/httpd/site_in_question-resp_log resp

but after hitting the website from different ISPs I saw there was no log in

/var/log/httpd/site_in_question-resp_log resp

as expected.
I thought that it might be permission so it I changed

chown apache:apache /var/log/httpd/site_in_question-resp_log

I did in vhost configuration of file on CentOS Guest OS on Ubuntu Server 10.04

On the other hand I tried to do the same on reverse proxy server
and then apache failed to restart.
Upon checking I found
a2enmod mod_log_config

ERROR: Module mod_log_config does not exist!

Searched but any thing fruitful did not came.
As per your apache documentation page I searched for this file

find / -name mod_log_config.c

but it was not there.



I would like to clear that the server is not running any thing other than eduCommons .
The server on which I am running it is physically mine.
8 GB RAM ,500 G hard disk and more than 256 kbps bandwidth.

What more should I do to check it and how can I over come this problem?

[Rapid2214]

Originally Posted by vtapas

1) The site http://site5.abc.com was loading slowly

By this are you impling that external sites are slow?
The most probable reason would be the proxy, try take that out the equation and see what you get.

Good luck

Marty

[vtapas]

Originally Posted by Rapid2214

By this are you impling that external sites are slow?
The most probable reason would be the proxy, try take that out the equation and see what you get.

Good luck :D

Marty

I could not understand your point completely.
The people on internet will be able to access my website when in browser they type
http://site5.abc.com

above is just a dummy name.

The site is hosted internally on a webserver.
Any how I will give some more information after posting here I tried some thing

After reading this page

I added in /etc/httpd/conf/httpd.conf on Server B (CentOS Guest OS)

<IfDefine RequestTime>
     Header set X-Request-Received: %t
     Header set X-Request-Processing-Time: %D
</IfDefine>

I am able to see some response log. in /var/log/httpd/site-in-question-resp
Now tell me what should I post

192.168.1.5 3062 403 5043
192.168.1.5 2826 403 5043

What does this tell me ?
What more should I do to check it and how can I over come this problem?

[Rapid2214]

So the web server is slow on a local connection? Then you have a problem.
Try 'top' or 'free' and see if the webservers have limited resorces also try 'df -h' too see if its a disk space problem.

The 403 part of that message would indicate a permission error, in theroy you should get 200 :S


Good luck

Marty

[vtapas]

Originally Posted by Rapid2214

So the web server is slow on a local connection? Then you have a problem.
Try 'top' or 'free' and see if the webservers have limited resorces also try 'df -h' too see if its a disk space problem.


Good luck :D

Marty

Well I Can not say exactly what you mean by slow on a local connection if some one on my Lan is browsing the website he is getting instant response if another user outside my lan tries to access the website on internet then it is giving a problem.The server on which website is running has 8 GB Ram and 500 Gb hard disk which is 88% free.There is no heavy load since the site is under construction.

[Rapid2214]

Ok, thanks for clearing that up.
Whats your upload speed? http://speedtest.net

Normally the upload speed will be the problem if a wan connection is appearing slow.

Pls post the 'Forum Link' here

[vtapas]

Download speed 6.02 Mbps
Upload speed 18.78 Mbps

[Rapid2214]

Oh man.
Umm you post you apache config file '/etc/apache2/apache2.conf'

[vtapas]

Ok on CentOS which in above diagram I gave as Sever B
I posted the file here
http://pastebin.com/8uwMxbPS
and for Server A it is in /etc/apache2/sites-enabled

site5.abc.com

ProxyPass  /  http://192.168.1.5:8080/eduCommons
ProxyPassReverse / http://192.168.1.5:8080/eduCommons

[Rapid2214]

#
#Listen 12.34.56.78:80
Listen 80
#Listen 8080

I see that you are requesting a page on port 8080 but the webserver isn't listening on that port, try uncomment it and restart the web server, only a suggestion.

Good luck

Marty