HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials

HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials (http://www.howtoforge.com/forums/index.php)
-   General (http://www.howtoforge.com/forums/forumdisplay.php?f=25)
-   -   apache log issues (http://www.howtoforge.com/forums/showthread.php?t=47559)

abubin 26th July 2010 11:09

apache log issues
 
My website is only serving around 10 websites.

One of the website is very active thus generating lots of logs.

I just found out about this only when I got alerted for diskspace running out email.

I tried searching in ispconfig and there are no option to disable logging for certain website.

I am using ispconfig 3.0.1.6.

Can I know where I can disable http logging for specified websites?? I think this is a pretty important feature and I assume it is already implemented? Maybe I didn't find properly?

What about compressions on the logs? Is there any?

What about log rotate?

As of now, these 3 features seems like not available. Without these, the logs will build up like crazy, even for a normal website, the logs generated daily is around 1GB already. With compression, this could reduce to 100mb.

Please help....thanks.

abubin 26th July 2010 11:18

Just manage to view back one of the thread about compression of log.

http://www.howtoforge.com/forums/sho...light=compress

I looked at my cron_daily.php file and there are lines already added for compression of logs and also removal of logs more than 30 days. But seems like this is not running properly. Don't know why...anyone can take a look into this?

abubin 26th July 2010 11:32

When I rung the cron_daily.php manually,

I get error: Warning: Truncating oversized referrer field

This could be the thing causing the failed access_log files processing?

till 26th July 2010 12:24

Quote:

This could be the thing causing the failed access_log files processing?
No. The truncate error is normal and happens on log lines that are too long.

Please update your server to ispconfig 3.0.2.2 as it has an improved log handling.

abubin 28th July 2010 08:18

i have upgrade to 3.0.2.2. There are no additional function for handling logs per website. I am assuming the improved log handling you meant was the compression and 30 days removal script is working now?

If not, please do point me to the right direction.

Thanks.

abubin 28th July 2010 12:11

i have looked at another server that is using 3.0.2.1. The logs compression is there for most of the domains except for 2. One is as11 and as11.domain.com. FYI, as11 is the server's hostname. So it's like the compression doesn't work on local hostname.

Upgrade to 3.0.2.2?

abubin 29th July 2010 05:46

I left the upgraded 3.0.2.2 server overnight. Checked the access logs in /var/log/ispconfig/httpd and no compressed access log file. The log file of the previous day is still uncompressed.

Please help....

till 29th July 2010 10:41

Please post the output of:

ls -la /var/log/ispconfig/httpd/yourdomain.com

replace yourdomain.com with the domain name of a affected website. The websites that are affected had been created in ispconfig or have you created them manually?

abubin 29th July 2010 11:46

server 1 (3.0.2.2):
- all access log also does not compress nor delete files older than 30 days:


ls -la /var/log/ispconfig/httpd/127.0.0.1
Code:

-rw-r--r--  1 root root  59973494 2010-07-01 23:59 20100701-access.log.gz
-rw-r--r--  1 root root  57519689 2010-07-02 23:59 20100702-access.log.gz
-rw-r--r--  1 root root  56580180 2010-07-03 23:59 20100703-access.log.gz
-rw-r--r--  1 root root  54573798 2010-07-04 23:59 20100704-access.log.gz
-rw-r--r--  1 root root  58566076 2010-07-05 23:59 20100705-access.log.gz
-rw-r--r--  1 root root  61830847 2010-07-06 23:59 20100706-access.log.gz
-rw-r--r--  1 root root  58912995 2010-07-07 23:59 20100707-access.log.gz
-rw-r--r--  1 root root  57487132 2010-07-08 23:59 20100708-access.log.gz
-rw-r--r--  1 root root  52348721 2010-07-09 23:59 20100709-access.log.gz
-rw-r--r--  1 root root  52555526 2010-07-10 23:59 20100710-access.log.gz
-rw-r--r--  1 root root  53555419 2010-07-11 23:59 20100711-access.log.gz
-rw-r--r--  1 root root  38464667 2010-07-12 23:59 20100712-access.log.gz
-rw-r--r--  1 root root  44748612 2010-07-13 23:59 20100713-access.log.gz
-rw-r--r--  1 root root  49687442 2010-07-14 23:59 20100714-access.log.gz
-rw-r--r--  1 root root  87718804 2010-07-15 23:59 20100715-access.log.gz
-rw-r--r--  1 root root  89333655 2010-07-16 23:59 20100716-access.log.gz
-rw-r--r--  1 root root  87059890 2010-07-17 23:59 20100717-access.log.gz
-rw-r--r--  1 root root  85310862 2010-07-18 23:59 20100718-access.log.gz
-rw-r--r--  1 root root  127998030 2010-07-19 23:59 20100719-access.log.gz
-rw-r--r--  1 root root 1830110297 2010-07-20 23:59 20100720-access.log
-rw-r--r--  1 root root 2011749233 2010-07-21 23:59 20100721-access.log
-rw-r--r--  1 root root 1764172052 2010-07-22 23:59 20100722-access.log
-rw-r--r--  1 root root 1754949565 2010-07-23 23:59 20100723-access.log
-rw-r--r--  1 root root 1677808377 2010-07-24 23:59 20100724-access.log
-rw-r--r--  1 root root 1670813799 2010-07-25 23:59 20100725-access.log
-rw-r--r--  1 root root 1729431133 2010-07-26 23:59 20100726-access.log
-rw-r--r--  1 root root 1687999078 2010-07-27 23:59 20100727-access.log
-rw-r--r--  1 root root 1700423428 2010-07-28 23:59 20100728-access.log
-rw-r--r--  1 root root 1429171939 2010-07-29 17:31 20100729-access.log
lrwxrwxrwx  1 root root        19 2010-07-29 12:48 access.log -> 20100729-access.log


ls -la /var/log/ispconfig/httpd/vn.domain.com/
Code:

-rw-r--r--  1 root root    206761957 2010-07-16 23:59 20100716-access.log.gz
-rw-r--r--  1 root root    174249432 2010-07-17 23:59 20100717-access.log.gz
-rw-r--r--  1 root root    149160599 2010-07-18 23:59 20100718-access.log.gz
-rw-r--r--  1 root root    179652719 2010-07-19 23:59 20100719-access.log.gz
-rw-r--r--  1 root root    1323127637 2010-07-20 23:59 20100720-access.log
-rw-r--r--  1 root root    1451718336 2010-07-21 23:59 20100721-access.log
-rw-r--r--  1 root root    1439908507 2010-07-22 23:59 20100722-access.log
-rw-r--r--  1 root root    1201661855 2010-07-23 23:59 20100723-access.log
-rw-r--r--  1 root root    898080288 2010-07-24 23:59 20100724-access.log
-rw-r--r--  1 root root    874027550 2010-07-25 23:59 20100725-access.log
-rw-r--r--  1 root root    1142218307 2010-07-26 23:59 20100726-access.log
-rw-r--r--  1 root root    1137946356 2010-07-27 23:59 20100727-access.log
-rw-r--r--  1 root root    1218598540 2010-07-28 23:59 20100728-access.log
-rw-r--r--  1 root root    799965574 2010-07-29 17:35 20100729-access.log
lrwxrwxrwx  1 root root            19 2010-07-29 12:48 access.log -> 20100729-access.log
-rw-r--r--  1 web2 client1    1183785 2010-07-29 17:31 error.log
-rw-r--r--  1 root root        25707 2010-07-29 00:30 webalizer.conf

This file I compressed them manually. And already removed files older than July 2010.

The domain of 127.0.0.1 is the default domain created manually (from ubuntu's default site) and vn.domain.com is created using ispconfig


server 2 (3.0.2.1):

ls -la /var/log/ispconfig/httpd/as69
Code:

drwxr-xr-x  2 root root  4096 2010-07-29 00:27 .
drwxr-xr-x 27 root root  4096 2010-07-23 16:51 ..
-rw-r--r--  1 root root 27627 2010-05-26 23:59 20100526-access.log
-rw-r--r--  1 root root 39069 2010-05-27 23:55 20100527-access.log
-rw-r--r--  1 root root 36137 2010-05-28 23:55 20100528-access.log
-rw-r--r--  1 root root 36178 2010-05-29 23:57 20100529-access.log
-rw-r--r--  1 root root 35886 2010-05-30 23:59 20100530-access.log
-rw-r--r--  1 root root 57538 2010-05-31 23:58 20100531-access.log
-rw-r--r--  1 root root 26358 2010-06-01 23:56 20100601-access.log
-rw-r--r--  1 root root 12807 2010-06-02 23:34 20100602-access.log
-rw-r--r--  1 root root 13814 2010-06-03 23:42 20100603-access.log
-rw-r--r--  1 root root 21864 2010-06-04 23:50 20100604-access.log
-rw-r--r--  1 root root  9855 2010-06-05 23:49 20100605-access.log
-rw-r--r--  1 root root 11110 2010-06-06 23:43 20100606-access.log
-rw-r--r--  1 root root  8399 2010-06-07 23:55 20100607-access.log
-rw-r--r--  1 root root  6039 2010-06-08 20:25 20100608-access.log
-rw-r--r--  1 root root  518 2010-06-09 07:38 20100609-access.log
-rw-r--r--  1 root root 22656 2010-06-10 17:43 20100610-access.log
-rw-r--r--  1 root root  495 2010-06-11 18:25 20100611-access.log
-rw-r--r--  1 root root  2202 2010-06-13 15:12 20100613-access.log
-rw-r--r--  1 root root  1425 2010-06-14 16:26 20100614-access.log
-rw-r--r--  1 root root 13509 2010-06-15 12:52 20100615-access.log
-rw-r--r--  1 root root  1148 2010-06-17 11:27 20100617-access.log
-rw-r--r--  1 root root  173 2010-06-18 21:55 20100618-access.log
-rw-r--r--  1 root root  1842 2010-06-20 06:33 20100620-access.log
-rw-r--r--  1 root root  1017 2010-06-21 22:19 20100621-access.log
-rw-r--r--  1 root root  258 2010-06-22 22:18 20100622-access.log
-rw-r--r--  1 root root 11764 2010-06-23 15:13 20100623-access.log
-rw-r--r--  1 root root  857 2010-06-24 23:04 20100624-access.log
-rw-r--r--  1 root root 15296 2010-06-25 21:48 20100625-access.log
-rw-r--r--  1 root root  512 2010-06-26 18:04 20100626-access.log
-rw-r--r--  1 root root  4870 2010-06-27 22:18 20100627-access.log
-rw-r--r--  1 root root  1762 2010-06-28 23:01 20100628-access.log
-rw-r--r--  1 root root  658 2010-06-29 19:58 20100629-access.log
-rw-r--r--  1 root root 13850 2010-06-30 15:26 20100630-access.log
-rw-r--r--  1 root root  335 2010-07-01 14:43 20100701-access.log
-rw-r--r--  1 root root  696 2010-07-02 23:39 20100702-access.log
-rw-r--r--  1 root root  206 2010-07-03 15:34 20100703-access.log
-rw-r--r--  1 root root  2575 2010-07-04 18:26 20100704-access.log
-rw-r--r--  1 root root 15167 2010-07-05 15:17 20100705-access.log
-rw-r--r--  1 root root  256 2010-07-06 14:00 20100706-access.log
-rw-r--r--  1 root root  528 2010-07-07 23:08 20100707-access.log
-rw-r--r--  1 root root  297 2010-07-08 23:32 20100708-access.log
-rw-r--r--  1 root root  1528 2010-07-09 23:32 20100709-access.log
-rw-r--r--  1 root root  239 2010-07-10 08:13 20100710-access.log
-rw-r--r--  1 root root  4666 2010-07-11 20:42 20100711-access.log
-rw-r--r--  1 root root  260 2010-07-12 16:47 20100712-access.log
-rw-r--r--  1 root root  333 2010-07-13 15:52 20100713-access.log
-rw-r--r--  1 root root  111 2010-07-14 23:23 20100714-access.log
-rw-r--r--  1 root root  1257 2010-07-15 22:57 20100715-access.log
-rw-r--r--  1 root root  444 2010-07-16 12:11 20100716-access.log
-rw-r--r--  1 root root  763 2010-07-17 16:33 20100717-access.log
-rw-r--r--  1 root root  2586 2010-07-18 23:34 20100718-access.log
-rw-r--r--  1 root root  2298 2010-07-19 19:39 20100719-access.log
-rw-r--r--  1 root root  128 2010-07-20 12:26 20100720-access.log
-rw-r--r--  1 root root 20287 2010-07-21 18:35 20100721-access.log
-rw-r--r--  1 root root  1942 2010-07-22 20:38 20100722-access.log
-rw-r--r--  1 root root 23792 2010-07-23 23:11 20100723-access.log
-rw-r--r--  1 root root  1925 2010-07-24 23:44 20100724-access.log
-rw-r--r--  1 root root  7010 2010-07-25 21:50 20100725-access.log
-rw-r--r--  1 root root 31775 2010-07-26 23:12 20100726-access.log
-rw-r--r--  1 root root  3219 2010-07-27 23:08 20100727-access.log
-rw-r--r--  1 root root  3348 2010-07-28 23:07 20100728-access.log
-rw-r--r--  1 root root 11323 2010-07-29 17:41 20100729-access.log
lrwxrwxrwx  1 root root    19 2010-07-29 00:27 access.log -> 20100729-access.log


ls -la /var/log/ispconfig/httpd/as69.domain.com
Code:

drwxr-xr-x  2 root root    4096 2010-05-26 11:31 .
drwxr-xr-x 27 root root    4096 2010-07-23 16:51 ..
-rw-r--r--  1 root root  257362 2010-03-22 18:57 20100322-access.log
-rw-r--r--  1 root root  24944 2010-03-23 18:00 20100323-access.log
-rw-r--r--  1 root root  41341 2010-03-24 18:19 20100324-access.log
-rw-r--r--  1 root root  187878 2010-03-25 23:53 20100325-access.log
-rw-r--r--  1 root root  46191 2010-03-26 18:24 20100326-access.log
-rw-r--r--  1 root root    256 2010-03-27 10:50 20100327-access.log
-rw-r--r--  1 root root    1386 2010-03-28 06:28 20100328-access.log
-rw-r--r--  1 root root    1592 2010-03-29 17:00 20100329-access.log
-rw-r--r--  1 root root  78299 2010-03-30 18:28 20100330-access.log
-rw-r--r--  1 root root    2930 2010-03-31 20:54 20100331-access.log
-rw-r--r--  1 root root    570 2010-04-01 16:10 20100401-access.log
-rw-r--r--  1 root root  48175 2010-04-02 18:32 20100402-access.log
-rw-r--r--  1 root root    130 2010-04-03 08:46 20100403-access.log
-rw-r--r--  1 root root    1893 2010-04-04 17:22 20100404-access.log
-rw-r--r--  1 root root  11802 2010-04-05 18:45 20100405-access.log
-rw-r--r--  1 root root    110 2010-04-07 21:34 20100407-access.log
-rw-r--r--  1 root root  14720 2010-04-08 21:38 20100408-access.log
-rw-r--r--  1 root root    129 2010-04-09 17:35 20100409-access.log
-rw-r--r--  1 root root    129 2010-04-10 21:44 20100410-access.log
-rw-r--r--  1 root root    1665 2010-04-11 20:05 20100411-access.log
-rw-r--r--  1 root root    130 2010-04-12 17:22 20100412-access.log
-rw-r--r--  1 root root 1436450 2010-04-13 23:59 20100413-access.log
-rw-r--r--  1 root root 2413651 2010-04-14 23:59 20100414-access.log
-rw-r--r--  1 root root 2378880 2010-04-15 23:59 20100415-access.log
-rw-r--r--  1 root root 2399040 2010-04-16 23:59 20100416-access.log
-rw-r--r--  1 root root 2062750 2010-04-17 23:59 20100417-access.log
-rw-r--r--  1 root root 2117572 2010-04-18 23:59 20100418-access.log
-rw-r--r--  1 root root 2481353 2010-04-19 23:59 20100419-access.log
-rw-r--r--  1 root root 2487938 2010-04-20 23:59 20100420-access.log
-rw-r--r--  1 root root 2542348 2010-04-21 23:59 20100421-access.log
-rw-r--r--  1 root root 2685128 2010-04-22 23:59 20100422-access.log
-rw-r--r--  1 root root 2393308 2010-04-23 23:59 20100423-access.log
-rw-r--r--  1 root root 2131292 2010-04-24 23:59 20100424-access.log
-rw-r--r--  1 root root 2097671 2010-04-25 23:59 20100425-access.log
-rw-r--r--  1 root root 2559121 2010-04-26 23:59 20100426-access.log
-rw-r--r--  1 root root 2536506 2010-04-27 23:59 20100427-access.log
-rw-r--r--  1 root root 2425253 2010-04-28 23:59 20100428-access.log
-rw-r--r--  1 root root 2429532 2010-04-29 23:59 20100429-access.log
-rw-r--r--  1 root root 2416206 2010-04-30 23:59 20100430-access.log
-rw-r--r--  1 root root 1832299 2010-05-01 23:59 20100501-access.log
-rw-r--r--  1 root root 1916858 2010-05-02 23:59 20100502-access.log
-rw-r--r--  1 root root 2274764 2010-05-03 23:59 20100503-access.log
-rw-r--r--  1 root root 2492260 2010-05-04 23:59 20100504-access.log
-rw-r--r--  1 root root 2217581 2010-05-05 23:59 20100505-access.log
-rw-r--r--  1 root root 2439680 2010-05-06 23:59 20100506-access.log
-rw-r--r--  1 root root 2346986 2010-05-07 23:59 20100507-access.log
-rw-r--r--  1 root root 2001514 2010-05-08 23:59 20100508-access.log
-rw-r--r--  1 root root 1994963 2010-05-09 23:59 20100509-access.log
-rw-r--r--  1 root root 2495807 2010-05-10 23:59 20100510-access.log
-rw-r--r--  1 root root 2385358 2010-05-11 23:59 20100511-access.log
-rw-r--r--  1 root root 2397078 2010-05-12 23:59 20100512-access.log
-rw-r--r--  1 root root 2428856 2010-05-13 23:59 20100513-access.log
-rw-r--r--  1 root root 2251266 2010-05-14 23:59 20100514-access.log
-rw-r--r--  1 root root 1963221 2010-05-15 23:59 20100515-access.log
-rw-r--r--  1 root root 2418050 2010-05-16 23:59 20100516-access.log
-rw-r--r--  1 root root 3335868 2010-05-17 23:59 20100517-access.log
-rw-r--r--  1 root root 3493862 2010-05-18 23:59 20100518-access.log
-rw-r--r--  1 root root 2840056 2010-05-19 23:59 20100519-access.log
-rw-r--r--  1 root root 3037030 2010-05-20 23:59 20100520-access.log
-rw-r--r--  1 root root 2642671 2010-05-21 23:59 20100521-access.log
-rw-r--r--  1 root root 2373177 2010-05-22 23:59 20100522-access.log
-rw-r--r--  1 root root 2192809 2010-05-23 23:59 20100523-access.log
-rw-r--r--  1 root root 2713228 2010-05-24 23:59 20100524-access.log
-rw-r--r--  1 root root 2667320 2010-05-25 23:59 20100525-access.log
-rw-r--r--  1 root root 1520034 2010-05-26 15:52 20100526-access.log
lrwxrwxrwx  1 root root      19 2010-05-26 11:31 access.log -> 20100526-access.log

Both as69 and as69.domain.com also is created manually. Again this is under "000-default" under apache configuration.

Other sites on server2 is no problem. The access log files are compressed and old file deleted.

Thanks.

till 29th July 2010 11:53

The log compression are only for sites created in ispconfig. Th examples that you posted above are not from sites that had been created in ispconfig, so the logs dont get compressed.

ISPConfig can only compress logs for sites that it knows off. If you want to compress logs for non ispconfig websites, you might have to write your own scripts for that.


All times are GMT +2. The time now is 13:37.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.