getting mailgraph spam and virus

[rayit]

I installed mailgraph but do not get spam and virus report.

I think I should enable amavis and spamassassin to log into /var/log/mail.log

I am using ISPconfig, so I should look I think somewhere in
/home/admispconfig/ispconfig/tools#

but what do I need to change to log virus and spam?

many thanks :o

RayIT

[TheRudy]

Heh, i'm about to do the same thing and it seems that you have to install amavisd and configurate it for spam and virus to work in the monitoring.. No idea yet how to do it..

Anyone done this before with Perfect Setup configuration? If so, can someone tell how..

http://www.howtoforge.com/virtual_postfix_mysql_quota_courier_p4

I'm betting that that is not enought for it to work.. Installation and configuring spamassasin..

[falko]

Have a look here:
http://www.ijs.si/software/amavisd/
http://gentoo-wiki.com/HOWTO_Spam_Filtering_with_Gentoo,_Postfix,_Amavis

[TheRudy]

great falko! Will look into it and report back how did it go..

[TheRudy]

well it didn't work first time :)
removed everything for now...

installed amavis, configured postfix, configuring amavis: got stuck :D

questions!
clamav? comes with ispconfig, yes? Can i use that one or do i have to install it? I keep getting that clamav can't be found like errors..
Jul 24 13:28:46 mercury amavis[21354]: (21354-01) Clam Antivirus-clamd av-scanner FAILED: Too many retries to talk to /var/ru$
Jul 24 13:28:46 mercury amavis[21354]: (21354-01) WARN: all primary virus scanners failed, considering backups
$Clam Antivirus-clamd av-scanner FAILED: Too many retries to talk to /var/run/clamav/clamd.ctl (Can't connect to UNIX socket $
Jul 24 13:28:46 mercury amavis[21354]: (21354-01) PRESERVING EVIDENCE in /var/lib/amavis/amavis-20060724T132839-21354

How do i use the one that comes with ispconfig?

[rayit]

strange normally I always saw virus warnings and spam warnings in my syslog, but this seems to have changed in the new ispconfig???

I tried following....
:confused: changing the file
/home/admispconfig/ispconfig/tools/clamav/etc/freshclam.conf
to log in to a file
I changed
# Path to the log file (make sure it has proper permissions)
# Default: disabled
UpdateLogFile /var/log/freshclam

made a file /var/log/freshclam with permissions of admispconfig


I start
mailgraph once with --only-mail-rrd -l /var/log/mail
and once with --only-virus-rrd -l /var/log/amavis.log

so:
/usr/bin/perl -w /usr/sbin/mailgraph.pl --only-mail-rrd -l /var/log/mail.log -d --daemon_rrd=/var/lib/mailgraph --ignore-localhost

/usr/bin/perl -w /usr/sbin/mailgraph.pl --only-virus-rrd -l /var/log/freshclam -d --daemon_rrd=/var/lib/mailgraph --ignore-localhost

I think this is a start??
http://www.rayit.com/cgi-bin/mailgraph.cgi

But is seems not to work????

Any suggestions, also spam does not yet work..
:confused:

[falko]

clamav? comes with ispconfig, yes? Can i use that one
Yes, but you need to adjust the paths to ClamAV in your amavisd.conf.

[falko]

strange normally I always saw virus warnings and spam warnings in my syslog, but this seems to have changed in the new ispconfig???

This has nothing to do with ISPConfig. You need to install amavisd.

[TheRudy]

Yes, but you need to adjust the paths to ClamAV in your amavisd.conf.
Right, i missed that part of the question.. Where is ClamAV? I can find config file but not the rest.. Probably looking at the wrong place AGAIN..

btw, what's the command to update 'database' of locate function? If you install something new, locate can't find it so i need to update the 'database'..

[falko]

Right, i missed that part of the question.. Where is ClamAV? I can find config file but not the rest.. Probably looking at the wrong place AGAIN..It's in /home/admispconfig/ispconfig/tools/clamav.

btw, what's the command to update 'database' of locate function? If you install something new, locate can't find it so i need to update the 'database'..The command is updatedb

[TheRudy]

right thanks!

[TheRudy]

ok, things done so far:

- installed amavis

- i made tmp, quarantine and db folder in /var/lib/amavis, chmoded 750 and chown amavis:amavis

- followed this http://gentoo-wiki.com/HOWTO_Spam_Filtering_with_Gentoo,_Postfix,_Amavis for postfix config..

- postfix responds over telnet..

- edited amavis.conf and changed path (CONTSCAN) for ClamAV to /home/adm42go/42go/temp/clamd, same path as in ClamAV config file..

- added amavis to admispconfig group
tried then:
- added admispconfig to amavis group

- went to mail client, sended email (it was send without any error in mail client!)

- checked logs and there we go:
Jul 27 09:22:19 mercury amavis[13776]: starting. amavisd-new at mercury.domain.tpl amavisd-new-20030616-p10, Unicode aware,$
Jul 27 09:22:19 mercury amavis[13776]: Perl version 5.008004
Jul 27 09:22:19 mercury amavis[13776]: Module Amavis::Conf 1.15
Jul 27 09:22:19 mercury amavis[13776]: Module Archive::Tar 1.23
Jul 27 09:22:19 mercury amavis[13776]: Module Archive::Zip 1.14
Jul 27 09:22:19 mercury amavis[13776]: Module Compress::Zlib 1.34
Jul 27 09:22:19 mercury amavis[13776]: Module Convert::TNEF 0.17
Jul 27 09:22:19 mercury amavis[13776]: Module Convert::UUlib 1.051
Jul 27 09:22:19 mercury amavis[13776]: Module MIME::Entity 5.417
Jul 27 09:22:19 mercury amavis[13776]: Module MIME::Parser 5.417
Jul 27 09:22:19 mercury amavis[13776]: Module MIME::Tools 5.417
Jul 27 09:22:19 mercury amavis[13776]: Module Mail::Header 1.62
Jul 27 09:22:19 mercury amavis[13776]: Module Mail::Internet 1.62
Jul 27 09:22:19 mercury amavis[13776]: Module Net::Cmd 2.26
Jul 27 09:22:19 mercury amavis[13776]: Module Net::SMTP 2.29
Jul 27 09:22:19 mercury amavis[13776]: Module Net::Server 0.87
Jul 27 09:22:19 mercury amavis[13776]: Module Time::HiRes 1.59
Jul 27 09:22:19 mercury amavis[13776]: Module Unix::Syslog 0.100
Jul 27 09:22:19 mercury amavis[13777]: Found $file at /usr/bin/file
Jul 27 09:22:19 mercury amavis[13777]: No $arc, not using it
Jul 27 09:22:19 mercury amavis[13777]: Found $gzip at /bin/gzip
Jul 27 09:22:19 mercury amavis[13777]: Found $bzip2 at /usr/bin/bzip2
Jul 27 09:22:19 mercury amavis[13777]: No $lzop, not using it
Jul 27 09:22:19 mercury amavis[13777]: No $lha, not using it
Jul 27 09:22:19 mercury amavis[13777]: Found $unarj at /usr/bin/arj
Jul 27 09:22:19 mercury amavis[13777]: Found $uncompress at /bin/uncompress
Jul 27 09:22:19 mercury amavis[13777]: No $unfreeze, not using it
Jul 27 09:22:19 mercury amavis[13777]: No $unrar, not using it
Jul 27 09:22:19 mercury amavis[13777]: Found $zoo at /usr/bin/zoo
Jul 27 09:22:19 mercury amavis[13777]: Found $cpio at /bin/cpio
Jul 27 09:22:19 mercury amavis[13777]: Using internal av scanner code for (primary) Clam Antivirus-clamd
Jul 27 09:22:25 mercury postfix/postfix-script: stopping the Postfix mail system
Jul 27 09:22:25 mercury postfix/master[12716]: terminating on signal 15
Jul 27 09:22:25 mercury postfix/postfix-script: starting the Postfix mail system
Jul 27 09:22:25 mercury postfix/master[13881]: daemon started -- version 2.1.5
Jul 27 09:22:25 mercury postfix/qmgr[13884]: 1FDE87AC090: from=<email@domain.tpl>, size=637, nrcpt=1 (queue active)
Jul 27 09:22:25 mercury postfix/qmgr[13884]: BA8B57AC0A8: from=<email@domain.tpl>, size=638, nrcpt=1 (queue active)
Jul 27 09:22:25 mercury postfix/qmgr[13884]: warning: connect to transport amavis: Connection refused
Jul 27 09:22:25 mercury postfix/qmgr[13884]: 57A177AC0AA: from=<email@domain.tpl>, size=640, nrcpt=1 (queue active)
Jul 27 09:22:26 mercury amavis[13778]: (13778-01) Clam Antivirus-clamd: Can't connect to UNIX socket /home/adm42go/42go/temp/$
Jul 27 09:22:26 mercury amavis[13779]: (13779-01) Clam Antivirus-clamd: Can't connect to UNIX socket /home/adm42go/42go/temp/$
Jul 27 09:22:32 mercury amavis[13778]: (13778-01) Clam Antivirus-clamd av-scanner FAILED: Too many retries to talk to /home/a$
Jul 27 09:22:32 mercury amavis[13778]: (13778-01) WARN: all primary virus scanners failed, considering backups
Jul 27 09:22:32 mercury amavis[13778]: (13778-01) TROUBLE in check_mail: virus_scan FAILED: ALL VIRUS SCANNERS FAILED: Clam A$
Jul 27 09:22:32 mercury amavis[13778]: (13778-01) PRESERVING EVIDENCE in /var/lib/amavis/amavis-20060727T092225-13778
Jul 27 09:22:32 mercury amavis[13779]: (13779-01) Clam Antivirus-clamd av-scanner FAILED: Too many retries to talk to /home/a$
Jul 27 09:22:32 mercury amavis[13779]: (13779-01) WARN: all primary virus scanners failed, considering backups
Jul 27 09:22:32 mercury amavis[13779]: (13779-01) TROUBLE in check_mail: virus_scan FAILED: ALL VIRUS SCANNERS FAILED: Clam A$
Jul 27 09:22:32 mercury amavis[13779]: (13779-01) PRESERVING EVIDENCE in /var/lib/amavis/amavis-20060727T092225-13779
Jul 27 09:22:32 mercury postfix/smtp[13885]: BA8B57AC0A8: to=<user@mercury.domain.tpl>, orig_to=<email@domain.$

Jul 27 09:22:33 mercury postfix/smtpd[13896]: connect from CLIENT_IP[CLIENT_IP]
Jul 27 09:22:33 mercury postfix/smtpd[13896]: 8CA397AC0AC: client=CLIENT_IP[CLIENT_IP], sasl_method=PLAIN,$
Jul 27 09:22:33 mercury postfix/cleanup[13898]: 8CA397AC0AC: message-id=<44C86975.1000104@domain.tpl>
Jul 27 09:22:33 mercury postfix/qmgr[13884]: 8CA397AC0AC: from=<email@domain.tpl>, size=657, nrcpt=1 (queue active)
Jul 27 09:22:33 mercury postfix/smtpd[13896]: disconnect from CLIENT_IP[CLIENT_IP]
Jul 27 09:22:33 mercury amavis[13778]: (13778-02) Clam Antivirus-clamd: Can't connect to UNIX socket /home/adm42go/42go/temp/$
Jul 27 09:22:34 mercury amavis[13779]: (13779-02) Clam Antivirus-clamd: Can't connect to UNIX socket /home/adm42go/42go/temp/$
Jul 27 09:22:39 mercury amavis[13778]: (13778-02) Clam Antivirus-clamd av-scanner FAILED: Too many retries to talk to /home/a$
Jul 27 09:22:39 mercury amavis[13778]: (13778-02) WARN: all primary virus scanners failed, considering backups
Jul 27 09:22:39 mercury amavis[13778]: (13778-02) TROUBLE in check_mail: virus_scan FAILED: ALL VIRUS SCANNERS FAILED: Clam A$
Jul 27 09:22:39 mercury amavis[13778]: (13778-02) PRESERVING EVIDENCE in /var/lib/amavis/amavis-20060727T092232-13778
Jul 27 09:22:39 mercury postfix/smtp[13885]: 57A177AC0AA: to=<user@mercury.domain.tpl>, orig_to=<user@mercury.domain.$
Jul 27 09:22:40 mercury amavis[13779]: (13779-02) Clam Antivirus-clamd av-scanner FAILED: Too many retries to talk to /home/a$
Jul 27 09:22:40 mercury amavis[13779]: (13779-02) WARN: all primary virus scanners failed, considering backups
Jul 27 09:22:40 mercury amavis[13779]: (13779-02) TROUBLE in check_mail: virus_scan FAILED: ALL VIRUS SCANNERS FAILED: Clam A$
Jul 27 09:22:40 mercury amavis[13779]: (13779-02) PRESERVING EVIDENCE in /var/lib/amavis/amavis-20060727T092233-13779
Jul 27 09:22:40 mercury postfix/smtp[13886]: 8CA397AC0AC: to=<user@mercury.domain.tpl>, orig_to=<email@domain.$
Jul 27 09:23:25 mercury postfix/qmgr[13884]: warning: connect to transport amavis: Connection refused



What am i missing? Probably some group problem or path for clamav..

PS: edited my ip and email with domain name from logs!

[TheRudy]

Actually that path:
Clam Antivirus-clamd: Can't connect to UNIX socket /home/adm42go/42go/temp/$
doesn't make any sense.. Why that path? It doesn't exist and its still in clamav.conf file..
Now i'm confused..

[TheRudy]

Right, sorry, just saw logs are not fully c/p'd..

Problem is:
Jul 27 10:17:09 mercury amavis[17995]: (17995-01) Clam Antivirus-clamd: Can't connect to UNIX socket /home/admispconfig/ispconfig/tools/clamav/bin: Permission denied, retrying (2)
Amavis should be in the same group as clamav, that is admispconfig. So if that's ok, that's solved.. The problem will now be in the path, what path to use.. tried everything..

[falko]

ISPConfig doesn't run clamd, that's why there's no socket. Use the backup virus scanner from amavisd.conf instead (clamscan or so), it's called whenever an email arrives.

[TheRudy]

Oh :( Sorry, didn't really go over the amavis config file...

Anyway.. i commented out the primary clamav and set path for secondary to point to the correct folder.. Email was sent from me to gmail.. Tried sending email to me (local) it didn't work..

delivery temporarily suspended: transport is unavailable

so what i did is this, edited main.cf postfix and changed
local_transport = no local mail delivery
local_recipient_maps =


to

#local_transport = no local mail delivery
local_recipient_maps = $alias_maps


And now its working everything!

Question! Is that the correct way? I mean the change in main.cf.. Just a precaution question if you will..

Transport file only contains this:
domain.tpl smtp:[192.168.168.100]

[TheRudy]

Someone woudl think that problems are over..
Fine, emails and stuff work.. haven't tested spam and test virus stuff but i'll do that later when i have some extra time to play with it and settings..

right, so i installed mailgraph and first thing first, i get 500 error.. wth? right so i changed owner and group from root - root to web3_internet - web3..

and voila, website is shown BUT no images at all, instead i get alt text!

i have suexec turned on and in suexec.log i have found this:
[2006-07-28 17:14:11]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:14:11]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:14:11]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:14:59]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:14:59]: target uid/gid (10007/10003) mismatch with directory (10007/10003) or program (10007/0)
[2006-07-28 17:16:53]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:16:53]: target uid/gid (10007/10003) mismatch with directory (10007/10003) or program (0/0)
[2006-07-28 17:17:45]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:45]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-07-28 17:17:46]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi

that part about mismatch with directory is me changing owned and group back to root..

/etc/group
web3:x:10003:admispconfig,web3_internet

/etc/passwd
web3_internet:x:10007:10003:internet email:/var/www/web3:/bin/false

all files and folders have correct ownership and groups.. Don't get it..


just turned OFF suexec and its working.. turned it back on, images go away and log file is filled with above errors.. :)

[falko]

just turned OFF suexec and its working.. turned it back on, images go away and log file is filled with above errors.. :)
What's the vhost configuration of that vhost (with suExec turned on)?

[TheRudy]

<VirtualHost 192.168.168.100:80>
SuexecUserGroup web3_internet web3
ServerName www.domain.tpl:80
ServerAdmin webmaster@domain.tpl
DocumentRoot /var/www/web3/web
ServerAlias domain.tpl
DirectoryIndex index.html index.htm index.php index.php5 index.php4 index.php3 index.shtml index.cgi index.pl index.jsp
ScriptAlias /cgi-bin/ /var/www/web3/cgi-bin/
AddHandler cgi-script .cgi
AddHandler cgi-script .pl
ErrorLog /var/www/web3/log/error.log
AddType application/x-httpd-php .php .php3 .php4 .php5
php_admin_flag safe_mode Off
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
Alias /error/ "/var/www/web3/web/error/"
ErrorDocument 400 /error/invalidSyntax.html
ErrorDocument 401 /error/authorizationRequired.html
ErrorDocument 403 /error/forbidden.html
ErrorDocument 404 /error/fileNotFound.html
ErrorDocument 405 /error/methodNotAllowed.html
ErrorDocument 500 /error/internalServerError.html
ErrorDocument 503 /error/overloaded.html
AliasMatch ^/~([^/]+)(/(.*))? /var/www/web3/user/$1/web/$3
AliasMatch ^/users/([^/]+)(/(.*))? /var/www/web3/user/$1/web/$3
</VirtualHost>


suexec users are in so.. don't see anything wrong..

[falko]

And what's the output of ls -la /var/www/web3/cgi-bin/?

[TheRudy]

drwxr-xr-x 2 web3_internet web3 4096 2006-07-28 16:48 .
drwxr-xr-x 9 web3_internet web3 4096 2006-07-28 17:26 ..
-rw-r--r-- 1 web3_internet web3 0 2006-07-28 16:48 .csc
-rwxr-xr-x 1 web3_internet web3 7009 2004-11-27 19:37 mailgraph.cgi
-r-------- 1 root root 0 2006-06-18 11:09 .no_delete

Before mailgraph.cgi was set as root - root with root, it didn't even show page.. I changed mailgrapgh.cgi manually to current permission..

Any more ideas?

[falko]

What's in suexec.log now? What's the output of grep web3_internet /etc/passwd and grep web3 /etc/group?

[TheRudy]

What's in suexec.log now?
The same as already posted..
[2006-08-01 09:53:07]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 09:53:07]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 09:53:07]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 09:53:08]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:12]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi
[2006-08-01 11:11:13]: uid: (10007/web3_internet) gid: (10003/10003) cmd: mailgraph.cgi


What's the output of grep web3_internet /etc/passwd
Still the same..
web3_internet:x:10007:10003:internet email:/var/www/web3:/bin/false


and grep web3 /etc/group?
And again the same..
users:x:100:web3_xxx,web3_xxx,web3_alex,web3_xxx,w eb3_xxx,web3_xxx,web3_xxx
web3:x:10003:admispconfig,web3_internet

xxx replaced by me.. Could it be the problem that web3_internet is not on the users list? I mean, all users are listed there except for web3_internet user who is admin and is under web3.. eh..

[falko]

What's the output of grep 10007 /etc/passwd? I think that maybe more than one user has this ID on your system (otherwise suexec.log should mention web3_internet instead of 10007).

[TheRudy]

web3_internet:x:10007:10003:internet email:/var/www/web3:/bin/false

[falko]

Hm... And the output of grep 10003 /etc/group?

[TheRudy]

web3:x:10003:admispconfig,web3_internet

I've done so much googling for this, trying a few things but the only way to get it working is by setting suexec to off.. I see that you also are a bit lost in the dark here.. its weird right?

[falko]

It's strange, indeed...

[Ovidiu]

I tried this too, long ago see this thread: http://www.howtoforge.com/forums/showthread.php?t=1598&highlight=mailgraph

my problem was that mailgraph was not picking up any viruses or spams although I was looking at the corect files and the expressions to find spams and viruses was right.

did anyone at all get it running?

[falko]

did anyone at all get it running?
Yes, we use it in this product: http://www.projektfarm.de/en_spam_filter.html

[Ovidiu]

Yes, we use it in this product: http://www.projektfarm.de/en_spam_filter.html

very funny :D

do you think you could share the regex (is that the right word?) expression which catches viruses and spams?

[falko]

It's a standard amavisd.conf. Nothing special...