HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials - The Perfect SpamSnake - Ubuntu Jeos 10.10 Maverick Meerkat

HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials (http://www.howtoforge.com/forums/index.php)

- HOWTO-Related Questions (http://www.howtoforge.com/forums/forumdisplay.php?f=2)

- - The Perfect SpamSnake - Ubuntu Jeos 10.10 Maverick Meerkat - transportmaps (http://www.howtoforge.com/forums/showthread.php?t=50346)

The Perfect SpamSnake - Ubuntu Jeos 10.10 Maverick Meerkat - transportmaps

Please help again. this setup is proving to be more challenging as its not too clear following the steps.

Heres my setup . I want the spam snake to scan and relay to an exchange server . Emails however are getting queued on the local server ( spam snake server) . I cant see the emails listed in Baruwa.

this is my config. I have changed the transports to use hash rather than mysql as in the config as i couldnt figure out how to put the entries in mysql

relay_recipients
#############

test@test.com OK
test@example.com OK

transport
#########

example.com smtp:[192.168.0.222]
test.com smtp:[192.168.0.222]

virtual
########
root root@example.com

main.cf
########

alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = $mydomain
mydestination = $myhostname, localhost.$mydomain, localhost
mynetworks = 127.0.0.0/8, 192.168.0.0/24
#mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
html_directory = /usr/share/doc/postfix/html
message_size_limit = 10485760
##local_transport = error:No local mail delivery
local_recipient_maps =
##relay_domains = mysql:/etc/postfix/mysql-relay_domains.cf
relay_domains = hash:/etc/postfix/transport
#relay_recipient_maps = mysql:/etc/postfix/mysql-relay_recipients.cf
relay_recipient_maps = hash:/etc/postfix/relay_recipients
##transport_maps = mysql:/etc/postfix/mysql-transports.cf
transport_maps = hash:/etc/postfix/transport
virtual_alias_maps = hash:/etc/postfix/virtual
disable_vrfy_command = yes
strict_rfc821_envelopes = no
smtpd_delay_reject = yes
smtpd_recipient_limit = 100
smtpd_helo_required = yes
smtpd_client_restrictions = permit_sasl_authenticated, permit_mynetworks, permit
smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, permit
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_sender, reject_unknown_sender_domain, permit
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_recipient_domain, reject_unauth_destination, whitelist_policy$
smtpd_data_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining
smtpd_restriction_classes = spf_policy, rbl_policy, grey_policy, whitelist_policy
spf_policy = check_policy_service unix:private/policy
rbl_policy = reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net
grey_policy = check_policy_service unix:private/greyfix
whitelist_policy = check_sender_access mysql:/etc/postfix/mysql-global_whitelist.cf
header_checks = regexp:/etc/postfix/header_checks
masquerade_domains = $mydomain

master.cf
########

smtp inet n - - - - smtpd
-o content_filter=dfilt:
#submission inet n - - - - smtpd
# -o smtpd_tls_security_level=encrypt
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticate d,reject
# -o milter_macro_daemon_name=ORIGINATING
#smtps inet n - - - - smtpd
# -o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticate d,reject
# -o milter_macro_daemon_name=ORIGINATING
#628 inet n - - - - qmqpd
pickup fifo n - - 60 1 pickup
-o content_filter=
-o receive_override_options=no_header_body_checks
cleanup unix n - - - 0 cleanup
qmgr fifo n - n 300 1 qmgr
#qmgr fifo n - - 300 1 oqmgr
tlsmgr unix - - - 1000? 1 tlsmgr
rewrite unix - - - - - trivial-rewrite
bounce unix - - - - 0 bounce
defer unix - - - - 0 bounce
trace unix - - - - 0 bounce
verify unix - - - - 1 verify
flush unix n - - 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - - - - smtp
# When relaying mail as backup MX, disable fallback_relay to avoid MX loops

Have you finished the entire guide or finished the greylisting section?

What messages are you getting in /var/log/mail.log?

For you to use hash, you would need to do the following:

vi relay_domains
domainA OK
domainB OK

postmap /etc/postfix/relay_domains

vi relay_recipients
@domainA OK
@domainB OK

postmap /etc/postfix/relay_recipients

vi transports
domainA smtp:[DNS or IP]
domainB smtp:[DNS or IP]

postmap /etc/postfix/transports

vi /etc/postfix/main.cf
relay_domains = hash:/etc/postfix/relay_domains
relay_recipient_maps = hash:/etc/postfix/relay_recipients
transport_maps = hash:/etc/postfix/transports

If you haven't done the FuzzyOCR or Greyfix sections, remove:

smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, look_ahead, whitelist_policy, grey_policy, rbl_policy, spf_policy, permit

And if you haven't applied look_ahead, remove that as well.

Hi thanks for the early reply.i followed the entire guide and as you can see from post i have added and ok the domains in my transports maps etc. i am using the script to pull the reciepients from ad as per the guide which works fine. Fuzzy is implemented as per the guide. I have taken out the sections in red as you stated just to test but same result

root@mail:/etc/postfix# mailq
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
D9F22808C1! 12005 Fri Dec 3 15:57:39 test@jung.com
test@example.com

and tail -f /var/log/mail.log

3:58 mail MailScanner[7482]: Config: calling custom init function BaruwaBlacklist
Dec 3 16:03:58 mail MailScanner[7482]: Config: calling custom init function BaruwaSQL
Dec 3 16:03:58 mail MailScanner[7482]: Config: calling custom init function BaruwaHighScore
Dec 3 16:03:58 mail MailScanner[7482]: Config: calling custom init function BaruwaWhitelist
Dec 3 16:03:59 mail MailScanner[7482]: Using SpamAssassin results cache
Dec 3 16:03:59 mail MailScanner[7482]: Connected to SpamAssassin cache database
Dec 3 16:03:59 mail MailScanner[7482]: Enabling SpamAssassin auto-whitelist functionality...
Dec 3 16:04:02 mail postfix/pickup[7158]: 32CD08055F: uid=0 from=<root>
Dec 3 16:04:02 mail postfix/cleanup[7166]: 32CD08055F: message-id=<20101203160402.32CD08055F@mail.example.com>
Dec 3 16:04:02 mail postfix/qmgr[7159]: 32CD08055F: from=<root@example.com>, size=766, nrcpt=1 (queue active)
Dec 3 16:04:02 mail postfix/smtp[7455]: 32CD08055F: to=<root@example.com>, orig_to=<root>, relay=192.168.0.212[192.168.0.212]:25, delay=0.32, delays=0.05/0/0/0.26, dsn=2.6.0, status=sent (250 2.6.0 <20101203160402.32CD08055F@mail.example.com> [InternalId=280] Queued mail for delivery)
Dec 3 16:04:02 mail postfix/qmgr[7159]: 32CD08055F: removed
Dec 3 16:04:06 mail MailScanner[7482]: Connected to Processing Attempts Database
Dec 3 16:04:06 mail MailScanner[7482]: Found 4 messages in the Processing Attempts Database
Dec 3 16:04:06 mail MailScanner[7482]: Using locktype = flock
Dec 3 16:05:01 mail postfix/pickup[7158]: 90DC2809D1: uid=0 from=<root>
Dec 3 16:05:01 mail postfix/cleanup[7166]: 90DC2809D1: message-id=<20101203160501.90DC2809D1@mail.example.com>
Dec 3 16:05:01 mail postfix/qmgr[7159]: 90DC2809D1: from=<root@example.com>, size=766, nrcpt=1 (queue active)
Dec 3 16:05:01 mail postfix/smtp[7455]: 90DC2809D1: to=<root@example.com>, orig_to=<root>, relay=192.168.0.212[192.168.0.212]:25, delay=0.35, delays=0.04/0/0.01/0.3, dsn=2.6.0, status=sent (250 2.6.0 <20101203160501.90DC2809D1@mail.example.com> [InternalId=281] Queued mail for delivery)
Dec 3 16:05:01 mail postfix/qmgr[7159]: 90DC2809D1: removed
Dec 3 16:05:42 mail postfix/scache[7456]: statistics: start interval Dec 3 16:03:01
Dec 3 16:05:42 mail postfix/scache[7456]: statistics: domain lookup hits=0 miss=1 success=0%
Dec 3 16:05:42 mail postfix/scache[7456]: statistics: address lookup hits=0 miss=1 success=0%
Dec 3 16:05:42 mail postfix/scache[7456]: statistics: max simultaneous domains=1 addresses=1 connection=1

Do this:

cd /opt/MailScanner/lib/MailScanner/CustomFunctions/
vi BaruwaUserSettings.pm and add:

Quote:

#
# Baruwa - Web 2.0 MailScanner front-end.
# Copyright (C) 2010 Andrew Colin Kissa <andrew@topdog.za.net>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
# vim: ai ts=4 sts=4 et sw=4

package MailScanner::CustomConfig;

use strict;
use DBI;

my ($db_name) = 'baruwa';
my ($db_host) = 'localhost';
my ($db_user) = 'baruwa';
my ($db_pass) = '';

my ($refresh_time) = 60;
my ( $ltime, $htime );
my ( %Lowscores, %Highscores, %ScanList );

my $high_query = <<END;
SELECT DISTINCT(email),sa_high_score AS score,is_superuser FROM
auth_user, profiles WHERE auth_user.id = profiles.user_id AND
sa_high_score > 0 UNION SELECT address AS email,sa_high_score,0
FROM user_addresses AS a, profiles AS b WHERE enabled=1 AND
sa_high_score > 0 AND a.user_id = b.user_id
END

my $low_query = <<END;
SELECT DISTINCT(email),sa_low_score,is_superuser AS score FROM
auth_user, profiles WHERE auth_user.id = profiles.user_id AND
sa_low_score > 0 UNION SELECT address AS email,sa_low_score,0
FROM user_addresses AS a, profiles AS b WHERE enabled=1 AND
sa_low_score > 0 AND a.user_id = b.user_id
END

my $scan_query = <<END;
SELECT DISTINCT(email),scan_mail,is_superuser FROM
auth_user, profiles WHERE auth_user.id = profiles.user_id
UNION SELECT address AS email,scan_mail,0 FROM
user_addresses AS a, profiles AS b WHERE enabled=1
AND a.user_id = b.user_id
END

sub PopulateScores {
my ( $type, $list ) = @_;
my ( $conn, $sth, $email, $spamscore, $query, $count, $isadmin );

$conn = DBI->connect( "DBI:mysql:database=$db_name;host=$db_host",
$db_user, $db_pass, { PrintError => 0, AutoCommit => 1 } );
if ( !$conn ) {
MailScanner::Log::WarnLog( "Baruwa Settings conn init failue: %s",
$DBI::errstr );
}
if ( $type == 1 ) {
$query = $low_query;
}
else {
$query = $high_query;
}

$sth = $conn->prepare($query);
$sth->execute();
$sth->bind_columns( undef, \$email, \$spamscore, \$isadmin );
$count = 0;
while ( $sth->fetch() ) {
$list->{ lc($email) } = $spamscore;
if ($isadmin) {
$list->{'admin'} = $spamscore;
}
$count++;
}
$sth->finish();
$conn->disconnect();
return $count;
}

sub CheckScores {
my ( $type, $message, $scores ) = @_;

return 0 unless $message;

my ( $todomain, @to, $to, @todomain );

@to = @{ $message->{to} };
@todomain = @{ $message->{todomain} };
$to = $to[0];
$todomain = $todomain[0];

return $scores->{$to} if $scores->{$to};
return $scores->{$todomain} if $scores->{$todomain};
return $scores->{"admin"} if $scores->{"admin"};

if ( $type == 1 ) {
return 5;
}
else {
return 10;
}
}

sub InitBaruwalowScore {
MailScanner::Log::InfoLog("Baruwa - Populating spam score settings");
my $total = PopulateScores( 1, \%Lowscores );
if ($total) {
MailScanner::Log::InfoLog( "Read %d spam score settings", $total );
}
else {
MailScanner::Log::InfoLog(
"no spam score settings found using defaults");
}
$ltime = time();
}

sub InitBaruwaHighScore {
MailScanner::Log::InfoLog("Baruwa - Populating high spam score settings");
my $total = PopulateScores( 2, \%Highscores );
if ($total) {
MailScanner::Log::InfoLog( "Read %d high spam score settings", $total );
}
else {
MailScanner::Log::InfoLog(
"no high spam score settings found using defaults");
}
$htime = time();
}

sub EndBaruwalowScore {
MailScanner::Log::InfoLog("Shutting down Baruwa spam score settings");
}

sub EndBaruwahighScore {
MailScanner::Log::InfoLog("Shutting down Baruwa high spam score settings");
}

sub BaruwaLowScore {
if ( ( time() - $ltime ) >= ( $refresh_time * 60 ) ) {
MailScanner::Log::InfoLog(
"Baruwa - spam score setting refresh time reached");
InitBaruwalowScore();
}
my ($message) = @_;
return CheckScores( 1, $message, \%Lowscores );
}

sub BaruwaHighScore {
if ( ( time() - $htime ) >= ( $refresh_time * 60 ) ) {
MailScanner::Log::InfoLog(
"Baruwa - high spam score setting refresh time reached");
InitBaruwaHighScore();
}
my ($message) = @_;
return CheckScores( 2, $message, \%Highscores );
}

sub PopulateScanList {
my $list = @_;

my ( $conn, $sth, $count, $shouldscan, $isadmin, $email );

$conn = DBI->connect( "DBI:mysql:database=$db_name;host=$db_host",
$db_user, $db_pass, { PrintError => 0, AutoCommit => 1 } );
if ( !$conn ) {
MailScanner::Log::WarnLog( "Baruwa Scan Settings conn init failue: %s",
$DBI::errstr );
}

$sth = $conn->prepare($scan_query);
$sth->execute();
$sth->bind_columns( undef, \$email, \$shouldscan, \$isadmin );
$count = 0;
while ( $sth->fetch() ) {
$list->{ lc($email) } = $shouldscan;
if ($isadmin) {
$list->{'admin'} = $shouldscan;
}
$count++;
}
$sth->finish();
$conn->disconnect();
return $count;
}

sub CheckShouldScan {
my ( $message, $list ) = @_;

return 0 unless $message;

my ( $todomain, @to, $to, @todomain );

@to = @{ $message->{to} };
@todomain = @{ $message->{todomain} };
$to = $to[0];
$todomain = $todomain[0];

return $list->{$to} if $list->{$to};
return $list->{$todomain} if $list->{$todomain};
return $list->{"admin"} if $list->{"admin"};
return 1;
}

sub InitBaruwaShouldScan {
MailScanner::Log::InfoLog("Starting Baruwa scanning settings");
my $total = PopulateScanList( \%ScanList );
MailScanner::Log::InfoLog( "Read %d settings", $total );
}

sub EndBaruwaShouldScan {
MailScanner::Log::InfoLog("Shutting down Baruwa scanning settings");
}

sub BaruwaShouldScan {
my $message = @_;
return CheckShouldScan( $message, \%ScanList );
}

1;

Change the Username and Password to your Baruwa DB settings.

vi /opt/MailScanner/etc/MailScanner.conf and make sure the following are set:
Always Looked Up Last = &BaruwaSQL
Is Definitely Not Spam = &BaruwaWhitelist
Is Definitely Spam = &BaruwaBlacklist
Required SpamAssassin Score = &BaruwaLowScore
High SpamAssassin Score = &BaruwaHighScore

Restart MailScanner
/etc/init.d/mailscanner restart

Let me know what happens

done still no logs in baruwa. On the other hand after a length period the mail did finally clear from the queue before i mad the change you asked me to .

the sender got a message

Our virus detector failed to completely analyse a message you sent:-
To: test@example.com
Subject: tester mike
Date: Fri Dec 3 16:15:44 2010
Any parts of the message that could not be analysed will not have been delivered.

If you are using Microsoft Outlook, we strongly recommend you change your outgoing message format from "Rich Text" to "HTML" or "Plain Text".

1) Click on the "Tools" menu and choose "Options..."
2) Go to the "Mail Format" tab
3) For message format, select "HTML" or "Plain text"
4) Click OK

The virus detector said this about the message:
Report: Report: MailScanner: Message attempted to kill MailScanner

--
MailScanner
Email Virus Scanner

The sende was sending message in html

Also in Baruwa system status says MTA 0 process

Scanners 6 processes Processed 0
MTA 0 processes Clean None
Load 0.28 0.53 0.37 High scoring spam None
AV 1 processes Low scoring spam None
Uptime3:22, 2Virii None
System restartBlocked files

I suspect the problem is caused by something not going right with the MailScanner setup script.

Please do vi /opt/MailScanner/etc/MailScanner.conf and verify everything is set according to my mailscanner.sh script. These are the settings you should be looking for:

Quote:

%org-name% = orgname
%org-long-name% = longorgname
%web-site% = www.domain.tld
Run As User = postfix
Run As Group = www-data
Incoming Work Group = clamav
Incoming Work Permissions = 0640
Incoming Queue Dir = /var/spool/postfix/hold
Outgoing Queue Dir = /var/spool/postfix/incoming
MTA = postfix
Quarantine User = root
Quarantine Group = www-data
Quarantine Permissions = 0660
Quarantine Whole Message = yes
Virus Scanners = clamd"
Monitors for ClamAV Updates = /var/lib/clamav/*.cld /var/lib/clamav/*.cvd
Clamd Socket = /var/run/clamav/clamd.ctl
Clamd Lock File = /var/run/clamav/clamd.pid
Spam Subject Text = ***SPAM***
Spam Actions = deliver store
Spam Actions High Scoring Spam Actions = store delete
Non Spam Actions = deliver store
SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin

If those are correctly set, then the script works like it should.

Do this:

ln -s /opt/MailScanner/bin/Quick.Peek /usr/sbin/Quick.Peek

Restart mailscanner:
/etc/init.d/mailscanner restart

See what you get in your logs now.

Mailscanner.conf is correct script did its job

root@mail:/opt/MailScanner/etc# ln -s /opt/MailScanner/bin/Quick.Peek /usr/sbin/Quick.Peek
ln: creating symbolic link `/usr/sbin/Quick.Peek': File exists

same problem . nothing logged in baruwa and sender gets message saying message could not be scanned

logs also say

anner[10714]: Virus and Content Scanning: Starting
Dec 3 17:59:22 mail MailScanner[10765]: MailScanner E-Mail Virus Scanner version 4.81.4 starting...
Dec 3 17:59:22 mail MailScanner[10765]: Reading configuration file /opt/MailScanner/etc/MailScanner.conf
Dec 3 17:59:22 mail MailScanner[10765]: Reading configuration file /opt/MailScanner/etc/conf.d/README
Dec 3 17:59:22 mail MailScanner[10765]: Read 866 hostnames from the phishing whitelist
Dec 3 17:59:22 mail MailScanner[10765]: Read 3810 hostnames from the phishing blacklists
Dec 3 17:59:22 mail MailScanner[10765]: Config: calling custom init function BaruwaLowScore
Dec 3 17:59:22 mail MailScanner[10765]: Config: calling custom init function BaruwaBlacklist
Dec 3 17:59:22 mail MailScanner[10765]: Config: calling custom init function BaruwaSQL
Dec 3 17:59:22 mail MailScanner[10765]: Config: calling custom init function BaruwaHighScore
Dec 3 17:59:22 mail MailScanner[10765]: Baruwa - Populating high spam score settings
Dec 3 17:59:22 mail MailScanner[10765]: no high spam score settings found using defaults
Dec 3 17:59:22 mail MailScanner[10765]: Config: calling custom init function BaruwaWhitelist
Dec 3 17:59:22 mail MailScanner[10765]: Using SpamAssassin results cache
Dec 3 17:59:22 mail MailScanner[10765]: Connected to SpamAssassin cache database
Dec 3 17:59:22 mail MailScanner[10765]: Enabling SpamAssassin auto-whitelist functionality...
Dec 3 17:59:26 mail MailScanner[10765]: Connected to Processing Attempts Database
Dec 3 17:59:26 mail MailScanner[10765]: Found 4 messages in the Processing Attempts Database
Dec 3 17:59:26 mail MailScanner[10765]: Using locktype = flock

Post your mail.log again.

Did you build using proper dns names?