PDA

View Full Version : Virtual Users + Domains With Postfix, Courier + MySQL (+SMTP-AUTH, SpamA, ClamAV)


savkar
31st January 2006, 13:11
Hi.

The tutorial was excellent! I am fully up and running/operational.

one question I have regards spamassassin. I understand that it works best if you train it with ham and spam.

I currently have been collecting from users various emails that have come into one or the other category (ham caught by SA, or spam that was not caught).

What is the best way after setting up amavis-spamassassin with the directions in the tutorial to set up a cron job to automatically train spamassassin?

Do I simply pipe the messages thru sa-learn --spam or sa-learn --ham, or is it more complicated?

When I have looked on the web, I see that people do things like --rebuild and so forth, but I am unsure what this all does.

Also, after I train spamassassin with the spam/ham, do I need to ever retain those messages or can they be disgarded?

Thanks in advance-- would be great if the sa-learn method was added to the tutorial since I got the impression this is such an important part of sa to have it being trained.

Sunil

falko
1st February 2006, 13:52
Do I simply pipe the messages thru sa-learn --spam or sa-learn --ham, or is it more complicated?
Normally you do it like this:
/usr/bin/sa-learn --spam -p /var/amavisd/.spamassassin/user_prefs --mbox /var/mail/spam for spam and
/usr/bin/sa-learn --ham -p /var/amavisd/.spamassassin/user_prefs --mbox /var/mail/notspam for ham where /var/mail/spam and /var/mail/notspam are mbox mailboxes with spam/ham (you can have your users send spam/ham to these mailboxes for training purposes).

I recommend to run man sa-learn to find the correct options for your setup.


Also, after I train spamassassin with the spam/ham, do I need to ever retain those messages or can they be disgarded?


You can delete the messages afterwards. :)

savkar
1st February 2006, 20:47
I noticed that in your description of the virtual setup, you never run razor2 to create a user account for reporting. Is there a reason you avoid this, or is it purely because you think it is a per user preference whether they report spam or not?

I also noticed after I created a user which I was logged into amavis that I had a new directly .razor under /var/lib/amavis. Is this .razor account taken over in preference over the /etc/razor account setup thru your tutorial? There is actually no conf file in /var/lib/amavis/.razor but there are the same server files that contain URLs.

Sunil

falko
2nd February 2006, 01:05
I noticed that in your description of the virtual setup, you never run razor2 to create a user account for reporting. Is there a reason you avoid this, or is it purely because you think it is a per user preference whether they report spam or not?
You don't need to report spam - I think most users will be satisfied if razor identifies spam for them.

I also noticed after I created a user which I was logged into amavis that I had a new directly .razor under /var/lib/amavis. Is this .razor account taken over in preference over the /etc/razor account setup thru your tutorial? There is actually no conf file in /var/lib/amavis/.razor but there are the same server files that contain URLs.

What's in /var/lib/amavis/.razor and /etc/razor?

savkar
4th February 2006, 14:48
Actually, the two directories are very similar. I think I am fine now, but it was just interesting.

However, next question-- I have SPAM tagged and then forwarded to the users, with individual SPAM folders the SPAM filters into so they can check it. They then have the ability to indicate whether the email was not really spam and flip to the inbox and also alert me with the message so I can then use sa-learn to update the bayesian filters, or vice versa for something that slips thru tag it as spam which alerts me the alternative.

My question: You also have us use the amavisd quarantine -- what really is the use of this given what I am doing? That is, what added value do I have quarantining user spam if it is tagged at a certain level? I presume for my setup, I would just disregard this and set the quarantine for spam as undef?

Do many people do this? or is there something special about quarantining I am losing?

falko
4th February 2006, 19:38
My question: You also have us use the amavisd quarantine -- what really is the use of this given what I am doing? That is, what added value do I have quarantining user spam if it is tagged at a certain level? I presume for my setup, I would just disregard this and set the quarantine for spam as undef?

Do many people do this? or is there something special about quarantining I am losing?
Quarantine makes emails over a defined threshold go to a quarantine folder which the recipient can check from time to time if it's spam or not.
Given what you're doing I don't think you have to use this feature. :)

savkar
17th February 2006, 07:25
Just as a follow up, I thought i'd inform you of my setup in case anyone else thinks it is useful:

(i) Since I run squirrelmail, I installed the spam buttons package and then set it up so if someone detects spam that was not caught they select it and press spam, which sends the email to my spam catching virtual email address. It comes to me and goes into a spam maildir folder.

(ii) For ham, it goes likewise when people select the ham button to a ham maildir folder.

These two folders are actually called "Check Ham" and "Check Spam". I then daily when i get on line take a look. If I decide what they sent me is really spam and ham, I flip the messages over to anohter folder that is monitored once a day with a script, called "Learn Ham" and "Learn Spam".

With a simple script in cron.daily, I copy all the data out to /var/lib/amavis/spamham/ where there are two directories, one to hold all spam emails and one to hold all ham emails. From this directory sa-learn is run for ham and spam, wtih the output piped to a file and emailed me to indicate the status of hte spam run.

The end of script simply deletes the emails that were reviewed/learned.

It all seems to be working beautifully! !

wr19026
25th February 2006, 00:30
Well this looks like yet another easy to implement solution :)

One quick (and hopefully easy) question though: can I use it without any changes on Ubunty 5.10 (Breezy) as well? I'm specifically asking with regards to the quota patch, as I'm pretty sure that the rest will work like a charm.

falko
25th February 2006, 11:54
I haven't tested this on Ubuntu, but one problem I can think of is the Postfix version. Debian Sarge uses Postfix 2.1.5, and I guess Ubuntu uses a Postfix 2.2.x. Therefore you should have a look here: http://www.howtoforge.com/forums/showpost.php?p=11463&postcount=2

wr19026
26th February 2006, 20:14
I set this up on Ubuntu 5.10 and it works real nice. I left out the quota bit as it'll be small group of people who will have mail accounts on the server, and that's also where I think the version incompatibility could start and create issues.

One question then though; when I set up users they also are assigned a quota. What if I do not want a user to have a quota. Do I set the number to 0?

Additionally, how do I change a user's password? Would this be something I'd install PostfixAdmin for?

Thanks in advance.

falko
27th February 2006, 00:20
One question then though; when I set up users they also are assigned a quota. What if I do not want a user to have a quota. Do I set the number to 0?Yes, use 0.

Additionally, how do I change a user's password? Would this be something I'd install PostfixAdmin for?
You can use phpMyAdmin for this task.

savkar
27th February 2006, 02:14
I let users change their password using Squirrelmail with teh change_sqlpass plugin. There are multiple ways you could do this...

wr19026
27th February 2006, 18:28
Yes, use 0.

Excellent, thanks!

You can use phpMyAdmin for this task.

Well, I can...But I want to enable my users to do this themselves (as I tend to be a bit lazy :)) SO I tried the change_sqlpasswd plugin for squirrelmail but as I needed to install the compatibility plugin that blew up something in the PHP code. So that's a no go.

My PostfixAdmin looks interesting as it is a frontend that allws me to easily add new users, aliases etc. without having to log in to phpMyAdmin. And it allows users to change their password and forwarding as well.

wr19026
1st March 2006, 11:07
I should've known better than to mess around with integrating two HOWTOs about this subject. So I used the one that works, this one :) My compliments, your HOWTOs are of great quality and work really well.

Few questions though:
- On Ubuntu 5.10, when I do an apt-get upgrade it tells me it wants to upgrade postfix. Is my assumption correct that this would overwrite the installed version which has the quota patch? And if so, is there a way in which I can exclude postfix from the updates/upgrades?
- Changing a user's encrypted password using phpMyAdmin. As I cannot just go in and plug another password in there, how do I do this for an encrypted password?

Thanks in advance.

falko
1st March 2006, 14:13
- On Ubuntu 5.10, when I do an apt-get upgrade it tells me it wants to upgrade postfix. Is my assumption correct that this would overwrite the installed version which has the quota patch?
Yes.
And if so, is there a way in which I can exclude postfix from the updates/upgrades?
You can do that with apt-pinning:
http://www.debian.org/doc/manuals/apt-howto/ch-apt-get.en.html
http://jaqque.sbih.org/kplug/apt-pinning.html
http://www.argon.org/~roderick/apt-pinning.html


- Changing a user's encrypted password using phpMyAdmin. As I cannot just go in and plug another password in there, how do I do this for an encrypted password?

It's explained here: http://www.howtoforge.com/virtual_postfix_mysql_quota_courier_p5

wr19026
3rd March 2006, 00:19
Well I finally have a working mail server :) Excellent HOWTO! And thanks for your help.

I do have a question though, when trying to send e-mail to an external domain it works when I use Squirrelmail. Next I have set up the exact same account on Thunderbird.

When sending an e-mail to the same external address I get the error message that Relay Access is denied. The mail is not sent.

/var/log/mail.log shows the following:
Mar 3 00:04:10 blabla postfix/smtpd[30093]: connect from bla.bla.net[10.0.0.150]
Mar 3 00:04:10 blabla postfix/smtpd[30093]: NOQUEUE: reject: RCPT from bla.bla.net[10.0.0.150]: 554 <wr19026@xyz.nl>: Relay access denied; from=<wr19026@bla.net> to=<wr19026@xyz.nl> proto=ESMTP helo=<[10.0.0.150]>
Mar 3 00:04:19 blabla postfix/smtpd[30093]: lost connection after RCPT from
bla.bla.net[10.0.0.150]
Mar 3 00:04:19 blabla postfix/smtpd[30093]: disconnect from bla.bla.net[10.0.0.150]

This is what's in my /etc/postfix/main.cf:
# See /usr/share/postfix/main.cf.dist for a commented, more complete version

smtpd_banner = $myhostname ESMTP ready
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

myhostname = bla.bla.net
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = bla.bla.net, localhost, localhost.localdomain
relayhost = mailrelay.direct-adsl.nl
mynetworks = 127.0.0.0/8
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
virtual_alias_domains =
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email
.cf
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf
virtual_mailbox_base = /home/vmail
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
smtpd_use_tls = yes
smtpd_tls_cert_file = /etc/postfix/smtpd.cert
smtpd_tls_key_file = /etc/postfix/smtpd.key
transport_maps = proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
virtual_create_maildirsize = yes
virtual_mailbox_extended = yes
virtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailbox_limit_maps.cf
virtual_mailbox_limit_override = yes
virtual_maildir_limit_message = "The user you are trying to reach is over quota."
virtual_overquota_bounce = yes
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtu
al_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relo
cated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps
content_filter = amavis:[127.0.0.1]:10024
receive_override_options = no_address_mappings

I've seen the suggestion to edit the /etc/postfix/local-host-names file and that may solve the issue, but what about POP3 access from outside my network? That wouldn't work then would it?

Another piece of information that might be useful is that my router does not support loopback. So on my LAN I have to define the IMAP server name as 10.0.0.x

Any suggestions? Your help is much appreciated.

falko
3rd March 2006, 10:16
/var/log/mail.log shows the following:
Mar 3 00:04:10 blabla postfix/smtpd[30093]: connect from bla.bla.net[10.0.0.150]
Mar 3 00:04:10 blabla postfix/smtpd[30093]: NOQUEUE: reject: RCPT from bla.bla.net[10.0.0.150]: 554 <wr19026@xyz.nl>: Relay access denied; from=<wr19026@bla.net> to=<wr19026@xyz.nl> proto=ESMTP helo=<[10.0.0.150]>
Mar 3 00:04:19 blabla postfix/smtpd[30093]: lost connection after RCPT from
bla.bla.net[10.0.0.150]
Mar 3 00:04:19 blabla postfix/smtpd[30093]: disconnect from bla.bla.net[10.0.0.150]

You must enable something like "Server requires authentication." in your email client.

wr19026
3rd March 2006, 10:47
You must enable something like "Server requires authentication." in your email client.

I tried that in Thunderbird (Tools -> Account Settings -> Server Settings -> Security Settings -> Use secure authentication) but then it completely refuses access.

So, here's where I am at now: I can read and write e-mail, but I cannot save copies to Sent etc. when using Thunderbird from outside my LAN. Outside my LAN I use a different SMTP server by the way.

From inside my LAN I cannot send e-mails due to the error mentioned earlier. It seems that my SMTP server is refusing connections from other machines than localhost.

Could it be that it has something to do with this line:
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

falko
3rd March 2006, 16:42
If your clients are within the mynetworks value in /etc/postfix/main.cf, then they are allowed to send without authentication. OTherwise you must enable "Server requires authentication." in your email client. In Outlook you do it like this: http://mail.cs.uiuc.edu/relay/outlook-config.html There must be a similar setting in Thunderbird.

Outside my LAN I use a different SMTP server by the way.Then your sending problem has to do with the different SMTP server.

wr19026
3rd March 2006, 17:19
If your clients are within the mynetworks value in /etc/postfix/main.cf, then they are allowed to send without authentication.

Bingo! That must be it! The howto specifies 127.0.0.0/8. So now it makes sense why it was a bit wonky :)

Thanks again!

coza
6th March 2006, 18:07
Hi Falco,

The difference in the postfix config for 2.2, is the only difference that u need to create a virtual.cf with the following ?

user = <user_name> password = <password> hosts = server dbname = maildb query = SELECT goto FROM virtual WHERE address='%s'

Or do you need to add that to each of the *virtual*.cf files ?

I cant make out if thats what you mean..

Thanks.

CoZa

falko
6th March 2006, 22:51
You have to change the sql queries in all .cf files.

coza
7th March 2006, 11:42
Thanks,

I see that in the new format the lines :

table =
select_field =
where_field =

have been replaced with :

query = SELECT goto FROM virtual WHERE address='%s'

What format does this line need to be changed in each of the .cf's ?
Are there any examples of this setup for your howto ?
Thanks in advance,

CoZa

falko
7th March 2006, 13:39
Thanks,

I see that in the new format the lines :

table =
select_field =
where_field =

have been replaced with :

query = SELECT goto FROM virtual WHERE address='%s'

What format does this line need to be changed in each of the .cf's ?
Are there any examples of this setup for your howto ?
Thanks in advance,

CoZa
In the old format you have something like

table = virtual
select_field = goto
where_field = address
In the new format you'd replace it with
query = SELECT goto FROM virtual WHERE address='%s'

coza
16th March 2006, 11:30
Thanks for that Falko, i finally manaded to change the queries for my Ubuntu server.

I am now however struggling with squirrelmail, Is there any special config changes to squirrelmail, to work with this howto ?

I have installed SM and configured the tables etc.
I have created mailboxes and sent mail to them , creating the actual mailboxes.
Everything seems fine exept....

I cannot loginto squirrlemail.
I get the error : Unknown user or password incorrect.
I have changed the passwords a few times and doesnt help.

syslog says :

Mar 16 12:16:34 localhost imaplogin: Connection, ip=[::ffff:127.0.0.1]
Mar 16 12:16:39 localhost imaplogin: LOGIN FAILED, ip=[::ffff:127.0.0.1]

netstat - tap says :

root@sipho:/etc/postfix# netstat -tap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 localhost.localdo:10024 *:* LISTEN 6469/amavisd (maste
tcp 0 0 localhost.localdo:10025 *:* LISTEN 6787/master
tcp 0 0 localhost.localdo:mysql *:* LISTEN 6674/mysqld
tcp 0 0 *:10000 *:* LISTEN 6978/perl
tcp 0 0 *:smtp *:* LISTEN 6787/master
tcp 0 0 localhost.localdo:mysql localhost.localdo:52235 ESTABLISHED6674/mysqld
tcp 0 0 localhost.localdo:mysql localhost.localdo:35347 ESTABLISHED6674/mysqld
tcp 0 0 localhost.localdo:52148 localhost.localdo:mysql ESTABLISHED6912/amavisd (child
tcp 0 0 localhost.localdo:52235 localhost.localdo:mysql ESTABLISHED6913/amavisd (child
tcp 0 0 localhost.localdo:35347 localhost.localdo:mysql ESTABLISHED6908/apache2
tcp 0 0 localhost.localdo:mysql localhost.localdo:52148 ESTABLISHED6674/mysqld
tcp6 0 0 *:imaps *:* LISTEN 6579/couriertcpd
tcp6 0 0 *:pop3s *:* LISTEN 6614/couriertcpd
tcp6 0 0 *:pop3 *:* LISTEN 6594/couriertcpd
tcp6 0 0 *:imap2 *:* LISTEN 6559/couriertcpd
tcp6 0 0 *:www *:* LISTEN 6858/apache2
tcp6 0 0 *:ssh *:* LISTEN 6821/sshd
tcp6 0 2668 sipho.werty.com:ssh ::ffff:192.168.45:54919 ESTABLISHED7095/2
tcp6 0 0 sipho.werty.com:ssh ::ffff:192.168.45:52956 ESTABLISHED7032/1
tcp6 0 0 sipho.werty.com:ssh ::ffff:192.168.45:51802 ESTABLISHED6922/0

Im all out of ideas , can you maybe shed some light on this one for me please ?

Regards,

CoZa

falko
16th March 2006, 18:21
Did you use the correct username (email address) and password?

There must be some kind of Squirrelmail configuration file. I don't know its correct location, so you must search a little bit for it.

coza
20th March 2006, 10:36
Have checked through the Squirrel mail config, and cand find anything..

Has anyone had Squirrelmail up and running with this setup on ubuntu 5.10 ?


Shot,

CoZa

Ovidiu
20th March 2006, 12:10
I let users change their password using Squirrelmail with teh change_sqlpass plugin. There are multiple ways you could do this...

as we have real system users why not use the change_syste-.passwort squirrelmail plugin? or are there separate passwords for system user and email access? how exactly did you integrate the change_mysql passwd ? anything special to be careful about?

Ovidiu
20th March 2006, 12:14
Hi.

The tutorial was excellent! I am fully up and running/operational.

one question I have regards spamassassin. I understand that it works best if you train it with ham and spam.

I currently have been collecting from users various emails that have come into one or the other category (ham caught by SA, or spam that was not caught).

What is the best way after setting up amavis-spamassassin with the directions in the tutorial to set up a cron job to automatically train spamassassin?

Do I simply pipe the messages thru sa-learn --spam or sa-learn --ham, or is it more complicated?

When I have looked on the web, I see that people do things like --rebuild and so forth, but I am unsure what this all does.

Also, after I train spamassassin with the spam/ham, do I need to ever retain those messages or can they be disgarded?

Thanks in advance-- would be great if the sa-learn method was added to the tutorial since I got the impression this is such an important part of sa to have it being trained.

Sunil

by the way it looks like you are using only one preference file (db) for all mail users , have you thought about givinf each mail user his own spam and ham db for bayes purposes? it won't take more than 2-3MB / user - if doing so how can one separate the sa-learn scripts so that each users spam gets learned only for himself?

I had a link to a good howto but can't find it anymore...

anyone gotten so far as to have sa with sa-learn and dcc razor2 and pyzor working on a per user basis?

###edit###

I found out you can use: --dbpath

but how would that look like so that I could use a variable and the path gets substituted by the actual user...

p.s. Falko once helped me do this, where would I find these files (I still have a backup of that old system)

coza
22nd March 2006, 11:43
Hi guys,

I have discovered that my problem is not only isolated to the webmail interface.

- I am not able to authenticate to any mailbox via a mail client or telnet session either.
- I am 100% that it is not me typing the password incorrectly. and i have re-set it plenty of times. I am using the whole email address as the username.
- I can send mail to the mailboxes and i have seen that the /home/vmail has the directories and new mail folders created.

I am running Ubuntu 5.10 with postfix 2.2 and i have updated the mysql queiries for 2.2

Can anyone help me out ?
Thanks again in advance.

CoZa

coza
22nd March 2006, 13:47
I managed to fix it, I hadn't #'ed out the line MYSQL_NAME_FIELD in /etc/courier/authmysqlrc.

thanks for your time though.

CoZa