Mailbox size policy & Cyrus

[oliver.blaha]

Hi,

I'm planning to implement a wrapper for Cyrus IMAP. I know, Cyrus is more complicated to install, but it has some advantages:

- More scalable than Courier
- Sieve mail filtering support
- Easy quota checking, e.g. for webmail clients

Integrating Cyrus support into ISPConfig won't be such a big deal, BUT:
Cyrus doesn't use system quotas, as it's running with its own user and group. This way quotas have to be setup directly in cyrus. That itself isn't a problem, as this could be handled similar to mbox files, but with mbox, you can limit the size by putting the mailboxes on a quota partition - with cyrus there is no such possibility.

This leads me to my question concerning a "mailbox size policy".

There are some different options how it could be managed:

1) The total size of all users' mailboxes (of one website) can't be more than the diskspace available to this web.
1a) This total size is subtracted from the group quota. Disadvantage: You might have used already more than the new group quota
1b) No subtraction takes place. Disadvantage: A website might use the double amount of space it was assigned.

2) Do it like the new SQL quota. Resellers have a limit_mail_space or sth like that, websites have a web_mail_space and can assign their users mailbox space from this mail space pool.

Advantage: Very flexible
Disadv.: A lot of different quotas

3) Resellers have just one total space limit, which includes sql, mail and web. Resellers then can setup webs:
sum(web) + sum(sql) + sum(web_mail_space) <= limit_space

Each website gets a web_mail_space quota, which is the maximum amount of space that can be dedicated to mailboxes:
sum(mailboxes) <= web_mail_space

Advantage: This gives (almost) maximum flexibility to resellers for their offers, but keeps handling resellers quite simple.
Disadvantage: I would have to change the sql quota handling, as currently there is a seperate reseller limit for it (but I guess as it's just been in trunk one day right now this shouldn't break anything *g*)

4) Just like 2 and 3 combined. One great total quota, but also seperate space quotas for web, mail, mysql. This way you can e.g. say "the reseller may use 1 GB of total space, but he may never assign more than 200 MB of database space"
Advantage: You can do whatever you want to Really maximum flexibility.
Disadv.: Much harder to implement.

What do you think?
I like 3), especially because it could be extended to 4) in future without breaking anything.

Hope you answer fast as I want to start as fast as possible... and I want to try getting it finished until friday.

[till]

Hi Oliver,

I think 3) is a good compromise between easy handling of reselers and the ability to define fine grained quotas.

Please make sure that this feature can be enabled / disabled by a setting in the system configuration (isp_server table) by either a checkbox or a select field where the POP3 / IMAP server type can be selected. As far as i know there is no easy way to anable mailquota with maildir + procmail, thats why we hide the field for maildir setups currently.

[oliver.blaha]

I thought about replacing the Maildir switch by a Mailsystem drop down list, if possible.
But: I think it would be much easier and safer to have an option in config.inc.php, because in my opinion this should never be changed in a running system as you could screw up to much.

I'd suggest to keep the Maildir option for compatibility, but introducing an option in config.inc.php - and if the latter is set, the option in the settings tab is ignored and hidden.

Opinions?

[till]

Originally Posted by oliver.blaha

I thought about replacing the Maildir switch by a Mailsystem drop down list, if possible.
But: I think it would be much easier and safer to have an option in config.inc.php, because in my opinion this should never be changed in a running system as you could screw up to much.

I'd suggest to keep the Maildir option for compatibility, but introducing an option in config.inc.php - and if the latter is set, the option in the settings tab is ignored and hidden.

I prefer if the settings remain in the isp_server table. It is easier to handle when the system is updated and almost all other options where stored there.

I guess most options in the server settings will crash your system when changed after initial setup. Maybe we shall place a warning message there.

Introducing 2 places for the same config setting where config.inc.php overrides the isp_server table may confuse poeple and will make support more difficult.

[oliver.blaha]

Just a little question is left:
Shall I implement the quota behaviour for cyrus only? Or should mbox mailboxes be restricted by this total quota, too? (Or should I simply add another option to enable/disable the behaviour? If yes, should that be a config file option or also a setting in isp_server? I guess you'll answer the latter *g*)

[till]

I think the best way will be to implement the new behaviour for cyrus and mbox, so we can reuse the mail quota field in the isp_isp_user table too.

[falko]

Originally Posted by till

As far as i know there is no easy way to anable mailquota with maildir + procmail, thats why we hide the field for maildir setups currently.

The main reason why we hide this field for Maildir is that Maildir is in the users' homedirs and therefore falls under the group quota of the web whereas mbox mailboxes may reside on another partition without quota enabled.

Originally Posted by till

I think the best way will be to implement the new behaviour for cyrus and mbox

I think we should also apply this to Maildir then, don't you think so?

[oliver.blaha]

Originally Posted by falko

I think we should also apply this to Maildir then, don't you think so?

I don't think this would make sense. I'm introducing a new value for limiting user_mailquota, and Maildir simply doesn't use that. Web quota of users shouldn't be touched by that, on the one hand because these are inside the boundaries of web space anyway, on the other hand because it's technically a WEB quota, that shouldn't be limited by some mail quota value.

But: As soon as Maildir would get its own mailbox quota, just like mbox and cyrus, I agree.

In short terms: I'd prefer to implement this behaviour for ALL mechanisms, but Maildir just doesn't use the value and thus isn't affected.

Another question:
As far as I understand the code, users are not really deleted in "user_delete", but only disabled, and get really purged in "web_user_clean". But the mbox file is already deleted in "user_delete"... Where should I delete cyrus mailboxes? Btw, it doesn't hurt creating mailboxes that already exist (e.g. when restoring).

And last: I'm quite sure I'll finish cyrus work today or tomorrow, cyrus interface itself is complete, just some cosmetic changes to do. But I still have to implement some imit checks.

Last question for this post: Cyrus IMAP is managed by an administrator user. This username is not hardcoded, but settable in /etc/imapd.conf. This means I need an option in ISPConfig to specify username and password of this user. Where should I add them? config.inc.php or isp_server? Maybe I should extend the installer to recognize if cyrus is installed, and if yes, ask during setup?

[till]

Originally Posted by oliver.blaha

Another question:
As far as I understand the code, users are not really deleted in "user_delete", but only disabled, and get really purged in "web_user_clean". But the mbox file is already deleted in "user_delete"... Where should I delete cyrus mailboxes? Btw, it doesn't hurt creating mailboxes that already exist (e.g. when restoring).

If you can recreate the mailboxes easily when the user is restored, put the code in user_delete.

And last: I'm quite sure I'll finish cyrus work today or tomorrow, cyrus interface itself is complete, just some cosmetic changes to do. But I still have to implement some imit checks.

Thats great. Please notify me when you checked in your changes, i would like to add some minor enhancements to the mail user form and wont break your large patch

Last question for this post: Cyrus IMAP is managed by an administrator user. This username is not hardcoded, but settable in /etc/imapd.conf. This means I need an option in ISPConfig to specify username and password of this user. Where should I add them? config.inc.php or isp_server?

1 Vote for isp_server.

Maybe I should extend the installer to recognize if cyrus is installed, and if yes, ask during setup?

That will be the optimum. Or you introduce a new variable in dist.txt which contains the default cyrus user of the linux distribution.

[oliver.blaha]

Originally Posted by till

If you can recreate the mailboxes easily when the user is restored, put the code in user_delete.

They can easily be recreated, but mails are lost when the mailbox is deleted. If I'm not completely wrong then currently mbox and Maildir behave different:

mbox: Mails are erased when deleting the user and won't be restored if you restore the mailbox.
Maildir: The mailbox is disabled when deleting, mail stays there until you empty the recycle bin.

Is this intended? Or am I wrong? (I currently have no ISPConfig ready for testing with Maildir or mbox, so I have to ask *g*)