Changing settings on some mailboxes makes ISPconfig delete them entirely

Discussion in 'General' started by s-andersen, May 6, 2016.

  1. s-andersen

    s-andersen New Member

    I have a rather annoying problem with ISPconfig 3. I'm currently running version 3.0.5.4p8, but the problem has always been there.
    Some mailboxes will always be deleted whenever a change is made to the account. It only happens for some accounts, but it seems to happen every time such an account is changed.
    I just got a support ticket from a client, that needed a password reset for a mailbox. I changed the password, it worked instantly, but a couple of minutes later, the account was deleted from /var/vmail/, but not from the database.
    Luckily I took a backup before the password reset, because this has happened to me several times with another account.
    With the other account it was a change of spamfilter settings, that provoked deletion. At that time I tried several things to prevent the account from being deleted (for instance I deleted it from database and created it manually again and backed up the files). But nothing worked so I eventually gave up and decided to make a backup every time I made a change.
    Can someone help me debug this problem. It's rather annoying to have to go through the extra step of taking a backup just for a trivial password reset.
     
  2. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    You can set the loglevel to debug to get some more informations. But i don`t know why a mailbox is deleted if you only update some values.
     
  3. felan

    felan Member HowtoForge Supporter

    I am experiencing the same problem here. I have had 3 mail accounts deleted when something in them have been updated.
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Never seen that on any server, maybe the mailbox files were damaged or you changed some global mail settings. As Florian mentioned, you can use the debug mode to find out why this happened.
     
  5. Jesse Norell

    Jesse Norell Well-Known Member

    This may be completely off-track to the issue, but I had something that sounds similar happen a few days ago in 3.1, but I believe it was in a mail forward -- logged in as a client I was seeing a forward for an address that was on another client's domain, and when viewing (not deleting, I think just changing tabs) it disappeared and I think ended up being owned by the reseller, rather than either client. I can easily attribute that to bad data either from months of 3.1 updates and/or my own mistakes as I learn/experiment, so just ignored it.

    Also possibly pointing to a race condition type issue in a similar area, I was adding some mail accounts to a new domain and I got an SQL error emailed to me; I didn't see it right away so don't know exactly what I did, but I would guess I may have created a mailbox, then deleted it, and recreated it all before cron ran. That doesn't necessarily explain updating an existing account and it deleting, and is in 3.1, but sometimes anecdotes help. The sql error emailed is:

    Code:
    To: [email protected]
    Subject: 14.09.2016-13:13 - WARNING - Falsche Anfrage / Wro...
    X-PHP-Originating-Script: 0:app.inc.php
    MIME-Version: 1.0
    Content-type: text/plain; charset=utf-8
    From: [email protected]
    Reply-To: [email protected]
    Message-Id: <[email protected]>
    Date: Wed, 14 Sep 2016 11:13:01 -0600 (MDT)
    Content-Transfer-Encoding: quoted-printable
    
    14.09.2016-13:13 - WARNING - Falsche Anfrage / Wrong QuerySQL-Query =3D R=
    EPLACE INTO `mail_user` (`mailuser_id`,`sys_userid`,`sys_groupid`,`sys_pe=
    rm_user`,`sys_perm_group`,`sys_perm_other`,`server_id`,`email`,`login`,`p=
    assword`,`name`,`uid`,`gid`,`maildir`,`maildir_format`,`quota`,`cc`,`send=
    er_cc`,`homedir`,`autoresponder`,`autoresponder_start_date`,`autoresponde=
    r_end_date`,`autoresponder_subject`,`autoresponder_text`,`move_junk`,`cus=
    tom_mailfilter`,`postfix`,`greylisting`,`access`,`disableimap`,`disablepo=
    p3`,`disabledeliver`,`disablesmtp`,`disablesieve`,`disablesieve-filter`,`=
    disablelda`,`disablelmtp`,`disabledoveadm`,`last_quota_notification`,`bac=
    kup_interval`,`backup_copies`) VALUES (NULL,NULL,NULL,NULL,NULL,'',NULL,N=
    ULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,'','',NULL,NULL,NULL,NULL,NUL=
    L,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,N=
    ULL,NULL,NULL,NULL) -> 1048 (Column 'sys_userid' cannot be null)
    
     
  6. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    Maybe there was an error from older installs? If you change a tab, this updates your data (as long as did not change the default in your config).

    Can you try this with the debug-level? stop cron, create a new domain, create a ne mailbox, delete the mailbox, create the mailbox again and run server.sh from the command-line?
     
  7. Jesse Norell

    Jesse Norell Well-Known Member

    I did this and I do not see the same sql error. I did get an email that's probably harmless which says:
    Code:
    19.09.2016-20:04 - WARNING - Unable to delete file: /var/vmail/domain.com/test/sieve/ispconfig.sieve
    The other day I got a similar email immediately before that sql error (same second timestamp):
    Code:
    14.09.2016-13:13 - WARNING - Unable to create symlink to active sieve filter
    Again that's probably harmless in itself, and may just be that I did something different the other day to create the problem, but there wouldn't be a problem where tasks in the queue can get run out of order, or even simultaneously, is there?
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    @Jesse Norell I think your errors are harmless, it's just that a sieve filter can not be added or deleted which might happen, the worst that could happen is that an email filter is not applied or an autoresponder is not send or that a file remained after deleting the mailbox. Tasks in the queue are always run in oder and they never run simultaniously.
     
  9. leejongsuk2222

    leejongsuk2222 New Member

    If such a data loss is there?
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    No, the sieve messages are not related to data loss and ispconfig. ISPConfig is not deleting any maildirs on update, we tested that and this is not reproducable.

    In case that a maildir is completely corrupted so that its structure is incorrect, then ispconfig moved it to /var/vmail/corrupted/ folder and adds a warning in the log that it found a invalid maildir and that its data was moved to the corrupted folder and that the maildir was recreated with correct structure.
     
  11. Andrea C.

    Andrea C. New Member

    Hi all,

    sorry if i resume this old post but we have same problem with last version updated from old versions. In this case we have changed password. In older cases we change spam filter or other parameters and trigger the same problem. Not in all account and not always
    We encounter the same problem.

    This is the LOG of yesterday:

    Code:
    Wed Apr 1 15:09:02 CEST 2020
    Wed Apr 1 15:09:03 CEST 2020
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Found 1 changes, starting update process.
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Calling function 'user_update' from plugin 'mail_plugin' raised by event 'mail_user_update'.
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Mailuser uid: 5000, gid: 5000
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Created Directory: /srv/data/vmail/customer.com
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Sent
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Drafts
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Trash
    Wed Apr 1 15:09:03 CEST 2020 01.04.2020-15:09 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Junk
    Wed Apr 1 15:09:04 CEST 2020 01.04.2020-15:09 - DEBUG - safe_exec cmd: chown -R 'vmail':'vmail' '/srv/data/vmail/customer.com/user01' - return code: 0
    Wed Apr 1 15:09:04 CEST 2020 01.04.2020-15:09 - DEBUG - Set ownership on /srv/data/vmail/customer.com/user01
    Wed Apr 1 15:13:02 CEST 2020 01.04.2020-15:13 - WARNING - There is already a lockfile set, but no process running with this pid (13332). Continuing.
    Wed Apr 1 15:13:02 CEST 2020
    Wed Apr 1 15:13:02 CEST 2020
    Wed Apr 1 15:13:02 CEST 2020 01.04.2020-15:13 - DEBUG - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    Wed Apr 1 15:13:02 CEST 2020 01.04.2020-15:13 - DEBUG - Found 1 changes, starting update process.
    Wed Apr 1 15:13:02 CEST 2020 01.04.2020-15:13 - DEBUG - Calling function 'user_update' from plugin 'mail_plugin' raised by event 'mail_user_update'.
    Wed Apr 1 15:13:02 CEST 2020 /usr/bin/fail2ban-client
    Wed Apr 1 15:13:02 CEST 2020 /sbin/iptables
    Wed Apr 1 15:13:02 CEST 2020 /sbin/ip6tables
    Wed Apr 1 15:13:02 CEST 2020 01.04.2020-15:13 - DEBUG - Mailuser uid: 5000, gid: 5000
    Wed Apr 1 15:13:02 CEST 2020 01.04.2020-15:13 - DEBUG - Created Directory: /srv/data/vmail/customer.com
    Wed Apr 1 15:13:03 CEST 2020 which: no tw_cli in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
    Wed Apr 1 15:13:03 CEST 2020 which: no hpacucli in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
    Wed Apr 1 15:13:03 CEST 2020 which: no megacli in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
    Wed Apr 1 15:13:03 CEST 2020 which: no megacli64 in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
    Wed Apr 1 15:13:03 CEST 2020 which: no arcconf in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
    Wed Apr 1 15:13:03 CEST 2020 01.04.2020-15:13 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Sent
    Wed Apr 1 15:13:03 CEST 2020 01.04.2020-15:13 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Drafts
    Wed Apr 1 15:13:04 CEST 2020 01.04.2020-15:13 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Trash
    Wed Apr 1 15:13:04 CEST 2020 01.04.2020-15:13 - DEBUG - Created Maildir /srv/data/vmail/customer.com/user01/Maildir with subfolder: Junk
    Wed Apr 1 15:13:04 CEST 2020 01.04.2020-15:13 - DEBUG - safe_exec cmd: chown -R 'vmail':'vmail' '/srv/data/vmail/customer.com/user01' - return code: 0
    Wed Apr 1 15:13:04 CEST 2020 01.04.2020-15:13 - DEBUG - Set ownership on /srv/data/vmail/customer.com/user01
    Wed Apr 1 15:14:01 CEST 2020 01.04.2020-15:14 - WARNING - There is already an instance of server.php running with pid 4205.
    Wed Apr 1 15:14:01 CEST 2020 01.04.2020-15:14 - DEBUG - safe_exec cmd: rm -fr '/srv/data/vmail/customer.com/user01' - return code: 15
    Wed Apr 1 15:14:01 CEST 2020 mv: cannot move `/srv/var/vmail/customer.com/user01' to a subdirectory of itself, `/srv/data/vmail/customer.com/user01/user01'
    Wed Apr 1 15:14:01 CEST 2020 01.04.2020-15:14 - DEBUG - safe_exec cmd: mv -f '/srv/var/vmail/customer.com/user01' '/srv/data/vmail/customer.com/user01' - return code: 1
    Wed Apr 1 15:14:02 CEST 2020 01.04.2020-15:14 - DEBUG - Moved Maildir from: /srv/var/vmail/customer.com/user01 to /srv/data/vmail/customer.com/user01
    Wed Apr 1 15:14:02 CEST 2020 01.04.2020-15:14 - DEBUG - Calling function 'update' from plugin 'maildeliver_plugin' raised by event 'mail_user_update'.
    Wed Apr 1 15:14:02 CEST 2020 01.04.2020-15:14 - DEBUG - Calling function 'user_settings_update' from plugin 'rspamd_plugin' raised by event 'mail_user_update'.
    Wed Apr 1 15:14:02 CEST 2020 01.04.2020-15:14 - DEBUG - Processed datalog_id 227
    Wed Apr 1 15:14:02 CEST 2020 01.04.2020-15:14 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    Wed Apr 1 15:14:02 CEST 2020 finished.
    Wed Apr 1 15:15:01 CEST 2020
    Wed Apr 1 15:15:01 CEST 2020
    Thanks you very much for any help
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    As you can see in the log, the system had to create new maildirs as the old ones did you match the settings that you have chosen under System > server config > mail. Maybe you switched from courier to dovecot under System > server config > mail or something similar so that the setting there does not match the actual services running on your system.
     
  13. Andrea C.

    Andrea C. New Member

    Thanks for the quick answer. We have verified and the settings in server config-mail are correct. This server was born with dovecot and we haven't change any structure. From the start in far 2013 the problem exist. The problem show at random and we tried to update 2 weeks ago without fix
     
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    I'm not aware of any other system with such an issue, as you can see this thread is very old and was not even reproducible at that time. It seems as if your system has maildirs in courier format and recreates them in the correct dovecot format now when you change something. So the question is, why they were originally in courier format, did you migrate them from courier based system in the past?
     
  15. Andrea C.

    Andrea C. New Member

    This server install was born with dovecot. Never installed courier. The mails was migrated using Imapsync.
    The Maildir structure is in standard dovecot format. We tried now and on another account created in same manner we don't have the problem. We do more tests today
     

Share This Page