IMAP and POP3 ports not responding after ISPConfig 3 update

After running ispconfig_update.sh, neither IMAP nor POP3 will not accept connections.
Worse, when trying to connect to the ISPConfig webpage, I get this:
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator at [email protected] to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
I'm wishing to roll back the configuration and get my mail working again ASAP!

Recently updated 14.04 to 16.04 however this issue did not appear until I ran ispconfig_update.sh

 

Further data...
The UFW seems to have lost all its settings as well. Created rules to open port 143 and 110. Server now responds to IMAP and POP3 requests but no logins are accepted.
Mail.log reports:
Jun 6 02:02:35 wmwisp dovecot: auth-worker(9137): Warning: mysql: Query failed, retrying: Unknown column 'maildir_format' in 'field list'
Jun 6 02:02:35 wmwisp dovecot: auth-worker(9137): Error: sql([email protected],199.xx.yy.zzz): Password query failed: Unknown column 'maildir_format' in 'field list'
So I now think the upgrade changed the database to a format that dovecot can't understand.
If I rollback the ISPconfig update, I'd expect dovecot would once again be able to allow users to get their email...
And fixing the email as quickly as possible is the immediate goal.
So, I expect I need to:
delete /usr/local/ispconfig and all of its contents...
copy ispconfig_software.tar.gz to /usr/local
tar xvf ispconfig_software.tar.gz to restore the directory's and files
The problem is I don't know how to remove and replace the mysql database...
I have the backup file ispconfig_db_backup.sql but I'm not sure how to use it...
Also, do I need to do anything with the etc.tar.gz file that is in the ispconfig_nnnn.net_date-here directory?
--

 

The file /usr/local/ispconfig/server/lib/mysql_clientdb.conf contains:
<?php
$clientdb_host = 'localhost';
$clientdb_user = 'root';
$clientdb_password = 'correctpassword';
?>

Ran ispconfig_update.sh as root. Chose git-stable as source. Selected all defaults. Mail system has same errors and attempts to access the ispconfig web page result in error 500 internal server error.

Jun 6 21:06:53 wmwisp dovecot: auth-worker(3102): Warning: mysql: Query failed, retrying: Unknown column 'maildir_format' in 'field list'
Jun 6 21:06:53 wmwisp dovecot: auth-worker(3102): Error: sql([email protected],199.xx.yy.zz): Password query failed: Unknown column 'maildir_format' in 'field list'
--

 

Additional data: Apache2 HTMLhosted websites are working... But any website hosted that uses PHP displays the ISPConfig Error 500 yellow banner page...
--

 

Update:
After compleatly removing the PHP packages and reinstalling them per this link https://www.virtualmin.com/documentation/system/os/ubuntu-trusty-to-xenial
My PHP based websites now respond... Including the ISPConfig login page. However, I can't actually login...
If I enter the correct username and password, it simply loops back for me to enter them again.
If I enter an incorrect username or password, it says 'Username or Password wrong'...
Progress... I guess.
--

 

Update:
Tried running ispconfig_update.sh using git-stable. It crashed out while making the backup with the following error...
PHP Fatal error: Uncaught Error: Call to undefined function mysqli_connect() in /tmp/update_from_dev_stable.sh.69871WVTbR/ispconfig3-stable-3.1-e6bd0a7dfa0abc5dc2b26913c059405ae468a768/install/update.php:240
Stack trace:
#0 {main}
thrown in /tmp/update_from_dev_stable.sh.69871WVTbR/ispconfig3-stable-3.1-e6bd0a7dfa0abc5dc2b26913c059405ae468a768/install/update.php on line 240

Now what?
--

 

This means a PHP module that code expects is not available.
This may be since you upgraded from Ubuntu 14.04 to 16.04. After an operating system upgrade you must follow the ISPConfig Perfect Server Guide for that new operating system and your insallation of web server et al to ensure you have installed the necessary packages, including PHP modules, and configured settings according to the new Guide.
If you compare the Perfect Server Guides you will note that they are not the same. Some packages may have been removed, added, or changed name, or ISPConfig is for reasons configured differently on the newer OS. Ubuntu update procedure does not know how to accommodate for the new requirements of ISPConfig.

 

I can find a guide for a fresh install... But not for an upgrade. I'd really prefer not to loose my mail...
--

 

I've completed sections 1-5 of the guide at https://www.howtoforge.com/tutorial...l-pureftpd-bind-postfix-doveot-and-ispconfig/ Postfix and Dovecot
Step 6 is:
6. Install Postfix, Dovecot, MariaDB, rkhunter and binutils
Postfix, Dovecot and MySQL are already installed and have mail , users and databases. Will reinstalling them erase the existing mail, users and/or database entries?
--

 

On Ubuntu if you do

Code:

apt-get install somepackage

and that somepackage is already installed, the apt-get command does nothing.
And if you already checked you have all packages mentioned in the Perfect Server Guide installed, you do not need to install those. Just check the configurations are correct.

 

Continued on...
The Mariadb install said it could not update my existing databases so it renamed the existing directory and created a new one.
The next issue was editing /etc/clamav/clamd.conf
to change: AllowSupplemen taryGroups false
to: AllowSupplemen aryGroups true
The line doesn't exist
--
There is a extra space in those Allow lines because the forum software doesn't like the word Supplemen t.

 

I have now completed the perfect server guide through section 16.
It now wants me to install ISPconfig. (section 17)
I want to be sure that this will not loose my users email accounts, passwords and contents.
I also wish to be sure not to loose my websites...
--

 

I did just try running the ispconfig_update.sh again using git-stable.

[email protected]:/# ispconfig_update.sh

removed ASCII Art
>> Update
Please choose the update method. For production systems select 'stable'.
WARNING: The update from GIT is only for development systems and may break your current setup. Do not use the GIT version on servers that host any live websites!
Note: Update all slave server, before you update master server.
Select update method (stable,git-stable,git-master) [stable]: git-stable
--2019-06-09 22:15:25-- https://git.ispconfig.org/ispconfig/ispconfig3/repository/archive.tar.gz?ref=stable-3.1
Resolving git.ispconfig.org (git.ispconfig.org)... 159.69.100.117
Connecting to git.ispconfig.org (git.ispconfig.org)|159.69.100.117|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3004444 (2.9M) [application/octet-stream]
Saving to: ‘ispconfig3-dev.tar.gz’
ispconfig3-dev.tar.gz 100%[============================================================================================>] 2.87M 2.44MB/s in 1.2s
2019-06-09 22:15:27 (2.44 MB/s) - ‘ispconfig3-dev.tar.gz’ saved [3004444/3004444]
Latest git version: ispconfig3-stable-3.1-e6bd0a7dfa0abc5dc2b26913c059405ae468a768
removed ASCII Art
>> Update
Operating System: Ubuntu 16.04.6 LTS (Xenial Xerus)
This application will update ISPConfig 3 on your server.
Shall the script create a ISPConfig backup in /var/backup/ now? (yes,no) [yes]: yes
Creating backup of "/usr/local/ispconfig" directory...
Creating backup of "/etc" directory...
PHP Fatal error: Uncaught Error: Call to undefined function mysqli_connect() in /tmp/update_from_dev_stable.sh.iICF2bSacz/ispconfig3-stable-3.1-e6bd0a7dfa0abc5dc2b26913c059405ae468a768/install/update.php:240
Stack trace:
#0 {main}
thrown in /tmp/update_from_dev_stable.sh.iICF2bSacz/ispconfig3-stable-3.1-e6bd0a7dfa0abc5dc2b26913c059405ae468a768/install/update.php on line 240
[email protected]:/#

Same error...
--

 

Please use CODE tags when pasting log listings.
What show the following commands:

Code:

lsb_release -a
apt-cache policy php-mysql

How was the upgrade from 14.04 to 16.04 done? The updater asks about configuration files whether to keep the old or use the new version. How were these questions answered? If the old configuration file was kept, this 16.04 system is running with 14.04 configuration.
Useful Internet Search Engines:

Code:

Call to undefined function mysqli_connect() ubuntu 16.04

 

Code:

[email protected]:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.6 LTS
Release:        16.04
Codename:       xenial
[email protected]:/# apt-cache policy php-mysql
php-mysql:
  Installed: (none)
  Candidate: 2:7.3+69+ubuntu16.04.1+deb.sury.org+2+php7.3
  Version table:
     2:7.3+69+ubuntu16.04.1+deb.sury.org+2+php7.3 500
        500 http://ppa.launchpad.net/ondrej/php/ubuntu xenial/main amd64 Packages
        500 http://ppa.launchpad.net/ondrej/php/ubuntu xenial/main i386 Packages
     1:7.0+35ubuntu6.1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages
     1:7.0+35ubuntu6 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu xenial/main i386 Packages
[email protected]:/# 

is the result of the commands you asked me to run.
--

 

Code:

apt-get do-release-upgrade

In general, if it told me that the config was unmodified, I let it replace it. If it said the configuration was not unmodified, I told it to keep the original...

I expect that is part of the issue.
--

 

OK... After some reading for that search results I ran this:

Code:

[email protected]:/etc/php/7.3# apt install php-mysqli
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package php-mysqli is a virtual package provided by:
  php7.3-mysql 7.3.6-1+ubuntu16.04.1+deb.sury.org+1
  php7.2-mysql 7.2.19-1+ubuntu16.04.1+deb.sury.org+1
  php7.1-mysql 7.1.30-1+ubuntu16.04.1+deb.sury.org+1
  php7.0-mysql 7.0.33-8+ubuntu16.04.1+deb.sury.org+1
  php5.6-mysql 5.6.40-8+ubuntu16.04.1+deb.sury.org+1
You should explicitly select one to install.

E: Package 'php-mysqli' has no installation candidate
[email protected]:/etc/php/7.3#

Not sure how to proceed from there...
--