ispconfig tool - migration errors?

Discussion in 'ISPConfig 3 Priority Support' started by craig baker, May 14, 2018.

  1. craig baker

    craig baker Member HowtoForge Supporter

    I just configured a new server centos 7.4 (ala perfect server) and installed the migration tool on my existing.
    the dry run gave errors:

    [ERROR] API call to billing_invoice_vat_rate_get failed. See log file for details.
    [ERROR] API call to billing_invoice_vat_rate_add failed. See log file for details.
    [ERROR] API call to billing_invoice_vat_rate_get failed. See log file for details.
    [ERROR] API call to billing_invoice_vat_rate_add failed. See log file for details.
    [ERROR] API call to billing_invoice_vat_rate_get failed. See log file for details.

    I assumed that though I bought and installed billing module I've never actually used it - so no customers,clients or invoices.
    can these errors be ignored?
    I thought I had set some defaults (USD and Usa address) but doing the real migration - same errors. but it continued.

    anything toworry about?
    and can I do migration more than once if this is a real error? will it overwrite entries without problem?

    thanks O great Till :)
     
  2. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Hello,
    you can have a look at the migrate.log to see details on the errors. I assume that you did not install the billing module on the target so the corresponding API methods are missing there.
    If you don't want to migrate billing, please try using --exclude=billing when running the migration.

    Yes, you can run the migration multiple times and it will overwrite the previously migrated data, unless you state "no" when asked if you want to overwrite.
     
  3. craig baker

    craig baker Member HowtoForge Supporter

    ok -so I just install billing on the new server we should be good? what is NOT migrated?? I had some custom settings on postfix, and had to do mailman in round about ways.
    also - if I rerun the migration will it 'only' migration anything new? after all the migration is taking many hours. how about mail that came in after migration started? is a second migration much faster and can pick up only newer things?

    log says:
    --snip--
    2018-05-13 21:42:29 - [INFO] Migration tool version 2.0.2 started.
    2018-05-13 21:42:57 - [INFO] Config file saved.
    2018-05-13 21:42:57 - [INFO] Found 0 Reseller entries.
    2018-05-13 21:42:57 - [INFO] Found 0 Client entries.
    2018-05-13 21:42:57 - [INFO] Found 63 Domain entries.
    2018-05-13 21:42:57 - [INFO] Found 63 WebDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 4 VHost SubDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 0 VHost AliasDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 0 AliasDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 0 SubDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 34 MailDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 0 MailAliasDomain entries.
    2018-05-13 21:42:57 - [INFO] Found 51 MailUser entries.
    2018-05-13 21:42:57 - [INFO] Found 52 MailForward entries.
    2018-05-13 21:42:57 - [INFO] Found 0 MailAlias entries.
    2018-05-13 21:42:57 - [INFO] Found 0 MailCatchAll entries.
    2018-05-13 21:42:57 - [INFO] Found 0 MailFilter entries.
    2018-05-13 21:42:57 - [INFO] Found 0 MailTransport entries.
    2018-05-13 21:42:57 - [INFO] Found 0 MailRelayRecipient entries.
    2018-05-13 21:42:57 - [INFO] Found 73 SpamFilter entries.
    2018-05-13 21:42:57 - [INFO] Found 3 SpamFilter Whitelist entries.
    2018-05-13 21:42:57 - [INFO] Found 0 SpamFilter Blacklist entries.
    2018-05-13 21:42:57 - [INFO] Found 4 Mailing List entries.
    2018-05-13 21:42:57 - [INFO] Found 6 FTP User entries.
    2018-05-13 21:42:57 - [INFO] Found 0 Cron Jobs entries.
    2018-05-13 21:42:57 - [INFO] Found 15 Database User entries.
    2018-05-13 21:42:57 - [INFO] Found 64 Database entries.
    2018-05-13 21:42:57 - [INFO] Found 0 Protected Folder entries.
    2018-05-13 21:42:57 - [INFO] Found 0 Protected Folder User entries.
    2018-05-13 21:42:57 - [INFO] Found 88 DNS Zone entries.
    2018-05-13 21:42:57 - [INFO] Found 563 DNS Record entries.
    2018-05-13 21:42:57 - [INFO] Found 0 Shell User entries.
    2018-05-13 21:42:57 - [INFO] Found 0 DataCopy entries.
    2018-05-13 21:42:57 - [INFO] Found 1 invoice company entries.
    2018-05-13 21:42:57 - [INFO] Found 1 invoice settings entries.
    2018-05-13 21:42:57 - [INFO] Found 31 invoice vat rate entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice sepa mandate entries.
    2018-05-13 21:42:57 - [INFO] Found 2 invoice message template entries.
    2018-05-13 21:42:57 - [INFO] Found 3 invoice payment term entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice client settings entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice item template entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice item template custom field entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice item category entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice recurring item entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice item entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice message entries.
    2018-05-13 21:42:57 - [INFO] Found 0 invoice sepa xml export entries.
    2018-05-13 21:43:07 - [INFO] Successfully executed command mysql -h localhost -u 'root' -p'#6Lives!' -e "SELECT VERSION();"
    2018-05-13 21:43:07 - [INFO] Config file saved.
    2018-05-13 21:43:07 - [INFO] Directory /etc/letsencrypt exists.
    2018-05-13 21:43:07 - [INFO] Successfully executed command rsync --version 2>/dev/null
    2018-05-13 21:43:08 - [INFO] Rsync copy from /etc/letsencrypt/ to /etc/letsencrypt/ successful.
    2018-05-13 21:47:18 - [ERROR] API call to billing_invoice_company_get failed.
    2018-05-13 21:47:18 - [ERROR] JSON API REPLY ERROR: Method billing_invoice_company_get does not exist - Arguments sent were: array (
    'primary_id' =>
    array (
    'company_name' => 'test corp',
    ),
    'session_id' => 'fbd91ec087d704615784a9b2703c997f',
    )
    2018-05-13 21:47:18 - [ERROR] API call to billing_invoice_company_add failed.
    2018-05-13 21:47:18 - [ERROR] JSON API REPLY ERROR: Method billing_invoice_company_add does not exist - Arguments sent were: array (
    'client_id' => '0',
    'params' =>
    array (
    'invoice_company_id' => '1',
    'sys_perm_user' => 'riud',
    'sys_perm_group' => 'riud',
    'sys_perm_other' => '',
    'company_name' => 'test corp',
    'company_name_short' => NULL,
    'contact_name' => 'John Doe',
    'street' => 'Mainstreet 12',
    'zip' => '12345',
    'city' => 'The City',
    'state' => 'MyState',
    'country' => 'DE',
    'email' => '[email protected]',
    'internet' => 'www.test.tld',
    'telephone' => '1234-5678',
    'fax' => '1234-5678',
    'company_logo' => '',
    'ceo_name' => 'John Doe',

    ---snip--

    though is is still running! these are only errors.

    thanks for the help
    cdb.
     
  4. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    No. Billing has to be the most current version. Older ones won't have all the necessary API options to be migrated.
    Global options and server configs are not migrated of course. Neither are client templates or getmail config.
    Migration 2.0.2 is highly outdated and not recommended. You have received an update message from sendowl with a link to the most current version (as is sent on each update).
     
  5. craig baker

    craig baker Member HowtoForge Supporter

    its most recent one link points to - can you send me link to newest version? and I've now installed new billing on new server so maybe I should just can migration and start over?? can I stop it safely?
    or upgrade migration tool (howto?) and then do 'no' for overwrite? much faster???
     
  6. craig baker

    craig baker Member HowtoForge Supporter

    went back and looked never got an update message about migration. I see there are now 2 migration toolkits? migration and migration update? explain?
     
  7. craig baker

    craig baker Member HowtoForge Supporter

    I note further that when I log into roundcube (via the ip address/webmail) I get:

    DATABASE ERROR: CONNECTION FAILED!

    Unable to connect to the database!
    Please contact your server-administrator.

    does not give any option to enter a username. I did the perfect server 7.4 centos install and did install roundcube.
    does migration move the roundcube database? and maybe itsgot the OLD user password from the old server?
     
  8. craig baker

    craig baker Member HowtoForge Supporter

    roundcube log has:
    [14-May-2018 11:00:51 -0400]: <g82tq2as> DB Error: [1146] Table 'roundcubedb.session' doesn't exist (SQL Query: SELECT `vars`, `ip`, `changed`, now() AS ts FROM `session` WHERE `sess_id` = 'g82tq2as2480denqlpgrmnn7f6') in /usr/share/roundcubemail/program/lib/Roundcube/rcube_db.php on line 543 (GET /webmail/)
    [14-May-2018 11:00:51 -0400]: <g82tq2as> DB Error: [1146] Table 'roundcubedb.session' doesn't exist (GET /webmail/)
    [14-May-2018 11:00:51 -0400]: <g82tq2as> DB Error: [1146] Table 'roundcubedb.identities' doesn't exist (SQL Query: SELECT * FROM `identities` WHERE `del` <> 1 AND `user_id` = NULL ORDER BY `standard` DESC, `name` ASC, `email` ASC, `identity_id` ASC) in /usr/share/roundcubemail/program/lib/Roundcube/rcube_db.php on line 543 (GET /webmail/)
    [14-May-2018 11:00:51 -0400]: <g82tq2as> DB Error: [1146] Table 'roundcubedb.session' doesn't exist (SQL Query: INSERT INTO `session` (`sess_id`, `vars`, `ip`, `created`, `changed`) VALUES ('g82tq2as2480denqlpgrmnn7f6', 'dGVtcHxiOjE7bGFuZ3VhZ2V8czo1OiJlbl9VUyI7dGFza3xzOjU6ImxvZ2luIjtyZXF1ZXN0X3Rva2VufHM6MzI6IjFhYmY5OWE4OTdlYTIwODkyNGQzODZkOGM0MTI0YWRlIjs=', '192.168.2.144', now(), now())) in /usr/share/roundcubemail/program/lib/Roundcube/rcube_db.php on line 543 (GET /webmail/)
    [[email protected] roundcubemail]#

    so its complaining about the database? no errors were observed in the perfect server install
     
  9. craig baker

    craig baker Member HowtoForge Supporter

    ALAS migration has filed. last lines:
    [INFO] Skipping system server as we have no job entries for this target.
    [WARN] Source dir/file /var/lib/mailman/messages/ for mail server does not exist. Skipping sync job.
    [WARN] Source dir/file /var/lib/mailman/qfiles/ for mail server does not exist. Skipping sync job.
    ERROR 2006 (HY000) at line 989: MySQL server has gone away
    [ERROR] job failed. Stopping Sync!

    from the mijgration log:
    ---
    2018-05-14 15:19:22 - [INFO] File /tmp/migrate-import-tmp.sql.gz successfully transferred.
    2018-05-14 15:19:22 - [INFO] Command gunzip -c /tmp/migrate-import-tmp.sql.gz | mysql --max_allowed_packet=1G -h localhost -D c0conciergecare -u 'root' -p'####mypasswordremoved' failed.
    2018-05-14 15:19:22 - [ERROR] Execution of job failed: array (
    'type' => 'exec',
    'command' => 'gunzip -c /tmp/migrate-import-tmp.sql.gz | mysql --max_allowed_packet=1G -h localhost -D c0conciergecare -u \'root\' -p\'###mypasswordremoved\'',
    'info_message' => 'Propagating database c0conciergecare',
    'stop_on_error' => true,
    )
    2018-05-14 15:19:23 - [WARN] JSON API ERROR: session expired. Trying re-login!

    what to do now? can I recover?
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    You can always download the latest version trough the original download link that you received after purchase.

    If you opted to receive messages about updates during purchase and you confirmed that in the email we sent you, thenb you should have received emails about updated versions.

    Regarding your roundcube problem, seems as if you did not have roundcube fully installed during ISPConfig installation. The roundcube database is not part of ISPConfig and therefore gets not migrated by the Migration Tool. To use roundcube on the new server, it has to be installed completely and it seems as if you did not do the installation with roundcube web based installer yet.

    Btw. Please contact the Migration tool support here if you have any questions about the Migration Tool:

    https://www.ispconfig.org/get-support/?type=migration

    This forum here is for ISPConfig OpenSource software, the support for the Migration and Billing Tool is available at this location: https://www.ispconfig.org/get-support/
     
  11. craig baker

    craig baker Member HowtoForge Supporter

    thanks - I just posted a question over there about the migration failure. but I DID download the version from the original email link and I unpacked it and 2.02 is the included version!
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    I just checked your purchase and you are an owner of a Migration Tool 1.x license with a free version 2 license upgrade. This explains the difference in version numbers as your original download link is for version 1.x of the tool and you probably downloaded the newer version from the blog bost at ispconfig.org. I'll update the link in that post, thank you for pointing that out. Users that purchased a 2.x version have the correct link. The migration tool support will send you an updated version.
     
  13. craig baker

    craig baker Member HowtoForge Supporter

    one more (not really) migration problem - httpd wont start on the new server because its complaining with errors:
    May 14 12:25:28 ns10 httpd: Either all Options must start with + or -, or no Option may.
    and the offending vhost on old server had:
    Options Indexes FollowSymLinks MultiViews +ExecCGI
    with + only on ExecCGI.
    Any reason I cant just put + in front of all of them? I thought that was the default!
    thanks again. who know server migration could be such fun!
     
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    I guess using + on all of them should be fine.
     
  15. craig baker

    craig baker Member HowtoForge Supporter

    thanks I put + on all options on the THREE sites that were blowing it up and now it starts.
    odd as I thought I had same version httpd on centos 6.5 as 7.4. guess not. but if '+' is the default why just not give a warning instead of blowing up httpd? most odd.
    and yes somehow I missed a few lines in Perfect Server on the 'initialize database' function under roundcube. really helps when that gets clicked. :)
     
  16. craig baker

    craig baker Member HowtoForge Supporter

    and as always many thanks. <- Supporter for Life

    one more question years ago you visited my server to harden things up a bit - how could I get that done now since I have a shiny new 7.5 server? would definitely be worth paying for a visit since I'm sure things have changed since then. new anti-spam stuff? better antivirus? etc etc.
    Whats the current procedure for setting up a visit?
     
    till likes this.
  17. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Hello craig, you can of course put a + in front of all not having an -.
     
  18. till

    till Super Moderator Staff Member ISPConfig Developer

    Support by remote login is now provided by Florian Schaal, I'm sure he is able to do the hardening of your setup for you. The rates should not be very different from what I charged in the past. You can contact Florian here to get a quote: https://www.ispconfig.org/get-support/?type=ispconfig
     

Share This Page