Ispconfig 3.1.14p2 web serv er config missing

Discussion in 'Installation/Configuration' started by neutrinohu, Aug 5, 2019.

  1. neutrinohu

    neutrinohu New Member

    Hi all,

    I've faced with the below problem:

    Until last friday my ispconfig setup was running fine. Tomorrow i've discovered all of the config is missing from "System -> Server config -> Web". How can i restore it (at least to the factory default)? I have no backup from this database.

    System test output:
    cat htf_report.txt | more

    ##### SERVER #####
    IP-address (as per hostname): ***.***.***.***
    [WARN] could not determine server's ip address by ifconfig
    [INFO] ISPConfig is installed.

    ##### ISPCONFIG #####
    ISPConfig version is 3.1.14p2


    ##### VERSION CHECK #####

    [INFO] php (cli) version is 7.3.7-2+ubuntu18.04.1+deb.sury.org+1

    ##### PORT CHECK #####

    [WARN] Port 22 (SSH server) seems NOT to be listening

    ##### MAIL SERVER CHECK #####


    ##### RUNNING SERVER PROCESSES #####

    [INFO] I found the following web server(s):
    Apache 2 (PID 1506)
    [INFO] I found the following mail server(s):
    Postfix (PID 1575)
    [INFO] I found the following pop3 server(s):
    Dovecot (PID 1120)
    [INFO] I found the following imap server(s):
    Dovecot (PID 1120)
    [INFO] I found the following ftp server(s):
    PureFTP (PID 1701)

    ##### LISTENING PORTS #####
    (only ()
    Local (Address)
    [anywhere]:42222 (1397/sshd)
    [anywhere]:110 (1120/dovecot)
    [anywhere]:143 (1120/dovecot)
    [anywhere]:80 (1506/apache2)
    [anywhere]:465 (1575/master)
    [anywhere]:21 (1701/pure-ftpd)
    ***.***.***.***:53 (773/systemd-resolve)
    [anywhere]:25 (1575/master)
    [anywhere]:993 (1120/dovecot)
    [anywhere]:995 (1120/dovecot)
    [localhost]:10023 (1475/postgrey)
    [localhost]:10024 (1739/amavisd-new)
    [localhost]:10025 (1575/master)
    [localhost]:10026 (1739/amavisd-new)
    [localhost]:10027 (1575/master)
    [anywhere]:587 (1575/master)
    [localhost]:11211 (947/memcached)
    *:*:*:*::*:42222 (1397/sshd)
    [localhost]10 (1120/dovecot)
    [localhost]43 (1120/dovecot)
    *:*:*:*::*:8080 (1506/apache2)
    *:*:*:*::*:8081 (1506/apache2)
    *:*:*:*::*:21 (1701/pure-ftpd)
    *:*:*:*::*:443 (1506/apache2)
    *:*:*:*::*:993 (1120/dovecot)
    *:*:*:*::*:995 (1120/dovecot)
    *:*:*:*::*:10023 (1475/postgrey)
    *:*:*:*::*:10024 (1739/amavisd-new)
    *:*:*:*::*:10026 (1739/amavisd-new)
    *:*:*:*::*:3306 (1473/mysqld)




    ##### IPTABLES #####
    Chain INPUT (policy ACCEPT)
    target prot opt source destination

    Chain FORWARD (policy ACCEPT)
    target prot opt source destination

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination

    System: Ubuntu 18.04 LTS
    Web server: Apache 2.4
    Mysql 5.7

    I've checked the DB and there is no "[web]" section in the "server" table's "config" field.
    [global]
    webserver=apache
    mailserver=postfix
    dnsserver=mydns

    [server]
    auto_network_configuration=n
    ip_address=192.168.1.100
    netmask=255.255.255.0
    v6_prefix=
    gateway=0.0.0.0
    firewall=ufw
    hostname=myhostname
    nameservers=8.8.8.8,8.8.4.4
    loglevel=1
    admin_notify_events=1
    backup_dir=/www/backup
    backup_tmp=/tmp
    backup_dir_is_mount=n
    backup_mode=rootgz
    backup_time=0:00
    backup_delete=n
    monit_url=
    monit_user=
    monit_password=
    munin_url=
    munin_user=
    munin_password=
    monitor_system_updates=y
    log_retention=10
    migration_mode=n

    [mail]
    module=postfix_mysql
    maildir_path=/home/vmail/[domain]/[localpart]
    maildir_format=maildir
    homedir_path=/home/vmail
    dkim_path=/var/lib/amavis/dkim
    dkim_strength=2048
    relayhost_password=
    pop3_imap_daemon=dovecot
    mail_filter_syntax=maildrop
    mailuser_uid=5000
    mailuser_gid=5000
    mailuser_name=vmail
    mailuser_group=vmail
    mailbox_virtual_uidgid_maps=n
    relayhost=
    relayhost_user=
    reject_sender_login_mismatch=n
    mailbox_size_limit=0
    message_size_limit=0
    mailbox_quota_stats=y
    realtime_blackhole_list=zen.spamhaus.org
    overquota_notify_admin=n
    overquota_notify_client=n
    overquota_notify_freq=7
    overquota_notify_onok=n

    [getmail]
    getmail_config_dir=/etc/getmail

    [dns]
    bind_user=root
    bind_group=bind
    bind_zonefiles_dir=/etc/bind
    named_conf_path=/etc/bind/named.conf
    named_conf_local_path=/etc/bind/named.conf.local
    disable_bind_log=n

    [fastcgi]
    fastcgi_starter_path=/var/www/php-fcgi-scripts/[system_user]/
    fastcgi_starter_script=.php-fcgi-starter
    fastcgi_alias=/php/
    fastcgi_phpini_path=/etc/php/7.2/cgi/
    fastcgi_children=8
    fastcgi_max_requests=5000
    fastcgi_bin=/usr/bin/php-cgi
    fastcgi_config_syntax=2

    [jailkit]
    jailkit_chroot_home=/home/[username]
    jailkit_chroot_app_sections=basicshell editors extendedshell netutils ssh sftp scp groups jk_lsh
    jailkit_chroot_app_programs=/usr/bin/groups /usr/bin/id /usr/bin/dircolors /usr/bin/lesspipe /usr/bin/basename /usr/bin/dirname /usr/bin/nano /usr/bin/pico /usr/bin/mysql /usr/bin/mysqldump /usr/bin/git /usr/bin/git-receive-pack /usr/bin/git-upload-pack /usr/bin/unzip /usr/bin/zip /bin/tar /bin/rm /usr/bin/patch /etc/localtime
    jailkit_chroot_cron_programs=/usr/bin/php /usr/bin/perl /usr/share/perl /usr/share/php

    [vlogger]
    config_dir=/etc

    [cron]
    init_script=cron
    crontab_dir=/etc/cron.d
    wget=/usr/bin/wget

    [rescue]
    try_rescue=n
    do_not_try_rescue_httpd=n
    do_not_try_rescue_mongodb=n
    do_not_try_rescue_mysql=n
    do_not_try_rescue_mail=n


    Any help is appreciated.

    Thanks
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Use this web section:

    Code:
    [web]
    server_type=apache
    website_basedir=/var/www
    website_path=/var/www/clients/client[client_id]/web[website_id]
    website_symlinks=/var/www/[website_domain]/:/var/www/clients/client[client_id]/[website_domain]/
    website_symlinks_rel=n
    network_filesystem=n
    vhost_conf_dir=/etc/apache2/sites-available
    vhost_conf_enabled_dir=/etc/apache2/sites-enabled
    nginx_vhost_conf_dir=/etc/nginx/sites-available
    nginx_vhost_conf_enabled_dir=/etc/nginx/sites-enabled
    security_level=20
    user=www-data
    group=www-data
    nginx_user=www-data
    nginx_group=www-data
    apps_vhost_enabled=y
    apps_vhost_port=8081
    apps_vhost_ip=_default_
    apps_vhost_servername=
    php_open_basedir=[website_path]/web:[website_path]/private:[website_path]/tmp:/var/www/[website_domain]/web:/srv/www/[website_domain]/web:/usr/share/php5:/usr/share/php:/tmp:/usr/share/phpmyadmin:/etc/phpmyadmin:/var/lib/phpmyadmin:/dev/random:/dev/urandom
    htaccess_allow_override=All
    awstats_conf_dir=/etc/awstats
    awstats_data_dir=/var/lib/awstats
    awstats_pl=/usr/lib/cgi-bin/awstats.pl
    awstats_buildstaticpages_pl=/usr/share/awstats/tools/awstats_buildstaticpages.pl
    php_ini_path_apache=/etc/php/7.2/apache2/php.ini
    php_ini_path_cgi=/etc/php/7.2/cgi/php.ini
    check_apache_config=y
    enable_sni=y
    enable_spdy=n
    skip_le_check=n
    enable_ip_wildcard=y
    overtraffic_notify_admin=y
    overtraffic_notify_client=y
    nginx_cgi_socket=/var/run/fcgiwrap.socket
    php_fpm_init_script=php7.2-fpm
    php_fpm_ini_path=/etc/php/7.2/fpm/php.ini
    php_fpm_pool_dir=/etc/php/7.2/fpm/pool.d
    php_fpm_start_port=9010
    php_fpm_socket_dir=/var/lib/php7.2-fpm
    php_default_name=Default
    set_folder_permissions_on_update=n
    add_web_users_to_sshusers_group=y
    connect_userid_to_webid=n
    connect_userid_to_webid_start=10000
    web_folder_protection=y
    php_ini_check_minutes=1
    overquota_notify_admin=y
    overquota_notify_client=y
    overquota_notify_freq=7
    overquota_db_notify_admin=y
    overquota_db_notify_client=y
    overquota_notify_onok=n
    logging=yes
    php_fpm_reload_mode=reload
    website_autoalias=
    vhost_rewrite_v6=n
    nginx_enable_pagespeed=n
    CA_path=
    CA_pass=
    php_handler=no
    php_fpm_incron_reload=n
     
  3. neutrinohu

    neutrinohu New Member


    Thanks, its working now :)
     
  4. nhybgtvfr

    nhybgtvfr Active Member

    and at the very least, start taking regular backups of the master dbispconfig database.
    it contains all your server configs, customers details, website details, database details, user logins, dns settings, etc.
    it's the database you least want to lose...…
     
  5. neutrinohu

    neutrinohu New Member


    Yeah yeah. After this case i know this product is not stable enought to just "let it run". I think its unacceptable in version 3 to loose data randomly from the internal database.... We are not in 2005!
     
  6. nhybgtvfr

    nhybgtvfr Active Member

    that's harsh. I've been running it for years, as have countless others, without any such issue, it's proven to be very stable, and does just run. even to the point that 'member' servers will still continue with their existing config and data even with the loss of the master server.
    something I don't see Plesk, cpanel et al doing.
    have you fully investigated the reason for the missing table data? you could have been hacked, a script run in the wrong location, an admin accidentally truncating the wrong table. to just blame randomness and unstable software is lazy and wrong.

    and I would say (and I hope at least 99% of sys admins would also say) it's unacceptable to not have multiple (confirmed recoverable) backups of all customer details, website/database/email/ftp user details/configs, server configs etc. especially when they're all in one easily backed up database.

    to assume your server will never crash, or can never be hacked, and it's data compromised, so there's no need for backups, is idiocy. we're not in 1982. ;)
     
  7. neutrinohu

    neutrinohu New Member

    My server is a test machine without direct internet access. I've reach it from internal network. At friday when i left the work every config was in place, but at monday morning i've found it without webserver config. Nobody logged into the machine, nobody has access to it (except me).
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    I have no Idea what you did and how you managed to wipe out parts of your server config. I develop and use ISPConfig since its beginning and never lost my server config. I know ISP's that run several thousand ISPConfig servers for their customers, they don't lose the server config and there are hundred thousands of ISPConfig systems out there which work perfectly fine.
     
  9. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Really … That is nonsense. If nobody logged in and nobody changed anything and the server was not altered in any other way there is no way the ISPConfig database can change so it looses server config data. It's not really part of good manners – although sadly often done – to blame software for human faults.
    The only case where this might (!) happen would be to install the master development branch from git, but I guess (and hope) you wouldn't do such a thing.
     
  10. Jesse Norell

    Jesse Norell Well-Known Member

    I've considered commenting a couple times recently how there sure has been an unusually high number of people losing their server config of late. I don't doubt it may be the 'fault' of the user, but I also wonder if either a different design for storing server config, or at minimum some automatic back-up and config syntax check would be worth considering to make things more robust.

    I remember I lost my server config one time by simply using phpmyadmin to examine that column to read the settings - iirc I didn't make any changes, but phpmyadmin changed the newlines so that ISPConfig could not read/parse the field, and of course random things start failing after that. My fault? Sure, I caused it. But it's a pretty reasonable scenario to use the included (in standard install) database editor to examine the database, and it certainly felt fragile, with room for improvement.

    I had never considered it before, but I wonder if there's any way to configure phpmyadmin to not allow inline editing of that column, and make that part of the perfect server guides. For that matter, maybe it's a bug in phpmyadmin that should be fixed.

    But does anyone have thoughts on ways the current design could be made more robust? Json encoding that field with a fallback to legacy ini parsing might be quite simple and more forgiving of errors, or at least of newline changes.
     
  11. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    That is at least worth thinking about. But in fact ini is a lot more user-friendly to edit in plain text than json is. And I doubt it would help in a situation like neutrinohu ran into. Personally, as a developer of course, I like json more than ini, but for the "normal user" I think ini is the better and more readable form of storing the data.
    Editing with phpmyadmin should not mess up things. I did that myself a lot of times without losing new line or other information. I have no idea why that could have happened in your case.
     
  12. Jesse Norell

    Jesse Norell Well-Known Member

    It's been a couple years ago, but I believe you can successfully use phpmyadmin if you click the link to edit the entire row in the servers table, but if you (double?) click an individual value it opens an inline editor, which is where the problem occurred.
     
  13. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Ah okay, never tried that. I'm a bit sceptic if that can successfully work for non-numeric (or alpha-numeric) values.
     
  14. Jesse Norell

    Jesse Norell Well-Known Member

    How about when reading this field, before ini parsing, another pass is made to normalize newlines to the expected format. Are there any fields within the (parsed) ini which include embedded newlines?

    This is of course assuming my memory is correct in what happened. :) I'll see if I can create a simple, reproducable failure scenario.
     
    Croydon likes this.
  15. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Not sure if that would work. The parser already replaces \r\n by \n and then splits by \n.
    So the failure could only occur on systems where \r\n or \n are replaces by \r (which is quite unusual). So it might be more probable that the newlines got lost completely and that could not be fixed by the parser.
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    We should consider migrating the server config data to the sys_ini table which is basically a key/value store where each record is stored in its own row and write an emulator to return the data in the same array format as we get it from the current storage field for server config and system config data.

    @Jesse Norell : Indeed, I remember the phpmyadmin inline edit issue as you mention it now, totally forgot that. was a real problem at that time and possibly it's still one of the sources of these problems.
     
    Jesse Norell likes this.
  17. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Agreed, we'll get that ready for one of the next releases :)
     
    neutrinohu and Jesse Norell like this.

Share This Page