Hi there, Since this morning i have a situation with my server. Every websites on the server i try to join display this error "DNS_PROBE_FINISHED_NXDOMAIN". I wasn't able to log in the server with ssh so i had to put the rescue mode on. Now i can log in the server with ssh but i don't know where to look. Is it just a problem or is it a hack ? I looked at the logs but i don't really know what to look. Any help would be appreciated. Regards, tonytroy.
So, i had a look at the logs, and i finally arrived in apache2 logs. As the server stop running correctly at 11h23 AM (my time), i look every clues around that time. In the file access.log, something changed : ::1 - - [27/Feb/2020:11:20:02 +0100] "GET / HTTP/1.1" 200 10975 "-" "Mozilla/5.0 (ISPConfig monitor)" 127.0.0.1 - - [27/Feb/2020:11:25:02 +0100] "GET / HTTP/1.1" 200 10975 "-" "Mozilla/5.0 (ISPConfig monitor)" The ::1 became 127.0.0.1 after the problem, i don't know if it's a clue.. In the file error.log, something changed too at exactly 11h23 : [ 2020-02-27 11:23:56.3152 1292/7fe172ad2980 age/Wat/WatchdogMain.cpp:1291 ]: Starting Passenger watchdog... [ 2020-02-27 11:23:56.3326 1300/7f98b43ee980 age/Cor/CoreMain.cpp:982 ]: Starting Passenger core... [ 2020-02-27 11:23:56.3327 1300/7f98b43ee980 age/Cor/CoreMain.cpp:235 ]: Passenger core running in multi-application mode. [ 2020-02-27 11:23:56.3707 1300/7f98b43ee980 age/Cor/CoreMain.cpp:732 ]: Passenger core online, PID 1300 [ 2020-02-27 11:23:56.3882 1315/7f6a602c1980 age/Ust/UstRouterMain.cpp:529 ]: Starting Passenger UstRouter... [ 2020-02-27 11:23:56.3909 1315/7f6a602c1980 age/Ust/UstRouterMain.cpp:342 ]: Passenger UstRouter online, PID 1315 [Thu Feb 27 11:23:56.400798 2020] [ssl:warn] [pid 1262] AH01906: ns.zenserveur.com:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) [Thu Feb 27 11:23:56.400849 2020] [ssl:warn] [pid 1262] AH01909: ns.zenserveur.com:8080:0 server certificate does NOT include an ID which matches the server name [Thu Feb 27 11:23:56.400949 2020] [ssl:error] [pid 1262] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: [email protected],CN=zenserver.dedibox.fr,OU=IT Department,O=TFSI,L=Marseille,ST=BdR,C=FR / issuer: [email protected],CN=zenserver.dedibox.fr,OU=IT Department,O=TFSI,L=Marseille,ST=BdR,C=FR / serial: 5BD85AB8E4CAE0F8EDAC4DBFB574C63370AA4C85 / notbefore: Oct 13 22:22:31 2019 GMT / notafter: Oct 10 22:22:31 2029 GMT] [Thu Feb 27 11:23:56.400957 2020] [ssl:error] [pid 1262] AH02604: Unable to configure certificate ns.zenserveur.com:8080:0 for stapling [Thu Feb 27 11:23:56.401011 2020] [suexec:notice] [pid 1262] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec) [ 2020-02-27 11:23:56.4040 1315/7f6a5bfff700 age/Ust/UstRouterMain.cpp:422 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown) [ 2020-02-27 11:23:56.4040 1300/7f98af7b4700 age/Cor/CoreMain.cpp:532 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown) [ 2020-02-27 11:23:56.4041 1315/7f6a602c1980 age/Ust/UstRouterMain.cpp:492 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected... [ 2020-02-27 11:23:56.4041 1300/7f98b43ee980 age/Cor/CoreMain.cpp:901 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected... [ 2020-02-27 11:23:56.4041 1315/7f6a5bfff700 Ser/Server.h:464 ]: [UstRouter] Shutdown finished [ 2020-02-27 11:23:56.4041 1315/7f6a5b7fe700 Ser/Server.h:817 ]: [UstRouterApiServer] Freed 0 spare client objects [ 2020-02-27 11:23:56.4042 1315/7f6a5b7fe700 Ser/Server.h:464 ]: [UstRouterApiServer] Shutdown finished [ 2020-02-27 11:23:56.4050 1300/7f9870ff9700 Ser/Server.h:817 ]: [ApiServer] Freed 0 spare client objects [ 2020-02-27 11:23:56.4050 1300/7f9870ff9700 Ser/Server.h:464 ]: [ApiServer] Shutdown finished [ 2020-02-27 11:23:56.4053 1300/7f98af7b4700 Ser/Server.h:817 ]: [ServerThr.1] Freed 128 spare client objects [ 2020-02-27 11:23:56.4053 1300/7f98af7b4700 Ser/Server.h:464 ]: [ServerThr.1] Shutdown finished [ 2020-02-27 11:23:56.4055 1300/7f98727fc700 Ser/Server.h:817 ]: [ServerThr.10] Freed 128 spare client objects [ 2020-02-27 11:23:56.4055 1300/7f98727fc700 Ser/Server.h:464 ]: [ServerThr.10] Shutdown finished [ 2020-02-27 11:23:56.4055 1300/7f98717fa700 Ser/Server.h:817 ]: [ServerThr.12] Freed 128 spare client objects [ 2020-02-27 11:23:56.4055 1300/7f9871ffb700 Ser/Server.h:817 ]: [ServerThr.11] Freed 128 spare client objects [ 2020-02-27 11:23:56.4055 1300/7f98717fa700 Ser/Server.h:464 ]: [ServerThr.12] Shutdown finished [ 2020-02-27 11:23:56.4055 1300/7f9871ffb700 Ser/Server.h:464 ]: [ServerThr.11] Shutdown finished [ 2020-02-27 11:23:56.4055 1300/7f98acd2a700 Ser/Server.h:817 ]: [ServerThr.6] Freed 128 spare client objects [ 2020-02-27 11:23:56.4055 1300/7f98acd2a700 Ser/Server.h:464 ]: [ServerThr.6] Shutdown finished [ 2020-02-27 11:23:56.4056 1300/7f98ae6b0700 Ser/Server.h:817 ]: [ServerThr.3] Freed 128 spare client objects [ 2020-02-27 11:23:56.4056 1300/7f9873fff700 Ser/Server.h:817 ]: [ServerThr.7] Freed 128 spare client objects [ 2020-02-27 11:23:56.4056 1300/7f98ae6b0700 Ser/Server.h:464 ]: [ServerThr.3] Shutdown finished [ 2020-02-27 11:23:56.4056 1300/7f9873fff700 Ser/Server.h:464 ]: [ServerThr.7] Shutdown finished [ 2020-02-27 11:23:56.4056 1300/7f98ad5ac700 Ser/Server.h:817 ]: [ServerThr.5] Freed 128 spare client objects [ 2020-02-27 11:23:56.4056 1300/7f98ad5ac700 Ser/Server.h:464 ]: [ServerThr.5] Shutdown finished [ 2020-02-27 11:23:56.4056 1315/7f6a602c1980 age/Ust/UstRouterMain.cpp:523 ]: Passenger UstRouter shutdown finished [ 2020-02-27 11:23:56.4057 1300/7f9872ffd700 Ser/Server.h:817 ]: [ServerThr.9] Freed 128 spare client objects [ 2020-02-27 11:23:56.4057 1300/7f9872ffd700 Ser/Server.h:464 ]: [ServerThr.9] Shutdown finished [ 2020-02-27 11:23:56.4057 1300/7f98aef32700 Ser/Server.h:817 ]: [ServerThr.2] Freed 128 spare client objects [ 2020-02-27 11:23:56.4057 1300/7f98aef32700 Ser/Server.h:464 ]: [ServerThr.2] Shutdown finished [ 2020-02-27 11:23:56.4057 1300/7f98737fe700 Ser/Server.h:817 ]: [ServerThr.8] Freed 128 spare client objects [ 2020-02-27 11:23:56.4057 1300/7f98ade2e700 Ser/Server.h:817 ]: [ServerThr.4] Freed 128 spare client objects [ 2020-02-27 11:23:56.4057 1300/7f98737fe700 Ser/Server.h:464 ]: [ServerThr.8] Shutdown finished [ 2020-02-27 11:23:56.4057 1300/7f98ade2e700 Ser/Server.h:464 ]: [ServerThr.4] Shutdown finished [ 2020-02-27 11:23:56.4637 1350/7fe9c0880980 age/Wat/WatchdogMain.cpp:1291 ]: Starting Passenger watchdog... [ 2020-02-27 11:23:56.4757 1354/7f2330e7f980 age/Cor/CoreMain.cpp:982 ]: Starting Passenger core... [ 2020-02-27 11:23:56.4758 1354/7f2330e7f980 age/Cor/CoreMain.cpp:235 ]: Passenger core running in multi-application mode. [ 2020-02-27 11:23:56.5151 1354/7f2330e7f980 age/Cor/CoreMain.cpp:732 ]: Passenger core online, PID 1354 [ 2020-02-27 11:23:56.5348 1368/7fcee0aa8980 age/Ust/UstRouterMain.cpp:529 ]: Starting Passenger UstRouter... [ 2020-02-27 11:23:56.5360 1368/7fcee0aa8980 age/Ust/UstRouterMain.cpp:342 ]: Passenger UstRouter online, PID 1368 [ 2020-02-27 11:23:56.5501 1300/7f98b43ee980 age/Cor/CoreMain.cpp:967 ]: Passenger core shutdown finished [Thu Feb 27 11:23:56.579584 2020] [ssl:warn] [pid 1347] AH01906: ns.zenserveur.com:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) [Thu Feb 27 11:23:56.579608 2020] [ssl:warn] [pid 1347] AH01909: ns.zenserveur.com:8080:0 server certificate does NOT include an ID which matches the server name [Thu Feb 27 11:23:56.579704 2020] [ssl:error] [pid 1347] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: [email protected],CN=zenserver.dedibox.fr,OU=IT Department,O=TFSI,L=Marseille,ST=BdR,C=FR / issuer: [email protected],CN=zenserver.dedibox.fr,OU=IT Department,O=TFSI,L=Marseille,ST=BdR,C=FR / serial: 5BD85AB8E4CAE0F8EDAC4DBFB574C63370AA4C85 / notbefore: Oct 13 22:22:31 2019 GMT / notafter: Oct 10 22:22:31 2029 GMT] [Thu Feb 27 11:23:56.579711 2020] [ssl:error] [pid 1347] AH02604: Unable to configure certificate ns.zenserveur.com:8080:0 for stapling [Thu Feb 27 11:23:56.581987 2020] [:error] [pid 1347] python_init: Python version mismatch, expected '2.7.5+', found '2.7.16'. [Thu Feb 27 11:23:56.583429 2020] [:error] [pid 1347] python_init: Python executable found '/usr/bin/python'. [Thu Feb 27 11:23:56.583443 2020] [:error] [pid 1347] python_init: Python path being used '/usr/lib/python2.7:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'. [Thu Feb 27 11:23:56.583467 2020] [:notice] [pid 1347] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads. [Thu Feb 27 11:23:56.583471 2020] [:notice] [pid 1347] mod_python: using mutex_directory /tmp [Thu Feb 27 11:23:56.603402 2020] [mpm_prefork:notice] [pid 1347] AH00163: Apache/2.4.38 (Debian) mod_fcgid/2.3.9 Phusion_Passenger/5.0.30 OpenSSL/1.1.1d mod_python/3.3.1 Python/2.7.16 configured -- resuming normal operations [Thu Feb 27 11:23:56.603458 2020] [core:notice] [pid 1347] AH00094: Command line: '/usr/sbin/apache2' I don't know if this is normal or not, but i wasn't able to find something about this in google. Do you see something in those logs that helps find the problem ? Or do you need other logs ? Regards, tonytroy.
An alternative solution (even if I would much prefer to repair the server) would be that I can at least recover the files and BDD's of the sites. For the files I can manage via the rescue mode, but for the BDD's I do not know how to do. Someone would have any idea ? I don't see how to dump in rescue mode ...
Hey Till, Yes I did, with [email protected]_address in normal mode i have no answer.. And in normal mode i can't join https://ip_address:8080 (for ISP config manager) nor http://ip_address Actually the server is not in normal mode but in rescue mode so i can join it with ssh.
What's weird is that i can access the server in ssh with kvm, but i can't access it with putty or with the terminal on my machine. I continue my investigation and if someone has an idea or lead, I am interested
