Impossibile to issue LE certificate during ISPConfig update

Discussion in 'Installation/Configuration' started by Aleeekoi, Jun 8, 2021.

  1. Aleeekoi

    Aleeekoi New Member

    Hello,
    I have already read other forum post but I can't find a solution.
    Today I have update ISPConfig using the standard command

    Code:
    ispconfig_update.sh update
    I have reloaded all services and say Yes to the certificate recreation but i get the following error:

    Code:
    Create new ISPConfig SSL certificate (yes,no) [no]: yes
    
    Checking / creating certificate for ***.net
    Using certificate path /etc/letsencrypt/live/***.net
    Using apache for certificate validation
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Plugins selected: Authenticator webroot, Installer None
    Obtaining a new certificate
    Performing the following challenges:
    http-01 challenge for ***.net
    Using the webroot path /usr/local/ispconfig/interface/acme for all unmatched domains.
    Waiting for verification...
    Challenge failed for domain ***.net
    http-01 challenge for ***.net
    Cleaning up challenges
    Some challenges have failed.
    Issuing certificate via certbot failed. Please check log files and make sure that your hostname can be verified by letsencrypt
    Could not issue letsencrypt certificate, falling back to self-signed.
    
    From LE logs:
    Code:
    Domain: ***.net
    Type:   connection
    Detail: Fetching http://***.net/.well-known/acme-challenge/L4qJU3t5F_j2G52WJPAo2QC5Wi7ooe5TjU4YKqkNwE0: Connection r>
    
    To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) >
    2021-06-08 10:51:25,750:DEBUG:certbot.error_handler:Encountered exception:
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 91, in handle_authorizations
        self._poll_authorizations(authzrs, max_retries, best_effort)
      File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 180, in _poll_authorizations
        raise errors.AuthorizationError('Some challenges have failed.')
    certbot.errors.AuthorizationError: Some challenges have failed.
    
    It seem that LE is not able to access the web server, the DNS is correct pointing with an A record the server IP. It's strange because if I try access the domain with a browser, during the isp configuration process, the page is unavailable.

    Whats the problem?

    Thank you
     
  2. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    If browser can not access the website neither can Let's Encrypt, and the certification fails.
    If site has subdomain www included in certificate then that also must work.
    There is this:
    https://www.howtoforge.com/community/threads/lets-encrypt-error-faq.74179/
     
  3. Aleeekoi

    Aleeekoi New Member

    It's not a website it is the mail host domain url.
    I have made many retry but every time ispconfig_update.sh script try to generate a new LE certificate apache become unavailable.
    It could be a problem related to an old self signed cert and the passphrase/challenge password?
    Thank you
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Please post the output of these commands:

    ls -la /root/.acme*
    ls /etc/letsencrypt
    which certbot
    which letsencrypt
    certbot --version
     
  5. Aleeekoi

    Aleeekoi New Member

    ls -la /root/.acme*
    Code:
    ls: cannot access '/root/.acme*': No such file or directory
    
    ls /etc/letsencrypt
    Code:
    accounts  archive  cli.ini  csr  keys  live  renewal  renewal-hooks
    
    which certbot
    Code:
    /usr/bin/certbot
    
    which letsencrypt
    Code:
    /usr/bin/letsencrypt
    
    certbot --version
    Code:
    certbot 0.40.0
    
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Ok, then please post the output of:

    ls -la /usr/bin/certbot
    ls -la /usr/bin/letsencrypt

    And which operating system do you use?
     
  7. Aleeekoi

    Aleeekoi New Member

    I'm using Ubuntu 20.04.1 LTS

    ls -la /usr/bin/certbot:
    Code:
    -rwxr-xr-x 1 root root 385 Oct 26  2020 /usr/bin/certbot
    
    ls -la /usr/bin/letsencrypt:
    Code:
    lrwxrwxrwx 1 root root 7 Oct 26  2020 /usr/bin/letsencrypt -> certbot
    
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    The certbot version is quite old, maybe you should consider installing a newer version. One way would be:

    sudo apt-get remove certbot
    sudo snap install core; sudo snap refresh core
    sudo snap install --classic certbot
    sudo ln -s /snap/bin/certbot /usr/bin/certbot
    sudo rm -f /usr/bin/letsencrypt

    and then to test the new certbot:

    sudo certbot --version
     
  9. Aleeekoi

    Aleeekoi New Member

    ok certbot is now v1.16.0
    but nothing changes, I cant issue a new certificate from ispconfig_update.sh script

    do I need to create a site in my ispconfig panel pointing to the host.example.com ?

    Thank you
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    Strange, and the web server is still down after the update?

    No, this would even prevent cert creation.
     
  11. Aleeekoi

    Aleeekoi New Member

    Nope, it is online. Other website LE certs work fine (also trying dryrun works).
    It's only the main host cert the problem, now I have the FPT and Mail client working on the self generated cert and I'm getting a lot of warnings. Also connection to the panel on the port 8080 give me warning about the self signed cert.

    It's strange because when the ispconfig_update.sh script run the host cert generation all website become unavailable, like apache stopped.

    Any other hint? :(

    Thank you
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    Please post the output of:

    ls -la /usr/local/ispconfig/interface/ssl/
     
  13. Aleeekoi

    Aleeekoi New Member

    ok, here it is:
    Code:
    total 36
    drwxr-x--- 2 root      root      4096 Jun  8 13:31 .
    drwxr-x--- 9 ispconfig ispconfig 4096 Jun  8 12:55 ..
    -rwxr-x--- 1 root      root        45 Jun  8 13:31 empty.dir
    -rwxr-x--- 1 root      root      1887 Jun  8 13:31 ispserver.crt
    -rwxr-x--- 1 root      root      1655 Jun  8 13:31 ispserver.csr
    -rwxr-x--- 1 root      root      3243 Jun  8 13:31 ispserver.key
    -rwxr-x--- 1 root      root      3311 Jun  8 13:30 ispserver.key.secure
    -rwxr-x--- 1 root      root      5130 Jun  8 13:31 ispserver.pem
    
     
  14. Aleeekoi

    Aleeekoi New Member

    Hi Till, problem still persist.. any hint? :( Thank you
     
  15. till

    till Super Moderator Staff Member ISPConfig Developer

    That's ok so far, try this:

    touch /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/test.txt

    Then try to access the test token from the internet:

    http://***.net/.well-known/acme-challenge/test.txt

    The domain name (subdomain) must be the same that is mentioned in the lets encrypt log in your first post.
     
  16. Aleeekoi

    Aleeekoi New Member

    Yes, it works.

    The problem is that the ispconfig .sh script when starts to update the certificate the web server crash and LE is not able to verify the request.

    Never happened? Thank you
     
  17. Aleeekoi

    Aleeekoi New Member

    Here some more logs:
    letsencrypt.log
    Code:
    2021-06-09 15:45:11,165:DEBUG:certbot.display.util:Notifying user:
    Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
      Domain: ***.net
      Type:   connection
      Detail: Fetching http:// ***.net/.well-known/acme-challenge/_-eMcivpEVilbwyuNuN6oHp4TV-CRNzFiUEqZRdI2g4: Connection refused
    
    Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded f>
    
    2021-06-09 15:45:11,166:DEBUG:certbot._internal.error_handler:Encountered exception:
    Traceback (most recent call last):
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 93, in handle_authorizations
        self._poll_authorizations(authzrs, max_retries, best_effort)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 181, in _poll_authorizations
        raise errors.AuthorizationError('Some challenges have failed.')
    certbot.errors.AuthorizationError: Some challenges have failed.
    
    2021-06-09 15:45:11,166:DEBUG:certbot._internal.error_handler:Calling registered functions
    2021-06-09 15:45:11,166:INFO:certbot._internal.auth_handler:Cleaning up challenges
    2021-06-09 15:45:11,166:DEBUG:certbot._internal.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/_-eMcivpEVilbwyuNuN6oHp4TV-CRNzFiUEqZRdI2g4
    2021-06-09 15:45:11,167:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
    2021-06-09 15:45:11,167:DEBUG:certbot._internal.log:Exiting abnormally:
    Traceback (most recent call last):
      File "/snap/certbot/1201/bin/certbot", line 8, in <module>
        sys.exit(main())
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/main.py", line 15, in main
        return internal_main.main(cli_args)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/main.py", line 1552, in main
        return config.func(config, plugins)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/main.py", line 1414, in certonly
        lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/main.py", line 128, in _get_and_save_cert
        lineage = le_client.obtain_and_enroll_certificate(domains, certname)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/client.py", line 445, in obtain_and_enroll_certificate
        cert, chain, key, _ = self.obtain_certificate(domains)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/client.py", line 375, in obtain_certificate
        orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/client.py", line 425, in _get_order_and_authorizations
        authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 93, in handle_authorizations
        self._poll_authorizations(authzrs, max_retries, best_effort)
      File "/snap/certbot/1201/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 181, in _poll_authorizations
        raise errors.AuthorizationError('Some challenges have failed.')
    certbot.errors.AuthorizationError: Some challenges have failed.
    2021-06-09 15:45:11,171:ERROR:certbot._internal.log:Some challenges have failed.
    
    apache.log
    Code:
    AH00526: Syntax error on line 20 of /etc/apache2/sites-enabled/000-apps.vhost:
    SSLCertificateFile: file '/usr/local/ispconfig/interface/ssl/ispserver.crt' does not exist or is empty
    [Wed Jun 09 15:57:35.312075 2021] [ssl:warn] [pid 258080] AH01906: ***.net:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 15:57:35.318245 2021] [ssl:warn] [pid 258080] AH01909: ***.net:8080:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 15:57:35.318398 2021] [ssl:error] [pid 258080] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 4AC7603557A6DE8C130B00D84>
    [Wed Jun 09 15:57:35.318408 2021] [ssl:error] [pid 258080] AH02604: Unable to configure certificate ***.net:8080:0 for stapling
    [Wed Jun 09 15:57:35.318717 2021] [ssl:warn] [pid 258080] AH01906: ***.net:8081:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 15:57:35.318724 2021] [ssl:warn] [pid 258080] AH01909: ***.net:8081:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 15:57:35.318794 2021] [ssl:error] [pid 258080] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 4AC7603557A6DE8C130B00D84>
    [Wed Jun 09 15:57:35.318800 2021] [ssl:error] [pid 258080] AH02604: Unable to configure certificate ***.net:8081:0 for stapling
    [Wed Jun 09 15:57:35.318869 2021] [suexec:notice] [pid 258080] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
    [Wed Jun 09 15:57:36.348523 2021] [ssl:warn] [pid 258085] AH01906: ***.net:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 15:57:36.348570 2021] [ssl:warn] [pid 258085] AH01909: ***.net:8080:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 15:57:36.350437 2021] [ssl:error] [pid 258085] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 4AC7603557A6DE8C130B00D84>
    [Wed Jun 09 15:57:36.350460 2021] [ssl:error] [pid 258085] AH02604: Unable to configure certificate ***.net:8080:0 for stapling
    [Wed Jun 09 15:57:36.353663 2021] [ssl:warn] [pid 258085] AH01906: ***.net:8081:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 15:57:36.353689 2021] [ssl:warn] [pid 258085] AH01909: ***.net:8081:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 15:57:36.353807 2021] [ssl:error] [pid 258085] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 4AC7603557A6DE8C130B00D84>
    [Wed Jun 09 15:57:36.353817 2021] [ssl:error] [pid 258085] AH02604: Unable to configure certificate ***.net:8081:0 for stapling
    [Wed Jun 09 15:57:36.368328 2021] [:error] [pid 258085] python_init: Python version mismatch, expected '2.7.17', found '2.7.18'.
    [Wed Jun 09 15:57:36.371146 2021] [:error] [pid 258085] python_init: Python executable found ''.
    [Wed Jun 09 15:57:36.371167 2021] [:error] [pid 258085] python_init: Python path being used '/lib/python2.7:/lib/python2.7/plat-x86_64-linux-gnu:/lib/python2.7/lib-tk:/lib/python2.7/lib-old:/lib/python2.7/lib-dynload'.
    [Wed Jun 09 15:57:36.371200 2021] [:notice] [pid 258085] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
    [Wed Jun 09 15:57:36.371207 2021] [:notice] [pid 258085] mod_python: using mutex_directory /tmp
    [Wed Jun 09 15:57:36.502139 2021] [core:warn] [pid 258085] AH00098: pid file /var/run/apache2/apache2.pid overwritten -- Unclean shutdown of previous Apache run?
    [Wed Jun 09 15:57:36.628717 2021] [mpm_prefork:notice] [pid 258085] AH00163: Apache/2.4.41 (Ubuntu) mod_fcgid/2.3.9 OpenSSL/1.1.1f mod_python/3.3.1 Python/2.7.18 configured -- resuming normal operations
    [Wed Jun 09 15:57:36.628755 2021] [core:notice] [pid 258085] AH00094: Command line: '/usr/sbin/apache2'
    [Wed Jun 09 16:36:26.100279 2021] [mpm_prefork:notice] [pid 258085] AH00171: Graceful restart requested, doing restart
    AH00526: Syntax error on line 20 of /etc/apache2/sites-enabled/000-apps.vhost:
    SSLCertificateFile: file '/usr/local/ispconfig/interface/ssl/ispserver.crt' does not exist or is empty
    [Wed Jun 09 16:37:22.894386 2021] [ssl:warn] [pid 262498] AH01906: ***.net:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 16:37:22.895236 2021] [ssl:warn] [pid 262498] AH01909: ***.net:8080:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 16:37:22.895392 2021] [ssl:error] [pid 262498] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 3BA4B1D07137326AF11ACE67B>
    [Wed Jun 09 16:37:22.895400 2021] [ssl:error] [pid 262498] AH02604: Unable to configure certificate ***.net:8080:0 for stapling
    [Wed Jun 09 16:37:22.895674 2021] [ssl:warn] [pid 262498] AH01906: ***.net:8081:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 16:37:22.895680 2021] [ssl:warn] [pid 262498] AH01909: ***.net:8081:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 16:37:22.895747 2021] [ssl:error] [pid 262498] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 3BA4B1D07137326AF11ACE67B>
    [Wed Jun 09 16:37:22.895752 2021] [ssl:error] [pid 262498] AH02604: Unable to configure certificate ***.net:8081:0 for stapling
    [Wed Jun 09 16:37:22.895817 2021] [suexec:notice] [pid 262498] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
    SIGTERM handler "exitall" not defined.
    [Wed Jun 09 16:37:23.751818 2021] [ssl:warn] [pid 262503] AH01906: ***.net:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 16:37:23.751855 2021] [ssl:warn] [pid 262503] AH01909: ***.net:8080:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 16:37:23.751933 2021] [ssl:error] [pid 262503] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 3BA4B1D07137326AF11ACE67B>
    [Wed Jun 09 16:37:23.751940 2021] [ssl:error] [pid 262503] AH02604: Unable to configure certificate ***.net:8080:0 for stapling
    [Wed Jun 09 16:37:23.785938 2021] [ssl:warn] [pid 262503] AH01906: ***.net:8081:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
    [Wed Jun 09 16:37:23.785974 2021] [ssl:warn] [pid 262503] AH01909: ***.net:8081:0 server certificate does NOT include an ID which matches the server name
    [Wed Jun 09 16:37:23.786095 2021] [ssl:error] [pid 262503] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: O=ABC,ST=Italy,C=IT / issuer: O=ABC,ST=Italy,C=IT / serial: 3BA4B1D07137326AF11ACE67B>
    [Wed Jun 09 16:37:23.786102 2021] [ssl:error] [pid 262503] AH02604: Unable to configure certificate ***.net:8081:0 for stapling
    [Wed Jun 09 16:37:23.802995 2021] [:error] [pid 262503] python_init: Python version mismatch, expected '2.7.17', found '2.7.18'.
    [Wed Jun 09 16:37:23.806360 2021] [:error] [pid 262503] python_init: Python executable found ''.
    [Wed Jun 09 16:37:23.806383 2021] [:error] [pid 262503] python_init: Python path being used '/lib/python2.7:/lib/python2.7/plat-x86_64-linux-gnu:/lib/python2.7/lib-tk:/lib/python2.7/lib-old:/lib/python2.7/lib-dynload'.
    [Wed Jun 09 16:37:23.806413 2021] [:notice] [pid 262503] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
    [Wed Jun 09 16:37:23.806430 2021] [:notice] [pid 262503] mod_python: using mutex_directory /tmp
    [Wed Jun 09 16:37:23.944287 2021] [core:warn] [pid 262503] AH00098: pid file /var/run/apache2/apache2.pid overwritten -- Unclean shutdown of previous Apache run?
    [Wed Jun 09 16:37:24.132481 2021] [mpm_prefork:notice] [pid 262503] AH00163: Apache/2.4.41 (Ubuntu) mod_fcgid/2.3.9 OpenSSL/1.1.1f mod_python/3.3.1 Python/2.7.18 configured -- resuming normal operations
    [Wed Jun 09 16:37:24.132536 2021] [core:notice] [pid 262503] AH00094: Command line: '/usr/sbin/apache2'
    
    I think during the issue of LE apache is restarted and fail cause of missing ispserver.crt defined in 000-apps.vhost
    After generating manually the self signed cert and completing the update procedure apache starts without problem anche the file ispserver.crt is not missing anymore.

    What is the 000.apps.vhost used for? Can I try to remove it from sites-available and retry to update ispconfig?
     
    Last edited: Jun 9, 2021
  18. till

    till Super Moderator Staff Member ISPConfig Developer

    The apps vhost is used for webmail and phpmyadmin, mostly required on nginx systems. you can try to remove the symlink in sites-enabled directory and try update again.
     
  19. Aleeekoi

    Aleeekoi New Member

    Finally I found the problem!
    Probably in one old installation/update of my ispconfig LE doesn't work and a self signed cert was generated.
    So, the cert file was linked in 000-apps.vhost and 000-ispconfig.vhost blocking next LE issues.
    I have resolved like this:
    1. remove 000-apps.vhost and 000-ispconfig.vhost links from sites-enabled dir
    2. run "ispconfig_update.sh --force" saying "no" to the question about re-initializing services
    3. at the end of the update, recreate the link of 000-apps.vhost and 000-ispconfig.vhost in sites-enabled dir
    4. run again "ispconfig_update.sh --force" but this time reinitialize services (so they will be restarted at the end an the new LE certs become available)
    Enjoy :cool:
     

Share This Page