Create Lets Encrypt SSL Certs via Certbot DNS Validation in Acme v02

Discussion in 'Tips/Tricks/Mods' started by ahrasis, May 6, 2018.

  1. ahrasis

    ahrasis Active Member

    Since certbot in Ubuntu 16.04 is upgraded to version 22, it is now ready to use Acme v2. I believe ISPConfig developers are already working on this but everybody have to be patient since it may not be out in the near future. As I am currently using CloudFlare as my dns server, I would like to share some tips/tricks that I recently did on my ubuntu nginx webserver in order to issue a Let's Encrypt wildcard SSL certs for one of my domains.

    Firstly, other than installing the default certbot via "apt -y install python-certbot-nginx", I have to install cloudflare plugin for it too. This I did by running "apt -y install python3-certbot-dns-cloudflare python3-cloudflare". This plugin is essential for this tip/trick.

    Secondly, create a hidden folder accessible only by root user and file for the required credentials to be filled in.
    Code:
    mkdir /etc/letsencrypt/.secrets
    chown root:root /etc/letsencrypt/.secrets
    chmod 600 /etc/letsencrypt/.secrets
    # Create the credential file
    nano /etc/letsencrypt/.secrets/domain.tld.ini
    
    Thirdly, add the required credential inside the file. You obtained this from CloudFlare control panel for your domain.
    Code:
    # CloudFlare API key information
    dns_cloudflare_api_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    dns_cloudflare_email = [email protected]
    
    And lastly, run certbot using the cloudflare plugin for the wanted domain(s) using dns validation to issue your domain certs, including for its wildcard subdomain, if you want.
    Code:
    certbot certonly \
      --dns-cloudflare \
      --dns-cloudflare-credentials ~/etc/letsencrypt/.secrets/domain.tld.ini \
      --server https://acme-v02.api.letsencrypt.org/directory \
      --agree-tos \
      --dns-cloudflare-propagation-seconds 60 \
      --rsa-key-size 4096 \
      -d domain.tld \
      -d *.domain.tld
    
    This new certs will be defaulted to the same usual Let's Encrypt folder which you can manually use with ISPConfig. Its renewal file should look like this:
    Code:
    # renew_before_expiry = 30 days
    version = 0.22.0
    archive_dir = /etc/letsencrypt/archive/domain.tld
    cert = /etc/letsencrypt/live/domain.tld/cert.pem
    privkey = /etc/letsencrypt/live/domain.tld/privkey.pem
    chain = /etc/letsencrypt/live/domain.tld/chain.pem
    fullchain = /etc/letsencrypt/live/domain.tld/fullchain.pem
    
    # Options used in the renewal process
    [renewalparams]
    account = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    server = https://acme-v02.api.letsencrypt.org/directory
    authenticator = dns-cloudflare
    installer = None
    dns_cloudflare_credentials = /etc/letsencrypt/domain.tld.ini
    
    I think renewing via certbot renew should cover renewing this as well after the lapse of 60 days and before 90 days but I cannot confirm its renewal will work since mine is not subject to any renewal yet. So, please be attentive to friendly-reminding-email from Let's Encrypt just in case its renewal somehow failed after 60 days as you may need to do it manually.
     
    Last edited: Aug 24, 2018
  2. ahrasis

    ahrasis Active Member

    In the addition to the above, since I think many ISPConfig servers use Bind, we may use certbot dns_rfc2136 plugin in almost similar way as above.

    The idea is to firstly install Bind plugin and then create the TSIG base files (key and private) for the dns server, for examples Kdns.server.tld.+165+28266.key and Kdns.server.tld.+165+28266.private via the followings:
    Code:
    # Install certbot and its bind plugin, if you haven't.
    apt -y install certbot
    apt -y install python3-certbot-dns-rfc2136
    # Get into bind directory and issue two TSIG base files.
    cd /etc/bind
    dnssec-keygen -a HMAC-SHA512 -b 512 -n HOST $(hostname -f).
    
    Secondly, create the TSIG credential file for the dns server as follows:
    Code:
    # Define the intended domain accordingly.
    domain4le=test.domain.tld <-- change this to your domain
    dnsserverip=192.168.0.101 <-- change this to your dns server ip
    # We now create tsig key credential file.
    Kfile=$(ls K$(hostname -f).*.key)
    Secret=$(cat $Kfile | awk '{print $7, $8}')
    cat <<EOF > tsig.$(hostname -f).key
    key "$(hostname -f)." {
      algorithm HMAC-SHA512;
      secret "$Secret";
    };
    
    zone "$domain4le." IN {
      type master;
      file "pri.$domain4le";
      update-policy {
       grant $(hostname -f). name _acme-challenge.$domain4le. txt;
      };
    };
    EOF
    
    Thirdly, add the above credential file at the end of the targeted domain bind file pri.test.domain.tld in /etc/bind as follows:
    Code:
    # Add it in the domain dns / bind file.
    cat <<EOF >> pri.$domain4le
    
    \$INCLUDE tsig.$(hostname -f).key
    EOF
    
    Fourthly, make sure you are created the required secrets folder to save the secrets as follows:
    Code:
    # Create and protect .secrets folder
    if [ -d "/etc/letsencrypt/.secrets" ]; then
      mkdir /etc/letsencrypt/.secrets
    fi
    chown root:root /etc/letsencrypt/.secrets
    chmod 600 /etc/letsencrypt/.secrets
    
    Fithfly, enter the secrets folder and use the ealier created TSIG credentials to fill the ini file as follows:
    Code:
    # Enter the .secrets folder & create the credential file.
    cd /etc/letsencrypt/.secrets
    cat <<EOF > $domain4le.ini
    # Target ISPConfig DNS server
    dns_rfc2136_server = $dnsserverip
    # Target ISPConfig DNS port
    dns_rfc2136_port = 53
    # TSIG key name
    dns_rfc2136_name = $(hostname -f).
    # TSIG key secret
    dns_rfc2136_secret = $Secret
    # TSIG key algorithm
    dns_rfc2136_algorithm = HMAC-SHA512
    EOF
    
    Lastly, issue LE SSL certs via the following command:
    Code:
    certbot certonly \
      --dns-rfc2136 \
      --dns-rfc2136-credentials ~/etc/letsencrypt/.secrets/domain.tld.ini \
      --server https://acme-v02.api.letsencrypt.org/directory \
      --agree-tos \
      --dns-rfc2136-propagation-seconds 60 \
      --rsa-key-size 4096 \
      -d domain.tld \
      -d *.domain.tld
    

    It will be nice if I can write at least this up in a proper ISPConfig way, so that it can be contributed to the git but my knowledge is quite limited. :( Hopefully in ISPConfig version 3.2 we will see an option to obtain Let's Encrypt SSL certs via dns to certain extends.

    Edited: The above code for both ISPConfig and CloudFlare is now ready to be rewritten in ISPConfig. ;)
     
    Last edited: Aug 24, 2018
    ztk.me likes this.
  3. ahrasis

    ahrasis Active Member

    Now I can confirm that the renewal of my domain and its wildcard via cloudflare dns is working.

    For those who are using the same method but got error in renewal for the domain and its wildcard, they should check the CAA entry which basically should consist of two CAA as follows:
    Code:
    CAA    domain.tld    0    issue "letsencrypt.org"    automatic
    CAA    domain.tld    0    issuewild"letsencrypt.org"    automatic
    The first is for the domain and the second is for its wildcard. That means if you want to create certs for sub.domain.tld only, you will need to create a CAA entry just for it. You can check you CAA entry via command "dig caa domain.tld" or via online service like at https://dnsspy.io/labs/caa-validator.

    I think I can now try to write the this modification for letsencrypt in ISPConfig related files.
     
    till likes this.
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Please consider setting permissions of the .secrets directory explicitly to a secure value. E.g.:

    Code:
    chown root:root /etc/letsencrypt/.secrets
    chmod 600 /etc/letsencrypt/.secrets
     
    ahrasis likes this.
  5. ahrasis

    ahrasis Active Member

    I just updated post #2 above with @till suggestions and with some extra, and I think they are now ready to be re-written for ISPConfig. Hopefully I am free to write them soon.
     
  6. ahrasis

    ahrasis Active Member

    1. The following script is self-explanatory for the purpose of testing whether the code can be automatically run from ISPConfig Perfect Server (single server setup with dns capability) before expanding the same to be used in a multi server setup.

    2. This script should be run before issuing certbot certonly command as described in post #2 above.

    3. It is still a work in progress because at the end they will partly be converted to php code plus kept in database and files when it's (hopefully) ready.

    4. Those who are interested may test this script on their test server at their own risk.
    Code:
    # 0 - Install certbot and its bind plugin, if you haven't.
    apt -y install certbot
    apt -y install python3-certbot-dns-rfc2136
    
    # 1 - Define domain, tsig and ip.
    domain=test.domain.tld
    tsig=$(hostname -f)
    dnsserverip=192.168.0.101
    
    
    # 2 - Get to bind directory & issue 2 TSIG base files.
    cd /etc/bind
    dnssec-keygen -a HMAC-SHA512 -b 512 -n HOST $tsig.
    
    
    # 3 - We now create tsig key file.
    kfile=$(ls K$tsig.*.key)
    secret=$(cat $kfile | awk '{print $7, $8}')
    conf=named.conf.local
    local=$(cat $conf)
    cat <<EOF > $conf
    key "$tsig." {
      algorithm hmac-sha512;
      secret "$secret";
    };
    $local
    EOF
    sed -i "s/file \"\/etc\/bind\/pri.$domain\";/allow-update {key $tsig.;};\n\tfile \"\/etc\/bind\/pri.$domain\";/g" $conf
    
    # Add "key dns.server.tld." in Update ACL to make key permanent.
    # Check its content if you want.
    # cat $conf
    
    
    # 4 - Add it in the domain dns / bind file.
    cat <<EOF >> pri.$domain
    
    \$INCLUDE /etc/bind/$kfile
    EOF
    
    # Again, check its content if you want.
    # cat pri.$domain
    
    
    # 5 - Create and protect .secrets folder
    sdir=/usr/local/ispconfig/interface/ssl/.secrets
    if [ ! -d "$sdir" ]; then
       mkdir $sdir
    fi
    chown root:root $sdir
    chmod 600 $sdir
    
    # 6 - Enter .secrets folder, create credential & chmod it 600
    cd $sdir
    cat <<EOF > $domain.ini
    # Target ISPConfig DNS server
    dns_rfc2136_server = $dnsserverip
    # Target ISPConfig DNS port
    dns_rfc2136_port = 53
    # TSIG key name
    dns_rfc2136_name = $tsig.
    # TSIG key secret
    dns_rfc2136_secret = $secret
    # TSIG key algorithm
    dns_rfc2136_algorithm = HMAC-SHA512
    EOF
    chmod 600 $domain.ini
    
    # Again, check its content if you want.
    # cat $domain.ini
    # cd /etc/bind
    # cat $conf
    # cat pri.$domain
    
    # 7 - Restart bind9 service
    service bind9 restart
    
     
    Last edited: Aug 27, 2018
    till likes this.
  7. ahrasis

    ahrasis Active Member

  8. ahrasis

    ahrasis Active Member

    All test using certbot dns-rfc2136 (bind) plugin end up with: "Unable to determine base domain for _acme-challenge.test.domain.tld using names: ['_acme-challenge.test.domain.tld', 'test.domain.tld', 'domain.tld', 'tld']". I am using the latest version but this seems to be a reported bug which is not clear whether it was or will be fixed, so I'll drop any attempt to use this plugin for the time being.

    The good news is I managed to simplify the process without using any plugin for issuing a dns validated LE ssl certs. This following code been tested working great for a single domain without any subdomain in a single ISCPConfig server setup, using certbot without any plugin.

    1. Run nano /etc/bind/auth.sh to create it and add the following bash code.
    Code:
    #!/bin/bash
    
    letoken=$CERTBOT_VALIDATION
    pridomain=pri.$domain
    pricatch=$(cat $pridomain)
    cat <<EOF > $pridomain
    $pricatch
    _acme-challenge.$domain. 60     TXT     "$letoken"
    EOF
    service bind9 restart
    sleep 10
    
    2. Run nano /etc/bind/clean.sh to create it and add the following bash code.
    Code:
    #!/bin/bash
    
    sed -i '/acme-challenge/d' pri.$domain
    sed -i '3d' auth.sh
    sed -i '3d' clean.sh
    service bind9 restart
    
    3. Make them both executable.
    Code:
    chmod +x /etc/bind/auth.sh && chmod +x /etc/bind/clean.sh
    4. Run the following code after changing test.domain.tld to your own domain.
    Code:
    cd /etc/bind
    domain=test.domain.tld
    sed -i "s/letoken=/domain=$domain\nletoken=/g" auth.sh
    sed -i -e "3idomain=$domain" clean.sh
    5. Issue LE ssl certs via the following commands:
    Code:
    certbot certonly \
      --preferred-challenges dns \
      --manual \
      --manual-public-ip-logging-ok \
      --manual-auth-hook /etc/bind/auth.sh \
      --manual-cleanup-hook /etc/bind/clean.sh \
      --server https://acme-v02.api.letsencrypt.org/directory \
      --agree-tos \
      --rsa-key-size 4096 \
      -d $domain
    
    I believe it is a matter of passing some arguments via ssh / plugins to the ISPCOnfig main dns sever from any other server in a multi server setup to achieve the same result. I'll try to venture into this later when I have a little bit more of free time.
     
    till likes this.
  9. ahrasis

    ahrasis Active Member

    I am able to extend the above hook to issue LE ssl certs to a remote server in a ISPConfig multiserver setup. For this extension to successfully works for you, the remote server must be given an access to the ISPConfig dns (bind) server via ssh keyless login in order to remotely run certain commands. The steps for it must be done in the remote server as follows:

    1. In the remote server, create /etc/bind/auth.sh file with the following code:
    Code:
    #!/bin/bash
    
    cat <<EOF > letoken
    $CERTBOT_VALIDATION
    EOF
    scp letoken [email protected]:/etc/bind/
    ssh [email protected] 'cd /etc/bind; CERTBOT_VALIDATION=$(cat letoken); . auth.sh'
    
    2. Then, at the same server, create /etc/bind/clean.sh as follows:
    Code:
    #!/bin/bash
    
    ssh [email protected]'cd /etc/bind; . clean.sh; rm letoken'
    
    3. Specify the domain that you intend to get its LE ssl certs and run ssh remote command to change the default auth.sh and clean.sh similar to post #9 above as follows:
    Code:
    ssh [email protected] 'cd /etc/bind; domain=test.domain.tld; sed -i "s/letoken=/domain=$domain\nletoken=/g" auth.sh; sed -i -e "3idomain=$domain" clean.sh
    4. Finally, issue a certbot command to request LE ssl certs for the targeted domain:
    Code:
    certbot certonly \
      --preferred-challenges dns \
      --manual \
      --manual-public-ip-logging-ok \
      --manual-auth-hook /etc/bind/auth.sh \
      --manual-cleanup-hook /etc/bind/clean.sh \
      --server https://acme-v02.api.letsencrypt.org/directory \
      --agree-tos \
      --rsa-key-size 4096 \
      -d $domain
    
    If nothing goes wrong, you should be getting your desired LE ssl certs to your remote server for whatever purpose you wish to use it for, whether for email , mysql, even another dns or web server.

    This above trick is basically grabbing certbot token for validation and pass it to dns server temporarily, that is until the LE verificationfor TXT for the targeted domain is finished; and afterwards, it will be removed.

    Please note that for this dns LE ssl certs to get properly renewed, you must make sure that command for renewal (certbot renew --dry-run) is setup as a cron job and run at least once in 24 hours especially for server(s) other than the ISPConfig web server.

    Hopefully, when I am done re-writing this in php for ISPConfig, there is no need for you to do so manually as all will be done by ISPConfig automatically.
     

Share This Page