How to use DuckDNS for multiple letsencrypt DNS challenges (normal + wildcard cert)

Question

The tools

Let's encrypt is the source of nearly all SSL/TLS certificates for HTTPS at the hobbyist level, offering automatic issuance and renewal of certificates, using challenges offered over HTTP or DNS.

In particular, a website must pass a DNS challenge to be issued a wildcard certificate for a domain of the form *.example.org, by setting a TXT record of the domain (or of the domain's CNAME, which Letsencrypt respects) in question to a specific value. Control of example.org. and of *.example.org. are both verified by checking the TXT record at _acme-challenge.example.org..

Automatic renewal of Letsencrypt certificates is typically handled by Certbot.

DuckDNS, while not as universal, is a popular free dynamic DNS provider. Accounts may register domains of the form example.duckdns.org. and update their A/AAAA records by means of a simple HTTP API. Often, it is used alongside a conventional DNS provider, by pointing (say) www.example.org. via CNAME at example.duckdns.org..

Additionally, the same API lets users set or clear a TXT record for their domain, specifically for interoperability with letsencrypt. (All A/AAAA/TXT records set for example.duckdns.org. are mirrored to *.example.duckdns.org..)

The goal

The goal is to use a reasonably standard setup of Letsencrypt/Certbot to pass DNS challenges using the DuckDNS API.

In particular, we want a certificate for both example.org and *.example.org, where we control example.duckdns.org. and the ordinary DNS provider serves *.example.org. CNAME example.duckdns.org..

If we have multiple front-end domains all pointed at the same back-end webserver, we may even want to handle renewals for all four of example.org, *.example.org, example.com, *.example.com.

The naive solution

To use the DuckDNS API per their spec, we can put this line in a simple shell script (after renaming the appropriate environment variables, and hardcoding the DuckDNS domain & auth-token)

V1='example.duckdns.org'
V2='our-duckdns-auth-token'
V3="$CERTBOT_VALIDATION"
curl -s -S "https://www.duckdns.org/update?domains=${V1}&token=${V2}&txt=${V3}"

and ask Certbot to run it using the following command

certbot certonly -v --manual --manual-auth-hook my-script.sh -d '*.example.org.'

after which renewal will be handled automatically.

The problem

DuckDNS only holds one TXT record for a domain at a time, with any additional records set overwriting the old one. At least using the default Certbot, Letsencrypt asks the client requesting the certificate to pass all challenges at once.

That is, while the naive solution above will work for *.example.org., if we extend it to

-d 'example.org' -d '*.example.org'

then Letsencrypt will run two challenges, one for each domain, and expect two different TXT records to be simultaneously present at _acme-challenge.example.org.. Certbot will run the hooked script twice, with the second TXT record added overwriting the first, leaving just one. At least one challenge will therefore fail and the full certificate can't be issued. In the four-domain extended version of the problem, there would be one success and three failures.

Since Letsencrypt is ubiquitous and DuckDNS is reasonably popular, there are scattered references [1] [2] [3] to this problem on the internet (including some more in the non-public DuckDNS google groups) but nothing that I can find on any Stack exchange sites.

Answer 1

Compiled below are some solutions I've found.

Mixed challenges

The DNS challenge is only strictly necessary for the wildcard certificate. We can ask Certbot to use HTTP challenges where available using --preferred-challenges.

certbot certonly -v --manual          \
 --preferred-challenges 'http,dns'    \
 --manual-auth-hook my-script.sh      \
 -d 'example.org.' -d '*.example.org.';

This requires a --manual-auth-hook script which can set up either challenge. Again consulting the documentation, this has roughly the form of

if [ -z "$CERTBOT_TOKEN" ]; then
    # DNS auth
    # See original question
else
    # HTTP auth
    # echo "$CERTBOT_VALIDATION" > "/the/correct/dir/$CERTBOT_TOKEN"
fi;

plus corresponding cleanup logic. This will allow example.org and *.example.org to be smoothly verified in parallel. It's also the method I've settled for, on account of it being the cleanest.

For comparison,

• It's basically correct usage of all the tools involved.
• The setup is slightly more complicated.
• It does not handle the (more general) four-domain .org & .com case in the question, since the two different wildcard challenges will still collide.

The example.org. and example.com. domains could be pointed to separate DuckDNS domains, like example0.duckdns.org. and example1.duckdns.org., though a free DuckDNS account is limited to five domains at a time. So long as there's only one wildcard cert, arbitrary domain combinations (say, *.example.org, example.org, example.com, www.example.com) work fine.

Brute forced serial challenges

Certbot will always try to run all challenges in parallel, but whenever a challenge for one domain succeeds, the Certbot client that passed it will remain authorized to redeem certificates for that domain for thirty days.

That is, while the naive solution of

certbot certonly -v --manual --manual-auth-hook my-script.sh \
 -d 'example.org' -d '*.example.org' ;

will fail the first time it's executed, it will succeed the second time, since control of example.org is already verified and *.example.org will encounter no collisions.

Renewals should work similarly. A typical Certbot installation will check daily for certificates that approach expiry, and attempt to renew any with less than 30 (out of the total 90) days left. The first attempted renewal of this certificate would fail, which means Certbot would try again a day later, at which point it would succeed. (I haven't found a way to properly test this without waiting 60 days.)

For comparison,

• While mostly harmless, this is somewhat abusive of the Letsencrypt API. I'm not sure whether rate limits might be a concern.
• The setup is as simple as can be, but will generate a lot of spurious error messages whenever renewal happens.
• Aside from rate limits, this natively handles a large number of colliding challenges.

Different client, upstream fixes

From what I've seen in the (non-public) DuckDNS Google groups, the DuckDNS admins have only recently been alerted to the existence of this problem, and are considering increasing the limit to two TXT records at a time. This would neatly cover the most common case. I'm not sure if they'd want to go higher, since the service overall has to be as simple (and cheap-to-operate) as possible.

As far as I can tell, Certbot doesn't have to run all challenges in parallel (though it may be the more "natural" way to use the ACME protocol). There even seem to exist other clients that can run challenges serially, but as of this writing I haven't looked into any all that closely. This should be entirely a question of client-side implementation, since the Letsencrypt authorization-caching behavior is present in any case.

The ideal solution here would of course be if Certbot had serial challenges as a feature. I'm not aware of any reason for why this would be difficult to implement, except perhaps that it's only useful in somewhat narrow cases such as this.