HTTPS and Let's Encrypt: how ACME issues your certificate

A certificate is a file that says "this public key belongs to whoever controls example.com", signed by an authority browsers already trust. Let's Encrypt gives them away because the interesting part is not the file, it is the proof: an automated exchange called ACME, in which a certificate authority sets you a small task only the real owner of a name could complete. Pass it and you get a certificate that lasts 90 days. Automate passing it and you never think about certificates again.

This is what happens during that exchange, what breaks it, and what you still have to do yourself once the padlock appears.

What a certificate actually proves#

Less than most people think. The common Let's Encrypt certificate is domain validated: the only claim it makes is that whoever requested it could demonstrate control of the domain name at that moment. It says nothing about who runs the site, whether the company exists, or whether the site is honest. A phishing site can and does get a valid certificate in thirty seconds.

What it does give you is worth having:

• Encryption. Nobody between the visitor and the server can read or change the traffic. On a shared wifi network that is the difference between a private session and a public one.
• Integrity. An intermediate cannot inject a script, a redirect or an advert into the page.
• Identity for the name. The visitor is talking to the machine the DNS points at, not to whatever answered first.

Organisation-validated and extended-validation certificates exist, cost money, and involve a human checking company records. They are worth buying if a contract requires one. Browsers stopped showing the company name in the address bar in 2019, so the visible benefit is now nil.

The ACME exchange, step by step#

ACME is a protocol - RFC 8555 - between a client on your side and the CA. Certbot is one client, your hosting panel runs another, and the steps are identical whichever it is.

1. Account. The client generates a key pair and registers it with the CA. This happens once and identifies you from then on.
2. Order. The client asks for a certificate covering one or more names: example.com, www.example.com.
3. Authorization and challenge. For each name, the CA replies with a token and a list of ways to prove control.
4. Preparation. The client publishes the answer - a file on your web server, or a DNS record.
5. Validation. The client tells the CA it is ready. The CA resolves the name from its own side, fetches the answer and checks it. This is the step that fails, and it fails because the CA sees something different from what you see.
6. Finalize. The client sends a certificate signing request with the public key it will use. The CA signs it.
7. Download. The client fetches the certificate and the intermediate chain, installs them, and reloads the web server.

The important thing in that picture is that the CA validates from the public internet, using public DNS. Your /etc/hosts file, your VPN, your browser cache and the DNS your laptop happens to use are all irrelevant. If the name resolves to the old host for the rest of the world, issuance fails no matter how correct the new server is.

The three challenge types#

HTTP-01 is the default. The client writes a file whose name is the token and whose contents are the token plus a hash of your account key, and the CA fetches http://example.com/.well-known/acme-challenge/<token> over plain HTTP on port 80. It follows redirects, so a site that sends everything to HTTPS is normally fine, but a rule that returns a 404, a login page or a WAF block for that path is not.

DNS-01 is the only way to get a wildcard such as *.example.com, and the only option when port 80 is closed or the machine is not publicly reachable at all. The cost is that automating it needs an API token for your DNS provider, and DNS propagation makes each renewal slower. Note that a wildcard covers one level: *.example.com matches app.example.com but not a.b.example.com.

TLS-ALPN-01 proves control over port 443 by answering a special handshake using the acme-tls/1 protocol. You will only meet it if you run a proxy that handles TLS itself.

Most people never choose. The client picks HTTP-01, it works, and the choice only becomes interesting when you want a wildcard.

Renewal, and the window that matters#

Certificates from Let's Encrypt last 90 days. That is deliberate: a short life forces automation, and automation is what stops the annual outage where somebody forgot. The renewal is not a special operation - it is the same exchange again, from order to download.

Clients renew early so there is room to fail. Certbot's timer runs twice a day and renews anything with 30 days or less remaining, which leaves a month of retries before anything is user-visible. Panels that do it for you use their own window: on RE:NODE a scheduled task renews any certificate that has fallen inside a 21-day window, without being asked, so a single failed attempt is not an incident.

What you should still do:

• Watch the expiry from outside. The failure mode that hurts is a renewal that has been failing quietly for weeks. Check the live certificate from another machine, not the file on disk.
• Know what reloads the server. A renewed certificate on disk does nothing until the web server reads it. Certbot handles this with a deploy hook; a panel does it for you.
• Do not pin anything. HPKP is dead, and pinning a certificate in a mobile app to a 90-day certificate is a scheduled outage.

$ echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \    | openssl x509 -noout -subject -issuer -dates

That prints the name the certificate covers, who signed it and when it expires, as the internet sees it. It is the first command to run when someone reports a certificate warning, because it distinguishes an expired certificate from a wrong name from a missing chain.

Running an ACME client yourself#

On your own server, certbot is the default and the nginx plugin does the whole job, including editing the server block and reloading:

$ certbot --nginx -d example.com -d www.example.com$ certbot renew --dry-run

If you would rather it only obtained the files and left your configuration alone, use the webroot method and point it at the directory your site is served from:

$ certbot certonly --webroot -w /var/www/site -d example.com -d www.example.com

Either way the result lands in /etc/letsencrypt/live/example.com/:

Use fullchain.pem. Serving cert.pem alone is the classic mistake: it works in your browser, because your browser has cached the intermediate from some other site, and fails for a visitor whose browser has not. Testing from a machine that has never visited the site is the only reliable check.

certbot renew --dry-run runs the whole renewal against the staging environment. Run it after any change to your server configuration, because the time to find out that renewal is broken is not 89 days later.

On managed hosting you do none of this. RE:NODE's app and web plans carry a proxy slot: point an A record at the address shown on the tab, and the certificate is issued and renewed for you. Your application never sees a certificate, never listens on 443, and gets the real visitor address in X-Forwarded-For - what a reverse proxy does walks through that request path.

Mixed content, redirects and HSTS#

A padlock is not the end of the work. Three things usually remain.

Mixed content. A page loaded over HTTPS that asks for a script, stylesheet or iframe over HTTP has that request blocked outright by every current browser; images and media are usually upgraded or blocked with a warning. The fix is to stop generating http:// URLs. In a CMS that means the database, where absolute URLs were saved years ago: run a proper search and replace across the content rather than editing pages by hand, and check for hard-coded URLs in theme templates. Content-Security-Policy: upgrade-insecure-requests is a useful belt-and-braces header while you clean up, not a substitute for cleaning up.

The redirect. Serving both schemes means half your links are the wrong one. Send everything to HTTPS at the server:

server {    listen 80;    server_name example.com www.example.com;    return 301 https://example.com$request_uri;}

That also folds www into the apex in one hop, which is what you want - www vs apex domains and redirects covers which way round to do it and why chains of redirects cost you.

HSTS. Strict-Transport-Security tells the browser to refuse plain HTTP for this name for a period, which closes the gap where the first request of the day goes out unencrypted.

add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

Start with a short max-age such as 300 while you confirm nothing on the site needs HTTP, then raise it. includeSubDomains applies to every subdomain including ones you forgot, and preload submits the name to a list baked into browsers, which is slow and awkward to reverse. HSTS is the one header here that can take a site off the air if you get it wrong, so add it last and deliberately.

When issuance fails#

Nearly every failure is one of these, in this order of likelihood.

The name does not resolve to this server yet. You added the record five minutes ago and the old value is still cached, or you edited the zone at a provider that is no longer authoritative. Check from outside your network with dig +short example.com @1.1.1.1 and compare with the address you were given. DNS records explained covers which record belongs where, and nameservers vs DNS records covers the case where you are editing a zone nobody reads.

Port 80 is closed. HTTP-01 needs it even though the finished site only uses 443. A firewall rule or a host that only forwards 443 will fail every attempt with a connection timeout.

Something intercepts the challenge path. A CDN in front, a maintenance page, a framework that routes every unknown path to a 404 handler, an authentication layer over the whole site. /.well-known/acme-challenge/ must return the file, unauthenticated.

A CAA record forbids it. A CAA record lists which authorities may issue for your domain, and the CA checks it at issuance. If you have one and it does not name the authority your host uses, issuance fails with a CAA error. Check with dig CAA example.com, and if you are on managed hosting, ask support which authority to allow rather than guessing.

The domain is behind a proxy that answers first. With an orange-cloud CDN enabled, the CA reaches the CDN, not you. That is fine when the CDN is doing the issuance, and a problem when your origin is. Cloudflare for websites and game servers covers which layer holds which certificate.

You hit a rate limit. Let's Encrypt limits how many certificates a registered domain can get in a week, and how many identical certificates you can request. The published figures have included 50 certificates per registered domain per week and 5 duplicates, and they change, so read the current rate limits before you write a retry loop. Test against the staging environment, which has far looser limits and issues certificates browsers do not trust.

FAQ#

Is a free certificate as good as a paid one?

For encryption, identical. A Let's Encrypt certificate uses the same algorithms and is trusted by the same browsers as a domain-validated certificate costing money. You pay for organisation validation, a warranty, or a support contract - not for stronger encryption.

Why 90 days instead of a year?

Short lifetimes limit the damage from a stolen key and force the renewal to be automated. The industry has been moving the same way for paid certificates: maximum lifetimes have been cut repeatedly, and anything that relies on a human remembering will eventually be the thing that breaks.

Do I need a certificate for a site with no logins?

Yes. Browsers mark plain HTTP as not secure, search engines prefer HTTPS, and modern browser features from geolocation to service workers refuse to run without it. There is also no such thing as a page too boring to be tampered with in transit.

What happens when the certificate expires?

Visitors get a full-page interstitial warning they have to click through, and anything automated - an API client, a webhook sender, a mobile app - simply fails. There is no grace period. This is why renewal windows are measured in weeks, not days.

Can one certificate cover several domains?

Yes. A certificate can list multiple names in its subject alternative name field, so example.com, www.example.com and shop.example.net can share one. A wildcard covers every direct subdomain of one name, but needs the DNS challenge.

Why does my site say not secure when the certificate is valid?

Mixed content, nearly always. The certificate is fine and the page is loading something over plain HTTP. Open the browser console: it names the exact URL that is being blocked.