Fix Nginx 404 When Certbot Can't Verify .well-known/acme-challenge for SSL Renewal

The Error

Run certbot renew and it dies with:

open() "/var/www/html/.well-known/acme-challenge/xxxxxxxxxxxxxxxx" failed (2: No such file or directory)

Let's Encrypt drops a token file at a known URL, then fetches it over HTTP port 80 to confirm you control the domain. Nginx returns 404. Validation fails. Certificate goes unrenewed.

Why This Happens

Three causes, ranked by how often they bite people:

HTTP→HTTPS redirect kills the challenge request — The ACME server always hits port 80. If Nginx redirects all port-80 traffic to HTTPS first, the 302 fires before the challenge file is ever checked. This accounts for the vast majority of cases.
Wrong webroot path — Certbot writes the token to one directory, but Nginx's root directive points somewhere else. The token exists on disk; Nginx just can't find it.
Missing directory — The .well-known/acme-challenge/ directory doesn't exist on disk at all.

Quick Fix

Step 1: Check what webroot Certbot is targeting

cat /etc/letsencrypt/renewal/yourdomain.com.conf

Find the webroot_path line. It must match a directory Nginx actually serves files from. Mismatch? Either correct the path in this file or update your Nginx root directive.

Step 2: Create the challenge directory if it's missing

mkdir -p /var/www/html/.well-known/acme-challenge
chmod 755 /var/www/html/.well-known/acme-challenge

Step 3: Add an explicit location block before any redirect

Open your Nginx server block for port 80. Add the challenge location above any return 301 or rewrite line:

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;

    # ACME challenge — must come before the HTTPS redirect
    location ^~ /.well-known/acme-challenge/ {
        root /var/www/html;
        default_type "text/plain";
        try_files $uri =404;
    }

    # Everything else goes to HTTPS
    return 301 https://$host$request_uri;
}

The ^~ modifier tells Nginx to stop regex processing once this block matches. The redirect never fires for challenge URLs.

Step 4: Test config and reload

nginx -t && systemctl reload nginx

Step 5: Retry with a dry run

certbot renew --dry-run

Permanent Fix: Shared Snippet for All Domains

Managing several domains? Create one reusable snippet so no server block ever misses this:

# Create: /etc/nginx/snippets/acme-challenge.conf
location ^~ /.well-known/acme-challenge/ {
    root /var/www/html;
    default_type "text/plain";
    try_files $uri =404;
}

Then pull it into every HTTP server block:

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;

    include snippets/acme-challenge.conf;

    return 301 https://$host$request_uri;
}

One include line. Every new domain inherits the correct challenge config automatically.

Edge Case: Multiple Domains With Different Webroots

Renewing a cert that covers domains in separate directories? Check the [[webroot_map]] section in the renewal config:

# /etc/letsencrypt/renewal/yourdomain.com.conf
[renewalparams]
authenticator = webroot
webroot_path = /var/www/html,

[[webroot_map]]
yourdomain.com = /var/www/html
www.yourdomain.com = /var/www/html
api.yourdomain.com = /var/www/api

Each domain maps to the directory Nginx serves it from. One wrong path fails the entire renewal — not just that domain.

Verification

Don't rely on guesswork. Drop a test file and fetch it over HTTP directly:

# Drop a test token
echo "ok" > /var/www/html/.well-known/acme-challenge/healthcheck

# Fetch it over HTTP — must return 200, not a redirect
curl -v http://yourdomain.com/.well-known/acme-challenge/healthcheck

You want HTTP/1.1 200 OK with body ok. Still getting 404 or a redirect? Run these:

# Check if the location block appears in the active config
nginx -T | grep -A5 'well-known'

# See what root path port 80 is using
nginx -T | grep root

Once curl returns 200, run the dry run:

certbot renew --dry-run

Dry run clean means the scheduled renewal from certbot.timer will work too. Clean up afterward:

rm /var/www/html/.well-known/acme-challenge/healthcheck