Fix AWS Secrets Manager EndpointConnectionError: Could Not Connect to Endpoint URL (VPC Endpoint Missing)

The Error

Your app is running inside a private VPC subnet. It calls AWS Secrets Manager — and gets slapped with this:

EndpointConnectionError: Could not connect to the endpoint URL: https://secretsmanager.us-east-1.amazonaws.com
ConnectionError: HTTPSConnectionPool(host='secretsmanager.us-east-1.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object>: Failed to establish a new connection: [Errno -2] Name or service not known'))

Shows up in Lambda logs, on an EC2 instance in a private subnet, inside an ECS task — anywhere with no path to the public internet.

Why This Happens

AWS SDK targets secretsmanager.<region>.amazonaws.com by default — a public endpoint. Private subnets without a NAT Gateway or Internet Gateway have no route there. DNS either fails to resolve the name, or the TCP handshake simply never completes.

The solution: a VPC Interface Endpoint (AWS PrivateLink) for Secrets Manager. It injects a private IP — something like 10.0.1.45 — into your VPC's DNS. After that, secretsmanager.us-east-1.amazonaws.com resolves to an address inside your VPC. No internet involved.

Step-by-Step Fix

Step 1 — Confirm the root cause

SSH into the affected instance (or use ECS Exec / Lambda test) and probe the endpoint directly:

# Can you reach the public endpoint at all?
curl -v https://secretsmanager.us-east-1.amazonaws.com

# Check DNS resolution
nslookup secretsmanager.us-east-1.amazonaws.com

Could not resolve host or a hanging connection confirms it: the VPC has no route to AWS APIs, and you need a private endpoint.

Step 2 — Create the VPC Interface Endpoint

In the AWS Console:

• Go to VPC → Endpoints → Create Endpoint
• Service category: AWS services
• Search secretsmanager, select com.amazonaws.<region>.secretsmanager
• Pick your VPC
• Select all private subnets where your workloads run
• Attach a Security Group allowing inbound HTTPS (port 443) from your workload's security group
• Enable Private DNS — this is the critical part

Prefer the CLI? Here it is:

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0abc123def456 \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.us-east-1.secretsmanager \
  --subnet-ids subnet-0aaa111 subnet-0bbb222 \
  --security-group-ids sg-0xxxxxx \
  --private-dns-enabled \
  --region us-east-1

Don't skip --private-dns-enabled. Without it the SDK still tries to hit the public endpoint and you're back to square one.

Step 3 — Enable DNS hostnames on the VPC

Private DNS only kicks in when the VPC has both DNS hostnames and DNS resolution enabled. Older VPCs sometimes have these off. Check first:

# Check current setting
aws ec2 describe-vpc-attribute \
  --vpc-id vpc-0abc123def456 \
  --attribute enableDnsHostnames

aws ec2 describe-vpc-attribute \
  --vpc-id vpc-0abc123def456 \
  --attribute enableDnsSupport

# Enable if disabled
aws ec2 modify-vpc-attribute \
  --vpc-id vpc-0abc123def456 \
  --enable-dns-hostnames

aws ec2 modify-vpc-attribute \
  --vpc-id vpc-0abc123def456 \
  --enable-dns-support

Step 4 — Update the endpoint's Security Group

The interface endpoint is a real network interface — it has its own security group. Port 443 must be open inbound from your Lambda, EC2, or ECS workloads:

aws ec2 authorize-security-group-ingress \
  --group-id sg-0endpoint-sg \
  --protocol tcp \
  --port 443 \
  --source-group sg-0your-workload-sg \
  --region us-east-1

Missed security group rules are one of the top reasons the endpoint exists but connections still fail.

Step 5 — Attach an Endpoint Policy (optional but recommended)

Out of the box, the endpoint grants full Secrets Manager access to any principal in your account. Lock it down to exactly what your app needs:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:role/MyAppRole"
      },
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:MyApp/*"
    }
  ]
}

Apply via Console (Endpoint → Policy tab) or CLI:

aws ec2 modify-vpc-endpoint \
  --vpc-endpoint-id vpce-0xxxxxxxxx \
  --policy-document file://endpoint-policy.json

Verify the Fix

The endpoint takes 1–2 minutes to reach Available state. Once it does, test from inside the private subnet:

# DNS should now resolve to a private IP (10.x.x.x or 172.x.x.x)
nslookup secretsmanager.us-east-1.amazonaws.com

# Fetch a secret — should return immediately
aws secretsmanager get-secret-value \
  --secret-id MyApp/prod/database \
  --region us-east-1

# Or with boto3
python3 -c "
import boto3
client = boto3.client('secretsmanager', region_name='us-east-1')
resp = client.get_secret_value(SecretId='MyApp/prod/database')
print(resp['SecretString'][:50])
"

If nslookup returns an IP inside your VPC range — say 10.0.1.45 instead of a public AWS IP — and the secret value returns cleanly, you're done.

Tips

Other endpoints you'll likely need

Secrets Manager is rarely the only AWS service a private workload calls. Each one needs its own VPC endpoint — or you're back to needing a NAT Gateway. Common additions:

• com.amazonaws.<region>.kms — when secrets use a customer-managed key
• com.amazonaws.<region>.sts — for IAM role assumption
• com.amazonaws.<region>.ssm — for Parameter Store

Subnet CIDR planning

Before attaching the endpoint across multiple AZs, double-check that your subnet CIDRs are sized correctly and don't overlap. The browser-based Subnet Calculator on ToolCraft is handy for quickly verifying ranges and counting available IPs without doing the binary math yourself.

Cost consideration

Interface endpoints aren't free: roughly $0.01/hour per AZ, plus data transfer fees. Spread across three AZs, that's ~$21/month per endpoint. If you're running a dozen services in private subnets, a single NAT Gateway at $32/month flat may actually cost less. Run the numbers against your traffic before adding an endpoint per service.

Terraform snippet

Managing this as infrastructure code:

resource "aws_vpc_endpoint" "secretsmanager" {
  vpc_id              = aws_vpc.main.id
  service_name        = "com.amazonaws.${var.region}.secretsmanager"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = aws_subnet.private[*].id
  security_group_ids  = [aws_security_group.vpc_endpoint.id]
  private_dns_enabled = true

  tags = {
    Name = "secretsmanager-endpoint"
  }
}