Fix HorizontalPodAutoscaler Not Scaling: "unable to get metrics for resource cpu: no metrics returned from resource metrics API"

The SituationYou set up a HorizontalPodAutoscaler expecting your deployment to scale out under load, but nothing happens. Running kubectl describe hpa gives you this:

Warning  FailedGetResourceMetric  15s   horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from resource metrics API

Your HPA shows TARGETS: <unknown>/50% and the replica count sits frozen — even when pods are clearly struggling.

Why This HappensThe HPA controller queries metrics-server for CPU and memory data. No metrics-server, no data. No data, no scaling decisions.

Three things typically cause this:

• metrics-server isn't installed in the cluster- metrics-server is running but failing health checks — usually a TLS/certificate mismatch- metrics-server can't reach the kubelet on each node (network policy or missing API flag)## Step 1: Confirm the ProblemStart with your HPA status:

kubectl get hpa -n your-namespace
kubectl describe hpa your-hpa-name -n your-namespace

Scroll to the Events section — the unable to get metrics warning should be right there. Next, check whether metrics-server is installed at all:

kubectl get deployment metrics-server -n kube-system

An Error from server (NotFound) means it's missing. If it exists, check whether it's actually running:

kubectl get pods -n kube-system | grep metrics-server
kubectl logs -n kube-system deployment/metrics-server

TLS trouble shows up clearly in the logs:

E0318 10:23:45.123456       1 scraper.go:139] "Failed to scrape node" err="Get \"https://192.168.1.10:10250/metrics/resource\": x509: certificate signed by unknown authority"

Quick Fix: Install or Patch metrics-server### Case A — metrics-server not installedInstall it with the official manifest:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Wait about 60 seconds, then verify:

kubectl top nodes
kubectl top pods -n your-namespace

Data in kubectl top nodes means metrics-server is up.

Case B — metrics-server installed but crashing (x509 / TLS error)Bare-metal clusters and kubeadm setups hit this constantly. Kubelets there use self-signed certificates, which metrics-server rejects by default. The fix is a one-line patch to disable TLS verification:

kubectl patch deployment metrics-server -n kube-system \
  --type='json' \
  -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--kubelet-insecure-tls"}]'

Watch for the rollout to finish:

kubectl rollout status deployment/metrics-server -n kube-system

Permanent Fix: Custom metrics-server ManifestRunning kubectl patch after every reinstall gets tedious. For production, keep a patched components.yaml in your infrastructure repo instead. Grab the official one first:

curl -LO https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Open the file and add --kubelet-insecure-tls under args in the Deployment section:

spec:
  template:
    spec:
      containers:
      - name: metrics-server
        args:
        - --cert-dir=/tmp
        - --secure-port=10250
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls   # add this line

Apply it:

kubectl apply -f components.yaml

Commit this file to your repo. The flag survives reinstalls and cluster upgrades.

Using Helm (alternative approach)Managing metrics-server with Helm? Pass the flag at install time:

helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server/
helm upgrade --install metrics-server metrics-server/metrics-server \
  --namespace kube-system \
  --set args={--kubelet-insecure-tls}

Verify the FixConfirm kubectl top is returning data:

kubectl top nodes
# Expected output:
# NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
# node-1       245m         6%     2312Mi          30%

kubectl top pods -n your-namespace
# NAME                        CPU(cores)   MEMORY(bytes)
# my-app-7d4b9c6f8-xk2pq      12m          128Mi

Check your HPA — the TARGETS column should now show real numbers instead of <unknown>:

kubectl get hpa -n your-namespace
# NAME       REFERENCE             TARGETS   MINPODS   MAXPODS   REPLICAS
# my-hpa     Deployment/my-app     8%/50%    2         10        2

Watch the HPA in real time:

kubectl get hpa -n your-namespace -w

To trigger a scale-up test, generate CPU load on your pods. Replicas should start climbing within a minute or two.

Still Seeing the Error?metrics-server looks healthy but TARGETS is still <unknown>? A few other things to check:

• Resource requests not set: HPA calculates CPU % relative to the pod's resources.requests.cpu. Without that field, the percentage math breaks — even when raw metrics exist.resources: requests: cpu: "100m" memory: "128Mi" limits: cpu: "500m" memory: "256Mi"- API aggregation not registered: On some clusters, the metrics API endpoint is missing entirely. Run kubectl api-versions | grep metrics — you should see metrics.k8s.io/v1beta1 in the output.- NetworkPolicy blocking metrics-server: Strict policies can silently drop traffic. Make sure metrics-server in kube-system can reach kubelets on port 10250 across all nodes.