What happened
Deployed a service with type: LoadBalancer, ran kubectl get svc, and the EXTERNAL-IP column just says <pending>. Waited five minutes. Still pending. Ten minutes. Still pending.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-app-svc LoadBalancer 10.96.123.45 <pending> 80:31234/TCP 12m
Nine times out of ten, this comes down to one of three things: no cloud-provider controller running, wrong cloud config, or you're on a bare-metal/local cluster where LoadBalancer provisioning simply doesn't exist out of the box.
Debug process
Step 1 — Check service events first
kubectl describe svc my-app-svc
Scroll to the Events section at the bottom. Nothing there — or just <none>? That means the cloud-provider controller isn't even attempting to provision an IP. It's almost certainly not running at all.
Step 2 — Check if a cloud-provider controller exists
# For cloud clusters, check cloud-controller-manager
kubectl get pods -n kube-system | grep cloud-controller
# For bare-metal with MetalLB
kubectl get pods -n metallb-system
# For k3s (uses ServiceLB/Klipper by default)
kubectl get pods -n kube-system | grep svclb
If none of these exist, you have your answer. Nobody is responsible for allocating external IPs.
Step 3 — Check node annotations on cloud clusters
The cloud controller manager reads node annotations to figure out which region, zone, and VPC to provision in.
kubectl describe nodes | grep -A5 'Labels\|Annotations' | grep -i 'region\|zone\|provider'
Look for topology.kubernetes.io/region (or the older failure-domain.beta.kubernetes.io/region on pre-1.17 clusters). Missing provider labels usually mean the nodes weren't bootstrapped with the correct cloud metadata flags.
Step 4 — For on-prem/bare-metal: confirm no IP pool is configured
# If MetalLB is installed, check if an IPAddressPool exists
kubectl get ipaddresspool -n metallb-system
# Older MetalLB uses ConfigMap
kubectl get configmap config -n metallb-system -o yaml
Solutions
Case A — Bare-metal or on-prem cluster: install MetalLB
Bare-metal Kubernetes ships with no LoadBalancer implementation. MetalLB fills that gap.
# Install MetalLB
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.5/config/manifests/metallb-native.yaml
# Wait for it to be ready
kubectl wait --namespace metallb-system \
--for=condition=ready pod \
--selector=app=metallb \
--timeout=90s
Now define an IP pool from your LAN range. Pick addresses your router isn't already handing out via DHCP:
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: local-pool
namespace: metallb-system
spec:
addresses:
- 192.168.1.200-192.168.1.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: local-advert
namespace: metallb-system
kubectl apply -f metallb-pool.yaml
Case B — Local dev cluster (minikube, kind, Docker Desktop)
Local clusters don't provision real IPs. On minikube, the tunnel command handles it:
# In a separate terminal — keep it running
minikube tunnel
Run that, then check kubectl get svc again. The external IP should flip to 127.0.0.1 within seconds.
Running kind? Install MetalLB the same way as bare-metal above, or just switch the service to NodePort for local testing — less setup for throwaway environments.
Case C — Cloud cluster (EKS, GKE, AKS): cloud-controller-manager not running
# Check controller manager logs
kubectl logs -n kube-system -l component=cloud-controller-manager --tail=50
# Or for managed clusters, check the provider-specific controller
# EKS: aws-load-balancer-controller
kubectl get pods -n kube-system | grep aws-load-balancer
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller --tail=50
EKS dropped the in-tree cloud provider years ago. You need the AWS Load Balancer Controller installed separately:
# Install via Helm
helm repo add eks https://aws.github.io/eks-charts
helm repo update
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=my-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller
GKE manages the controller for you — if it's misbehaving, check node pool health and confirm the GKE service account has compute.networks.use permission on your VPC.
Case D — Subnet or annotation misconfiguration on cloud providers
A LoadBalancer needs to know which subnet to attach to. Skip the annotation and it waits silently forever.
# EKS: annotate the service with the subnet ID
kubectl annotate svc my-app-svc \
service.beta.kubernetes.io/aws-load-balancer-subnets=subnet-abc123,subnet-def456
# GKE: internal LB annotation if using internal-only networking
kubectl annotate svc my-app-svc \
cloud.google.com/load-balancer-type=Internal
Case E — Firewall or Security Group blocking health checks
Cloud load balancers send health check traffic back to your nodes. If the security group blocks that traffic, the LB silently stalls during provisioning.
# EKS: ensure node security group allows inbound from load balancer SG
# Check what ports the health check uses
kubectl describe svc my-app-svc | grep -i 'node.*port\|health'
Verification
# Watch until EXTERNAL-IP is populated
kubectl get svc my-app-svc -w
# Once IP appears, test connectivity
curl http://<EXTERNAL-IP>:80
# Confirm endpoints are healthy
kubectl get endpoints my-app-svc
The -w flag streams changes in real time. After applying the fix, <pending> should flip to a real IP within 30–60 seconds.
Lessons learned
- First question every time: does this cluster have anything responsible for
LoadBalancerprovisioning? On bare-metal and local clusters, the answer is no — unless you explicitly install MetalLB or a similar controller. - Cloud clusters that break after an upgrade often have a version mismatch: the cloud-controller-manager wasn't bumped in sync with the control plane. Silent failures follow.
- Just need something unblocked fast? Switch
type: LoadBalancertotype: NodePortand hit<node-IP>:<node-port>directly. Not elegant, but it works in under a minute. - MetalLB in L2 mode needs no BGP setup — a simple IP range config covers most home labs and on-prem environments. BGP mode is more production-grade but requires router support.
kubectl describe svcevents are underrated. They often tell you exactly what the controller tried and why it failed, saving 20 minutes of blind debugging.