The Problem: Pods stuck in Pending stateYou have just deployed a new application to your Kubernetes cluster. Everything looks correct in your YAML file, but when you check the status of your pods, they are stuck in Pending. Running a kubectl describe pod reveals the following warning in the events section:
0/1 nodes are available: 1 node(s) had taint {key: value}, that the pod didn't tolerate.
This error indicates a mismatch between the Node's Taints and the Pod's Tolerations. In Kubernetes, Taints are like a "Keep Out" sign on a node. Unless a Pod has a specific "Pass" (Toleration) that matches that sign, the scheduler will refuse to place the pod on that node.
Step 1: Identify the Taints on your NodesTo fix this, you first need to find out exactly what taint is blocking your pod. You can list all nodes and their current taints using this command:
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
If you have a specific node in mind, you can describe it to see more details:
kubectl describe node <node-name> | grep Taints
You will see output similar to key=production:NoSchedule or node-role.kubernetes.io/control-plane:NoSchedule. This tells you the Key, the Value, and the Effect of the taint.
Step 2: Inspect the Pod's requirementsCheck your Pod or Deployment manifest. If you haven't explicitly defined a tolerations section, your pod will not be able to land on any tainted node. Even if you have defined them, a tiny typo in the key or value will cause the scheduler to reject the pod.
Common effects you will see:
- NoSchedule: New pods won't be scheduled unless they tolerate the taint.- PreferNoSchedule: The system tries to avoid placing pods there, but it's not a hard requirement.- NoExecute: Existing pods will be evicted if they don't tolerate the taint.### Step 3: Choose a SolutionThere are two primary ways to resolve this error depending on your goal.
Option A: Add a Toleration to your Pod (Recommended)If the node is tainted for a good reason (e.g., it has specialized GPU hardware or is reserved for production workloads), you should update your Pod or Deployment manifest to include a matching toleration.
In your deployment.yaml, add the tolerations field under spec.template.spec:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
replicas: 1
template:
spec:
containers:
- name: my-container
image: nginx:latest
tolerations:
- key: "key"
operator: "Equal"
value: "value"
effect: "NoSchedule"
If the taint only has a key and no value, use the Exists operator:
tolerations:
- key: "dedicated"
operator: "Exists"
effect: "NoSchedule"
Option B: Remove the Taint from the NodeIf the node was tainted by mistake or you no longer want to restrict it, you can remove the taint using the kubectl taint command. Notice the minus sign (-) at the end of the command—this is what removes it.
kubectl taint nodes <node-name> key=value:NoSchedule-
Step 4: Verify the FixAfter applying the changes (either by updating the deployment or removing the taint), check the status of your pod again:
kubectl get pods -w
The pod should transition from Pending to ContainerCreating and finally to Running. You can also verify which node it landed on:
kubectl get pod <pod-name> -o wide