When Your Container Ignores the Stop Command
Few things are as annoying as a container that refuses to die. You run docker stop, wait the default 10 seconds, and instead of a clean exit, your terminal hangs. Then Docker throws a 'cannot stop container' error. Standard commands like docker kill or docker-compose down suddenly feel useless against these zombie processes. Your container is caught in a ghost state: it's not fully running, but it's not dead either.
The Exact Error Message
You will likely see this specific output in your terminal or Docker logs:
Error response from daemon: cannot stop container my-container: tried to kill container, but did not receive an exit event
Why Containers Get Stuck
Docker stops containers by sending a SIGTERM signal to PID 1. If the process doesn't exit within 10 seconds, Docker escalates to SIGKILL. This error happens when even SIGKILL fails or the Docker daemon loses track of the process state. This usually boils down to a few technical bottlenecks:
- Signal Mismatch: Your application is running as PID 1 but isn't programmed to catch or forward Linux signals.
- Uninterruptible Sleep (D State): The process is stuck waiting for I/O, like a 90-second timeout on a hung NFS mount or a failing SSD.
- Security Locks: AppArmor or SELinux profiles are blocking the daemon from sending the final kill signal.
- Kernel Desync: The Linux kernel and
containerdhave lost their shared state regarding that specific process.
Step-by-Step Fixes
Method 1: Forceful Kill via Command Line
Don't jump into complex system changes yet. Try the most aggressive signal first to see if the daemon can still reach the process.
docker kill --signal=SIGKILL my-container
If the terminal returns the same 'exit event' error, the Docker daemon has lost control. We need to bypass Docker and talk directly to the host OS.
Method 2: Kill the Process via the Host PID
Since Docker containers are just isolated processes on your Linux host, you can terminate them using standard Linux tools. First, we need to find the real Process ID (PID) from the host's perspective.
- Extract the host PID:
docker inspect --format '{{.State.Pid}}' my-container
- If you get a number (e.g., 4502), force-kill it manually:
```
sudo kill -9 4502
If the PID returns as 0, it means Docker thinks the process is gone, but the container metadata is still stuck in the 'Running' state.
Method 3: Clear the Containerd Shim
Docker uses containerd-shim to manage container lifecycles. Sometimes the main process dies, but this shim lingers and confuses the daemon.
- Find the shim process associated with your container:
ps aux | grep containerd-shim | grep my-container
- Identify the PID and remove it:
```
sudo kill -9 <shim_pid>
Method 4: Identify 'D' State Processes
If a process refuses to die even after kill -9, it is likely in Uninterruptible Sleep (D). These processes are waiting on hardware and ignore all signals. You can spot them by checking the process status column in ps.
ps -eo pid,stat,comm | grep " D "
If your container appears here, check your mounts. A hung NFS share or a disk with 100% iowait is usually the culprit. You must fix the hardware or network hang before the process can vanish. Often, a host reboot is the only way to clear a D-state process.
Method 5: Restarting the Docker Engine
When the internal state of containerd becomes corrupted, a service refresh often clears the ghost events.
sudo systemctl restart docker
Warning: This will stop all running containers unless you have "live-restore": true configured in your daemon.json.
How to Prevent Stuck Containers
1. Use a Proper Init Process
Most apps aren't built to be PID 1. They don't handle signals like a system init would. Use the --init flag to wrap your app in tini, a lightweight init binary that handles signal forwarding correctly.
docker run --init my-image
2. Add Graceful Shutdown Logic
Ensure your application listens for SIGTERM. If you're using Node.js, your app should look like this:
process.on('SIGTERM', () => {
server.close(() => {
console.log('Shutting down gracefully...');
process.exit(0);
});
});
3. Tune the Stop Timeout
If your database needs 20 seconds to flush buffers, Docker's 10-second default will cause a force-kill. Give your container more time to exit cleanly.
docker stop --time=30 my-container
Final Verification
Confirm the container has been released from memory:
docker ps -a | grep my-container
If the status is Exited or the container is gone, the fix worked. You can now safely remove the remains with docker rm my-container and start fresh.