Fix "No space left on device" Error on Linux

The Scenario

Mid-deploy. Mid-build. Mid-restart. Then everything stops:

No space left on device

Writes fail. Logs go silent. Databases bail out. The good news — once you know where to look, you're usually back in 5 minutes.

Analysis — Find What's Actually Full

Step 1: Check overall disk usage

df -h

Scan the Use% column. Anything at or near 100% is your problem. Nine times out of ten it's the / (root) partition.

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        50G   50G     0 100% /
tmpfs           2.0G  1.2G  800M  60% /dev/shm

Step 2: Check inode usage — this one catches people off guard

Disk space can show gigabytes free while inodes are completely exhausted. PHP session files, mail queues, and Node.js cache directories are notorious for this — a single app can create millions of tiny files.

df -i

Filesystem      Inodes  IUsed   IFree IUse% Mounted on
/dev/sda1      3276800 3276800       0  100% /

IUse% at 100%? You need to delete many small files, not hunt for large ones.

Step 3: Track down the bloat

Start at root and drill down until you find the offender:

# Top-level — see where the bulk lives
du -sh /* 2>/dev/null | sort -rh | head -20

# Narrow it down
du -sh /var/* 2>/dev/null | sort -rh | head -20
du -sh /var/log/* 2>/dev/null | sort -rh | head -20

On a busy server, /var/log growing to 20–30 GB overnight is not unusual.

Quick Fix — Free Space Right Now

Clear old logs

# Truncate the biggest offenders without breaking running processes
> /var/log/syslog
> /var/log/auth.log

# Trim systemd journal logs to 100 MB
journalctl --vacuum-size=100M

# Or drop anything older than 7 days
journalctl --vacuum-time=7d

Purge old package cache

# Debian/Ubuntu — often clears 500 MB to 2 GB
apt clean
apt autoremove -y

# RHEL/CentOS
yum clean all
dnf clean all

Remove old kernel images (Ubuntu/Debian)

Old kernels pile up after updates. Each one takes ~200–400 MB. Check what's installed, then let apt clean house:

dpkg --list | grep linux-image

# Remove old kernels automatically (keeps current + one previous)
apt autoremove --purge

Hunt large files manually

# Files over 100 MB
find / -type f -size +100M 2>/dev/null

# Core dumps — these can be several GB each
find / -name "core" -type f 2>/dev/null
find /var/crash -type f 2>/dev/null

Docker taking up space?

# Wipes unused images, stopped containers, and dangling volumes
docker system prune -a --volumes

On active dev machines, Docker bloat can quietly consume 20–40 GB. Worth checking first.

The sneaky one: deleted files still held open

A process can hold a file descriptor open even after the file is deleted. The disk space won't release until that process restarts. Run this to expose them:

lsof | grep deleted | sort -k7 -rn | head -20

The 7th column shows file size. Restart whichever service shows up at the top:

systemctl restart nginx   # substitute the actual service name

Permanent Fix — Stop It Happening Again

Configure log rotation for your app

Check /etc/logrotate.conf and /etc/logrotate.d/. If your app writes its own logs, add a config like this:

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    delaycompress
}

Cap journald before it grows unchecked

Edit /etc/systemd/journald.conf:

[Journal]
SystemMaxUse=500M
SystemKeepFree=1G

systemctl restart systemd-journald

Set up a disk alert before it's an emergency

# Fires an email when disk hits 80%
*/10 * * * * root df -h / | awk 'NR==2 {gsub("%","",$5); if($5>80) print "Disk usage: "$5"%" | "mail -s \"Disk Alert\" admin@example.com"}'

Getting paged at 80% gives you time to act. Getting paged at 100% means you're already on fire.

Expand the partition if you've outgrown it

On VMs and cloud instances, resize the volume in your provider's console first, then grow the filesystem:

# ext4
resize2fs /dev/sda1

# xfs
xfs_growfs /

# Confirm
df -h /

Verify the Fix

# Space back?
df -h

# Inodes back?
df -i

# Write test — exit code 0 means success
dd if=/dev/zero of=/tmp/test_write bs=1M count=10 && rm /tmp/test_write
echo $?

Write succeeds, df -h shows breathing room — you're done.

Quick Reference Checklist

• df -h — which partition is full
• df -i — inode exhaustion
• du -sh /* | sort -rh — find the big directories
• lsof | grep deleted — files held open by processes
• journalctl --vacuum-size=100M — trim systemd logs
• apt clean && apt autoremove — package cache (Debian/Ubuntu)
• docker system prune -a — if Docker is involved