The 2 AM Production CrisisYou receive a critical monitoring alert: a database or application has gone offline. You log into the server, try to inspect the data directory, and the terminal spits back a cryptic message:
ls: cannot access '/data/mysql': Structure needs cleaning
This error—technically EUCLEAN—is a protective measure. It means the filesystem has detected metadata corruption so severe that it has locked itself down to prevent further data loss. This usually happens after a sudden power failure, a SAN connectivity hiccup, or a hardware drive failing at the worst possible moment.
Step 1: Pinpoint the Damaged PartitionDon't guess which disk is failing. You need to look at the kernel's play-by-play log to see exactly what happened. Run this command to see the most recent system events:
dmesg | grep -Ei 'xfs|ext4|error' | tail -n 20
You are looking for specific hardware addresses or mount points. A typical error might look like this: XFS (sdb1): Internal error xfs_trans_cancel at line 984. Once you see the device name (e.g., /dev/sdb1), confirm its mount point:
df -h | grep /dev/sdb1
Step 2: Unmount the Target PartitionAttempting to fix a mounted filesystem is a recipe for total data destruction. You must take the partition offline first. If it is a data drive, stop the associated services (like Nginx or MySQL) and unmount it:
sudo umount /dev/sdb1
If the system claims the device is busy, identify the culprit processes using lsof or fuser:
sudo fuser -mv /mnt/data_folder
Kill those processes and try the unmount again. If the drive is the root partition (/), you will need to boot from a Live ISO or enter Rescue Mode.
Step 3: Repairing XFS (The Most Common Scenario)XFS is the default for RHEL, Rocky, and CentOS. If your disk uses XFS, xfs_repair is your primary tool. Start with a standard repair attempt:
sudo xfs_repair /dev/sdb1
When the Log is 'Dirty'If the system crashed recently, xfs_repair might refuse to run. It will tell you the log contains unwritten data and suggest mounting the drive to 'replay' it. If mounting fails, you are stuck. In this specific scenario, you may need the 'Last Resort' flag:
sudo xfs_repair -L /dev/sdb1
The -L flag (Force Log Zeroing) wipes the filesystem log. This gets the partition back online but may sacrifice the last few seconds of data written before the crash. Use this only if a standard repair fails.
Step 4: Repairing ext4 FilesystemsFor Ubuntu or Debian systems using ext4, use the fsck utility instead. The -f flag is necessary to force a check even if the filesystem marks itself as 'clean':
sudo fsck.ext4 -fy /dev/sdb1
The -y flag automatically answers 'yes' to every repair prompt. This is helpful because a badly corrupted ext4 drive can trigger hundreds of individual fix prompts.
Step 5: Verify and RemountOnce the repair tool reports success, try mounting the drive again. Check the directory contents to ensure your files are visible:
sudo mount /dev/sdb1 /mnt/data_folder
ls -la /mnt/data_folder
Keep an eye on dmesg for the next few minutes. If the error returns immediately, the underlying hardware is likely failing.