The Problem: Git Push Rejected Due to Large File Size
You're trying to push your latest changes to GitHub, perhaps it's late, and suddenly Git throws a rejection. You see something like this:
remote: error: File path/to/file.bin is 150.00 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
to github.com/your-org/your-repo.git
! [remote rejected] main -> main (pre-receive hook declined)
error: failed to push some refs to 'github.com/your-org/your-repo.git'
This message tells you exactly what's wrong: you've got a file in your repository that's bigger than GitHub's individual file size limit, which is 100MB.
Why This Happens
GitHub has a hard limit of 100MB for individual files within a Git repository. While the total repository size can go up to 5GB (or more for paid plans), no single file can exceed 100MB. This isn't just about the files currently in your working directory; it's about any file that has ever been committed to your repository's history.
Git tracks every version of every file. If you commit a large file (like a video, a large archive, a database dump, or a compiled binary), even if you later remove it with git rm, its presence remains in the repository's history. When you push, GitHub's pre-receive hooks scan the entire history of the incoming commits. If it finds a blob (a file's content) larger than 100MB anywhere in that history, it rejects the push.
Analysis: Identifying the Culprit
The error message usually points directly to the offending file (e.g., path/to/file.bin). If you need to find other large files or investigate further, you can use tools like git rev-list and git ls-tree, but the most straightforward way, especially when preparing for a fix, is to let a tool like git filter-repo analyze your history.
The Immediate Fix: Rewriting History with git filter-repo
To fix this, you need to remove the large file(s) from your repository's history. This means rewriting history, which is a powerful operation. It's critical to understand the implications, especially if you're working in a shared repository.
WARNING: Rewriting history is destructive. It changes commit SHAs. If others have cloned your repository, they will need to re-clone or perform specific Git commands to synchronize with the new history. Only proceed if you understand this or are working on a fresh clone / personal branch.
Step 1: Install git filter-repo
git filter-repo is the recommended tool for rewriting Git history, replacing the older, slower, and more complex git filter-branch. It's a Python script.
On macOS (with Homebrew):
brew install git-filter-repo
On Linux (with pip):
pip3 install git-filter-repo
Or install from source if you prefer: https://github.com/newren/git-filter-repo
Step 2: Create a Fresh Clone (Crucial Safety Step)
Before you mess with history, always work on a fresh clone. This way, if something goes wrong, you haven't corrupted your primary working copy.
cd ..
git clone git@github.com:your-org/your-repo.git your-repo-cleaned
cd your-repo-cleaned
Step 3: Run git filter-repo to Strip Large Blobs
This command will rewrite your repository's history, removing any files larger than 100MB. It's smart enough to remove them from all commits where they existed.
git filter-repo --strip-blobs-bigger-than 100M
git filter-repo will run and show you its progress. It might take a while for large repositories. Once done, your local repository's history is rewritten, and the large file(s) are gone.
Step 4: Force Push to GitHub
Since you've rewritten history, a regular git push will be rejected because your local history diverges from the remote. You need to force push.
Use --force-with-lease for safety, especially in shared repositories, as it prevents overwriting remote changes if someone else has pushed in the meantime.
git push --force-with-lease origin <your-branch-name>
Replace <your-branch-name> with the branch you're pushing (e.g., main or master).
Step 5: Inform Your Team (Mandatory for Shared Repositories)
If this is a shared repository, you MUST inform your team immediately. Everyone else will need to rebase their work or re-clone the repository to synchronize with the new history. The easiest path for them is often to delete their local copy and re-clone.
Preventing Future Issues: Managing Large Files with Git LFS
After you've cleaned your history, you still need a strategy for handling large files properly. This is where Git Large File Storage (Git LFS) comes in.
Git LFS replaces large files in your Git repository with small text pointers. The actual file content is stored on a remote LFS server (like GitHub's LFS server). When you clone or checkout a branch, Git LFS downloads the actual large files. This keeps your Git repository lightweight and avoids the 100MB limit.
Step 1: Install Git LFS
First, ensure Git LFS is installed on your system.
On macOS (with Homebrew):
brew install git-lfs
On Linux (Debian/Ubuntu):
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
On Windows, you can download the installer from git-lfs.github.com or use Chocolatey:
choco install git-lfs
Step 2: Set up LFS for Your Repository
Navigate to your repository (your original, clean one, or the your-repo-cleaned one if you want to continue working there) and initialize Git LFS:
git lfs install
This command sets up Git LFS hooks in your local repository.
Step 3: Track File Types
Tell Git LFS which types of files to track. Use wildcards for common large file extensions. This adds entries to your .gitattributes file.
git lfs track "*.bin"
git lfs track "*.zip"
git lfs track "*.mp4"
git lfs track "*.psd"
After tracking, you need to commit the .gitattributes file:
git add .gitattributes
git commit -m "Configure Git LFS to track large files"
Step 4: Add and Commit Large Files as Usual
Now, any new files (or existing files that haven't been committed yet) matching the tracked patterns will be handled by LFS. Add them and commit normally:
git add path/to/another_large_file.bin
git commit -m "Add another large binary file via LFS"
git push origin <your-branch-name>
You should see Git LFS output during the push, indicating that it's uploading the large files to the LFS server.
Verification Steps
After performing the fix and setting up LFS:
- Confirm Push Success: Your
git pushcommand should now complete without the remote rejection error. - Check GitHub Repository Size: Go to your repository settings on GitHub. The reported size might take a little while to update, but it should eventually reflect the removal of the large files from history.
- Verify LFS Tracking: If you've set up Git LFS, check a file that should be tracked. In your local repository, if you open a large file that's LFS-tracked, you should see a small text pointer instead of the actual file content. The actual content is downloaded when Git LFS is active.
With these steps, your repository is clean, your push is unblocked, and you have a solid strategy for managing large files going forward.