Fix AWS S3 Error: Your proposed upload exceeds the maximum allowed object size

The Error

You're uploading a file to S3 and get this back:

An error occurred (EntityTooLarge) when calling the PutObject operation:
Your proposed upload exceeds the maximum allowed object size.

Upload stops dead. The file never lands in the bucket.

Root Cause

S3 has two size limits that trip people up constantly:

• Single PUT upload: max 5 GB per request
• Maximum object size: 5 TB (multipart upload only)

Hit the limit with a standard PutObject call — SDK, CLI's s3api put-object, or a raw HTTP PUT — and AWS rejects it immediately. No partial upload, no retry. Just this error.

Multipart upload is the solution. It splits the file into chunks of at least 5 MB each (the last part can be smaller), pushes them in parallel, and S3 reassembles them server-side. A 50 GB database dump, a 200 GB video archive — both work fine this way.

Fix 1: AWS CLI (Recommended for Quick Uploads)

Good news: aws s3 cp and aws s3 sync already handle multipart automatically. If you're seeing this error from the CLI, you're probably using s3api put-object — switch to the high-level commands instead.

# Automatically uses multipart for files > 8 MB by default
aws s3 cp /path/to/largefile.tar.gz s3://your-bucket/largefile.tar.gz

Want to tune the chunk size? Larger chunks mean fewer requests, which helps on fast connections:

aws s3 cp /path/to/largefile.tar.gz s3://your-bucket/largefile.tar.gz \
  --multipart-threshold 64MB \
  --multipart-chunksize 64MB

Check what's currently configured:

aws configure get s3.multipart_threshold
aws configure get s3.multipart_chunksize

Or set them permanently in your AWS config file:

[profile default]
s3 =
    multipart_threshold = 64MB
    multipart_chunksize = 64MB
    max_concurrent_requests = 10

Fix 2: boto3 — TransferConfig for Programmatic Uploads

Using put_object() in your code? That's the culprit. Replace it with upload_file() or upload_fileobj() and attach a TransferConfig:

import boto3
from boto3.s3.transfer import TransferConfig

s3 = boto3.client('s3')

config = TransferConfig(
    multipart_threshold=1024 * 1024 * 64,   # 64 MB
    multipart_chunksize=1024 * 1024 * 64,   # 64 MB chunks
    max_concurrency=10,
    use_threads=True
)

s3.upload_file(
    Filename='/path/to/largefile.tar.gz',
    Bucket='your-bucket',
    Key='largefile.tar.gz',
    Config=config
)
print('Upload complete')

Uploading from a stream or file object? Use upload_fileobj() instead:

with open('/path/to/largefile.tar.gz', 'rb') as f:
    s3.upload_fileobj(f, 'your-bucket', 'largefile.tar.gz', Config=config)

Both methods handle multipart transparently once the file crosses the threshold. put_object() does a single PUT — always — and will fail on anything over 5 GB.

Fix 3: Manual Multipart Upload via boto3 API

Need full control? Custom retry logic, resumable uploads, progress tracking per part — this is the approach:

import boto3
import os
import math

s3 = boto3.client('s3')
BUCKET = 'your-bucket'
KEY = 'largefile.tar.gz'
FILE_PATH = '/path/to/largefile.tar.gz'
CHUNK_SIZE = 64 * 1024 * 1024  # 64 MB

# 1. Initiate
mpu = s3.create_multipart_upload(Bucket=BUCKET, Key=KEY)
upload_id = mpu['UploadId']

parts = []
try:
    file_size = os.path.getsize(FILE_PATH)
    total_parts = math.ceil(file_size / CHUNK_SIZE)

    with open(FILE_PATH, 'rb') as f:
        for part_number in range(1, total_parts + 1):
            data = f.read(CHUNK_SIZE)
            response = s3.upload_part(
                Bucket=BUCKET,
                Key=KEY,
                PartNumber=part_number,
                UploadId=upload_id,
                Body=data
            )
            parts.append({'PartNumber': part_number, 'ETag': response['ETag']})
            print(f'Uploaded part {part_number}/{total_parts}')

    # 2. Complete
    s3.complete_multipart_upload(
        Bucket=BUCKET,
        Key=KEY,
        UploadId=upload_id,
        MultipartUpload={'Parts': parts}
    )
    print('Multipart upload completed')

except Exception as e:
    # 3. Always abort on failure — incomplete uploads still incur storage charges
    s3.abort_multipart_upload(Bucket=BUCKET, Key=KEY, UploadId=upload_id)
    raise e

Verify the Fix Worked

Once the upload finishes, confirm the object is actually there and the size matches:

# Check the object exists and see its size
aws s3 ls s3://your-bucket/largefile.tar.gz

# Compare against the local file
ls -lh /path/to/largefile.tar.gz

# Inspect object metadata
aws s3api head-object --bucket your-bucket --key largefile.tar.gz

Multipart-uploaded objects have a distinctive ETag — it ends with -N where N is the part count (e.g., "abc123-12" for a 12-part upload). You'll also see x-amz-mp-parts-count in the metadata response.

Prevention

• Drop put_object() for anything that might grow large. Use upload_file() with TransferConfig as your default — it handles both small and large files gracefully.
• Set a lifecycle rule to auto-abort incomplete multipart uploads. Failed uploads leave orphaned parts in S3. You get charged for them. A 7-day cleanup rule costs nothing to set up:

aws s3api put-bucket-lifecycle-configuration \
  --bucket your-bucket \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "abort-incomplete-multipart",
      "Status": "Enabled",
      "Filter": {"Prefix": ""},
      "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7}
    }]
  }'

• Audit existing incomplete uploads occasionally:

aws s3api list-multipart-uploads --bucket your-bucket