The problem
A single huge upload over one connection is fragile. One dropped packet late in the transfer wastes everything, and one stream cannot saturate available bandwidth. Multipart upload solves both.
How it works
- The client initiates an upload and receives an upload id.
- It splits the object into parts, each a contiguous byte range, and uploads them in parallel, often from many threads.
- Each part returns an ETag that fingerprints its bytes.
- The client sends a complete call listing part numbers and ETags. The store stitches the parts into one object in order.
Why it is better
- Parallelism uses the full network pipe.
- A failed part is retried alone, not the whole file.
- Parts that are never completed can be swept by a lifecycle rule so they do not leak storage.
Key idea
Multipart upload breaks a big object into independently retryable parts uploaded in parallel, then commits them as one object by part list.