Object Storage & File Uploads
Intermediate · 15 min read · High-Level Design, Storing Data
How S3-style storage works, presigned URLs, multipart and resumable uploads.
Photos, videos, documents, backups, logs, ML datasets — most bytes in most systems are unstructured blobs, not rows. Storing them in a relational database bloats it, slows backups and replication, and wastes expensive IOPS.
The object storage model
Object storage is a giant, flat key-value store optimised for large values:
Metadata lives in a database
The database row knows who owns the file, where it is (bucket + key), its size, type, checksum and processing status. Object storage only knows bytes. Queries like "list this user's files sorted by date" go to the database, never to the object store's LIST.
Uploads: keep bytes off your servers
Client → your API server → object storage. Every byte passes through your fleet, tying up connections and bandwidth for minutes on large uploads, and making your servers the bottleneck.
Serving blobs
Common mistakes
Interview questions
- How would you let users upload 2 GB videos from a mobile app?
- Why not store images in the database?
- What is erasure coding and why do object stores use it?
Practice
Write your answer before opening the suggested one. Anything you mark "Partly" or "Missed it" comes back in your review deck.
Further reading
Key takeaways
- Store blobs in object storage; keep metadata and the object key in a database.
- Object storage is a flat, immutable, HTTP-accessed key-value store with very high durability and low cost.
- Use presigned URLs so clients upload and download directly; confirm uploads via storage events.
- Use multipart and resumable uploads for large files; chunking enables deduplication and delta sync.
- Serve blobs through a CDN with immutable, versioned keys.