Object storage

Object storage — Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compatible systems such as MinIO and Cloudflare R2 — stores files ("objects") of any size under keys in buckets. It's where images, video, uploads, backups, exports and data-lake files belong.

#What it's good at

• Durability: objects are replicated across devices and facilities; S3 is designed for 99.999999999% (eleven nines) durability.
• Scale and cost: effectively unlimited capacity at a few cents per gigabyte per month, with cheaper tiers for rarely read data.
• Throughput: many clients can read and write in parallel; large objects upload in parts.

#What it isn't

• Not a database: no queries beyond listing keys by prefix, and no transactions across objects.
• Not a filesystem: no in-place edits, and (outside a few specialised storage classes) no appends; you replace the whole object.
• Higher latency per request than a local disk (tens of milliseconds), so put a CDN in front for public content (content delivery networks).

#Patterns

• Metadata in the database, bytes in object storage. The row holds the key, size, content type and owner; the object holds the content. Databases full of blobs are slow to back up, replicate and restore.
• Presigned URLs. Your server signs a short-lived URL that lets a client upload or download one object directly. Large uploads never pass through your API servers:

1. client → API: "I want to upload photo.jpg (4 MB)"
2. API → client: presigned PUT URL for uploads/7/9f2c.jpg, valid 10 minutes
3. client → object storage: PUT the bytes directly
4. client → API: "done" (or an object-created event notifies you)

• Immutable keys. Write a new key for a new version (avatars/7/v3.jpg) instead of overwriting, which makes caching simple.
• Lifecycle rules move old objects to cheaper storage classes or delete them automatically.

Keep buckets private by default and grant access through presigned URLs or a CDN with origin authentication.