Two Ways to Split Data
Both split a big table into smaller pieces, but they differ in scope.
- Partitioning divides a table into parts that usually live on the same database server.
- Sharding spreads parts across multiple independent servers to scale beyond one machine.
Partitioning
- Split by range, list, or hash of a key.
- The engine can prune partitions so a query reads only relevant ones.
- Helps with manageability and large table maintenance on one node.
Sharding
- Each shard is a full database holding a subset of rows.
- Spreads load and storage across many machines for horizontal scale.
- Adds complexity: routing queries, cross shard joins, and rebalancing.
Choosing
Partition first for size and maintenance on a single server. Shard when one server can no longer hold the data or serve the traffic.
Key idea
Partitioning splits a table within one server, while sharding spreads it across many servers for horizontal scale at added complexity.