Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Today, shard splitting works by partitioning the hash range in half (or user defined hash ranges). This works fine when pseudo-random distribution is assumed. However, composite ID routing can lead to uneven distribution of data due to co-location requirements. In such cases, splitting hash range by half may not create equal shards anymore. Although we cannot split the hash domains across ranges but we may be able to select a suitable mid-point such that two hash domains having semi-equal document counts go to separate ranges.