[SOLR-5025] Implement true re-sharding for SolrCloud

• Type: Wish
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: SolrCloud
• Assignee: Unassigned
• Reporter: Shawn Heisey
• Created: 10/Jul/13 05:45
• Updated: 13/Sep/16 18:59
• Resolved: 13/Sep/16 18:58

Description

Shard splitting is an incredibly nice thing to have, but it doesn't completely address the idea of re-sharding.

Let's say that you currently have three shards, only your index is three or four times as big as you ever expected it to get when you first built it. You've added nodes, which helps, but doesn't address the fundamental fact that each of your shards is too big for an individual server. If you had created eight shards up front, everything would be smooth. It's not possible with shard splitting to go from three equal size shards to eight equal size shards.

A new feature to accomplish true re-sharding would solve this. One implementation possibility: Create a new collection with the new numShards, split all the documents accordingly to the new replicas, then rename/swap the collection and core names.

There are a number of sticky points to iron out regardless of the implementation method chosen, some of which could be really hairy.

Issue Links

• duplicates: SOLR-9241 Rebalance API for SolrCloud