[LUCENE-2435] A Sharding Split Policy For Load-Sharing Index Writes

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.1
• Component/s: core/index
• Lucene Fields: New, Patch Available
• Assignee: Unassigned
• Reporter: Karthick Sankarachary
• Created: 02/May/10 19:52
• Updated: 28/Aug/22 12:25

Description

(Please note that the feature described in LUCENE-2425 is required reading for this issue.)

The sharding split policy builds on the remoting split policy, treating each sub-index as a shard of the split index. Optionally, the super-index on which the split writer is built may also be considered to be a shard. The motivation for this policy is to allow a given index to be partitioned across multiple nodes (that define a cluster), which helps to (a) load-balance document writes across the nodes in that cluster and (b) serve search requests from a (narrower) set of sub-indices (that the split rule routes the query to).

Unlike other sharding solutions for Lucene, this approach gives users the ability to (a) pick and choose the types of directories underlying the mirror, and (b) add/remove shards on the fly.

Issue Links

• requires: LUCENE-2425 An Anti-Merging Multi-Directory Indexing Framework