[GIRAPH-1033] Remove zookeeper from input splits handling

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Assignee: Maja Kabiljo
• Reporter: Maja Kabiljo
• Created: 12/Oct/15 17:56
• Updated: 14/Oct/16 00:58
• Resolved: 19/Oct/15 17:31

Description

Currently we use zookeeper for handling input splits, by having each worker checking each split, and when a lot of splits are used this becomes very slow. We should have master coordinate input splits allocation instead, making the complexity proportional to #splits instead of #workers*#splits.