[MESOS-993] Performance issue during replicated log catch-up when the initial log position is large

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.17.0
• Fix Version/s: 0.17.0
• Assignee: Jie Yu
• Reporter: Jie Yu
• Created: 12/Feb/14 22:33
• Updated: 24/Feb/14 19:40
• Resolved: 13/Feb/14 23:11

Description

Inside Replica::persist(const Action& action), we update the in-memory 'holes' data structure

// Update holes if we just wrote many positions past the last end.
for (uint64_t position = end + 1; position < action.position(); position++)

{ holes.insert(position); }

During catch-up (starts with an empty log), if the 'position' is very large, we may spend a lot of time in this loop. Also, the data structure 'holes' might become extremely large.

Issue Links

• relates to: MESOS-736 Support catch-up replicated log
• relates to: MESOS-1004 Provide IntervalSet abstraction for more compact representation of set