[BOOKKEEPER-862] Add tracing and stats to OrderedSafeExecutor for debugging slow tasks

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 4.4.0
• Component/s: bookkeeper-server
• Assignee: Leigh Stewart
• Reporter: Leigh Stewart
• Created: 02/Aug/15 21:35
• Updated: 16/May/16 21:47
• Resolved: 06/Oct/15 08:08

Description

Porting a change form the Twitter branch to improve stats and logging in OrderedSafeExecutor
These changes have been helpful for us in debugging latency issues in Bookkeeper server/client
Summary of changes is

• add a config option for op stats
• add stats for task execution time, task pending time
• add a config option for logging a warning when an op takes longer than x micros
• add toString implementations for submitted tasks so make it easier to track down slow ops
• start using Builder for OrderedSafeExecutor
• add a very simple test to make sure that the slow op logging path is exercised

Most of this came from Sijie originally, with some changes from me