[HBASE-6134] Improvement for split-worker to speed up distributed log splitting

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.95.0
• Component/s: wal
• Hadoop Flags: Reviewed
• Assignee: Chunhui Shen
• Reporter: Chunhui Shen
• Created: 30/May/12 06:33
• Updated: 23/Sep/13 18:31
• Resolved: 14/Jun/12 06:31

Description

First，we do the test between local-master-splitting and distributed-log-splitting

Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths splitting work), 400 regions in one hlog file

local-master-split:60s+
distributed-log-splitting:165s+

In fact, in our production environment, distributed-log-splitting also took 60s with 30 regionservers for 34 hlog files (regionserver may be in high load)

We found split-worker split one log file took about 20s
(30ms~50ms per writer.close(); 10ms per create writers )

I think we could do the improvement for this:
Parallelizing the create and close writers in threads

In the patch, change the logic for distributed-log-splitting same as the local-master-splitting and parallelizing the close in threads.

Issue Links

• is related to: HBASE-6318 SplitLogWorker exited due to ConcurrentModificationException
• relates to: HBASE-5843 Improve HBase MTTR - Mean Time To Recover