[HBASE-13097] Use same EventLoopGroup for different AsyncRpcClients if possible

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0, 2.0.0
• Fix Version/s: 1.1.0, 2.0.0
• Component/s: IPC/RPC, test
• Hadoop Flags: Reviewed
• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 25/Feb/15 03:32
• Updated: 03/Jun/15 15:28
• Resolved: 13/Mar/15 15:10

Description

In some unit tests(such as TestAcidGuarantees) we create multiple Connection instance. If we use AsyncRpcClient, then there will be multiple netty Bootstrap and every Bootstrap has its own PooledByteBufAllocator.
I haven't read the code clearly but it uses some threadlocal technics and jmap shows io.netty.buffer.PoolThreadCache$MemoryRegionCache$Entry is the biggest things on Heap.

See https://builds.apache.org/job/HBase-TRUNK/6168/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.TestAcidGuarantees-output.txt

2015-02-24 23:50:29,704 WARN  [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(167): Detected pause in JVM or host machine (eg GC): pause of approximately 20133ms
GC pool 'PS MarkSweep' had collection(s): count=15 time=55525ms

Update: We use a singleton PooledByteBufAllocator so the reason should be too many threads. So we will work on reduce the connections and rpclients in unit tests.

Issue Links

• links to: reviewboard