[HBASE-13460] Revise the MetaLookupPool executor-related defaults (introduced in HBASE-13036)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0, 2.0.0
• Assignee: Devaraj Das
• Reporter: Devaraj Das
• Created: 14/Apr/15 00:05
• Updated: 06/Apr/18 04:34
• Resolved: 15/Apr/15 14:57

Description

The value of the max threads, max tasks, etc. can be improved to not unnecessarily reject tasks. In one of our internal tests, we saw tasks getting rejected (with the defaults that is currently there in the unpatched code):

2015-04-09 12:36:34,111|beaver.machine|INFO|15134|140028818736896|MainThread|Caused by: java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task 
org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@6658c97b rejected from 
java.util.concurrent.ThreadPoolExecutor@5a06eeee[Running, pool size = 3, active threads = 3, queued tasks = 300, completed tasks = 0]
2015-04-09 12:36:34,111|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:208)
2015-04-09 12:36:34,112|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.ClientSmallReversedScanner.next(ClientSmallReversedScanner.java:145)
2015-04-09 12:36:34,112|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1246)
2015-04-09 12:36:34,112|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1152)
2015-04-09 12:36:34,113|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:370)
2015-04-09 12:36:34,113|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:321)
2015-04-09 12:36:34,113|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
2015-04-09 12:36:34,113|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
2015-04-09 12:36:34,114|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1414)
2015-04-09 12:36:34,114|beaver.machine|INFO|15134|140028818736896|MainThread|at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1010)

Issue Links

• is related to: HBASE-13036 Meta scanner should use its own threadpool