[HBASE-21764] Size of in-memory compaction thread pool should be configurable

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
• Component/s: Compaction, in-memory-compaction
• Labels: compaction
• Hadoop Flags: Reviewed

Release Note

Introduced an new config key in this issue: hbase.regionserver.inmemory.compaction.pool.size. the default value would be 10. you can configure this to set the pool size of in-memory compaction pool. Note that all memstores in one region server will share the same pool, so if you have many regions in one region server, you need to set this larger to compact faster for better read performance.

• Assignee: Zheng Hu
• Reporter: Zheng Hu
• Created: 23/Jan/19 15:37
• Updated: 03/May/19 20:35
• Resolved: 01/Feb/19 02:39

Description

In RegionServicesForStores, we can see :

  private static final int POOL_SIZE = 10;
  private static final ThreadPoolExecutor INMEMORY_COMPACTION_POOL =
      new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, 60, TimeUnit.SECONDS,
          new LinkedBlockingQueue<>(),
          new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
              String name = Thread.currentThread().getName() + "-inmemoryCompactions-" +
                  System.currentTimeMillis();
              return new Thread(r, name);
            }
          });

The pool size should be configurable, because if many regions on a rs, the default 10 threads will be not enough.

Issue Links

• breaks: HBASE-21840 TestHRegionWithInMemoryFlush fails with NPE
• links to: Review board