Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20188 [TESTING] Performance
  3. HBASE-21764

Size of in-memory compaction thread pool should be configurable

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Introduced an new config key in this issue: hbase.regionserver.inmemory.compaction.pool.size. the default value would be 10. you can configure this to set the pool size of in-memory compaction pool. Note that all memstores in one region server will share the same pool, so if you have many regions in one region server, you need to set this larger to compact faster for better read performance.
      Show
      Introduced an new config key in this issue: hbase.regionserver.inmemory.compaction.pool.size. the default value would be 10. you can configure this to set the pool size of in-memory compaction pool. Note that all memstores in one region server will share the same pool, so if you have many regions in one region server, you need to set this larger to compact faster for better read performance.

      Description

      In RegionServicesForStores, we can see :

        private static final int POOL_SIZE = 10;
        private static final ThreadPoolExecutor INMEMORY_COMPACTION_POOL =
            new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, 60, TimeUnit.SECONDS,
                new LinkedBlockingQueue<>(),
                new ThreadFactory() {
                  @Override
                  public Thread newThread(Runnable r) {
                    String name = Thread.currentThread().getName() + "-inmemoryCompactions-" +
                        System.currentTimeMillis();
                    return new Thread(r, name);
                  }
                });
      

      The pool size should be configurable, because if many regions on a rs, the default 10 threads will be not enough.

        Attachments

        1. HBASE-21764.v10.patch
          18 kB
          Zheng Hu
        2. HBASE-21764.v9.patch
          17 kB
          Zheng Hu
        3. HBASE-21764.v8.patch
          16 kB
          Zheng Hu
        4. HBASE-21764.v7.patch
          16 kB
          Zheng Hu
        5. HBASE-21764.v6.patch
          13 kB
          Zheng Hu
        6. HBASE-21764.v5.patch
          9 kB
          Zheng Hu
        7. HBASE-21764.v4.patch
          8 kB
          Zheng Hu
        8. HBASE-21764.v3.patch
          7 kB
          Zheng Hu
        9. HBASE-21764.v2.patch
          5 kB
          Zheng Hu
        10. HBASE-21764.v1.patch
          5 kB
          Zheng Hu

          Issue Links

            Activity

              People

              • Assignee:
                openinx Zheng Hu
                Reporter:
                openinx Zheng Hu
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: