Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.95.2
    • Fix Version/s: 0.95.0
    • Component/s: Compaction
    • Labels:
      None
    • Release Note:
      Hide
      The CompactionTool works at file-system level, so the table should be disabled.

      The compaction process uses the same hbase-site.xml configuration property used by the server, like
      "hbase.hstore.compactionThreshold" & co.

      You can compact the whole table or just a single region or family,
      and the input of the CompactionTool is a fs path.

      You can run the compaction as a MapReduce Job, or as a local process.
      Each family can be compacted in parallel if you use the -mapreduce option.

      To compact "TestTable" family "cf1" of region "e450da04b1a10099b618bec031e0f951"
      bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool hdfs:///hbase/TestTable/e450da04b1a10099b618bec031e0f951/cf1

      To compact all the families of region "e450da04b1a10099b618bec031e0f951":
      bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool hdfs:///hbase/TestTable/e450da04b1a10099b618bec031e0f951

      To compact all regions and family of the Table:
      bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool -mapred hdfs:///hbase/TestTable
      Show
      The CompactionTool works at file-system level, so the table should be disabled. The compaction process uses the same hbase-site.xml configuration property used by the server, like "hbase.hstore.compactionThreshold" & co. You can compact the whole table or just a single region or family, and the input of the CompactionTool is a fs path. You can run the compaction as a MapReduce Job, or as a local process. Each family can be compacted in parallel if you use the -mapreduce option. To compact "TestTable" family "cf1" of region "e450da04b1a10099b618bec031e0f951" bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool hdfs:///hbase/TestTable/e450da04b1a10099b618bec031e0f951/cf1 To compact all the families of region "e450da04b1a10099b618bec031e0f951": bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool hdfs:///hbase/TestTable/e450da04b1a10099b618bec031e0f951 To compact all regions and family of the Table: bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool -mapred hdfs:///hbase/TestTable

      Description

      In HBASE-5616, as part of the compaction code refactor, a CompactionTool was added.

      but there are some issues:

      • The tool is under test/
      • mockito is required, so the "test" scope should be removed from the pom.xml, otherwise the tool doesn't start
      • The mock, used by the tool, is mocking HRegion.getRegionInfo() but some code (Store) uses HRegion.regionInfo directly HStore.java#L2021, HStore.java#L1389, HStore.java#L1402 and you end up with a NPE in the tool.
      • The Mocked Store uses a dummy family and the compacted files doesn't get the same family properties specified (compression, encoding, ...)
      • at the end of compaction CompactionTool.java#L155, on by default, the compaction file is removed (note that the compacted one are already removed inside the store.compact()... and you end up with an empty dir, if you compact everything.

      I've fixed some stuff and added support to:

      • Run the compaction as a MR Job
      • Specify a Table (compact each region/family)
      • Specify a Region (compact each family)
      • Specify a Family (as before)
      1. HBASE-7253-v0.patch
        31 kB
        Matteo Bertozzi
      2. HBASE-7253-v1.patch
        32 kB
        Matteo Bertozzi
      There are no Sub-Tasks for this issue.

        Activity

        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/)
        HBASE-7253 Backport Compaction Tool to 0.94; REVERT (Revision 1417740)
        HBASE-7253 Backport Compaction Tool to 0.94; includes HBASE-5616, HBASE-5693, HBASE-6327, HBASE-7253 (Revision 1417559)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java

        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/ ) HBASE-7253 Backport Compaction Tool to 0.94; REVERT (Revision 1417740) HBASE-7253 Backport Compaction Tool to 0.94; includes HBASE-5616 , HBASE-5693 , HBASE-6327 , HBASE-7253 (Revision 1417559) Result = FAILURE stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #86 (See https://builds.apache.org/job/HBase-0.94-security/86/)
        HBASE-7253 Backport Compaction Tool to 0.94; REVERT (Revision 1417740)
        HBASE-7253 Backport Compaction Tool to 0.94; includes HBASE-5616, HBASE-5693, HBASE-6327, HBASE-7253 (Revision 1417559)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java

        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #86 (See https://builds.apache.org/job/HBase-0.94-security/86/ ) HBASE-7253 Backport Compaction Tool to 0.94; REVERT (Revision 1417740) HBASE-7253 Backport Compaction Tool to 0.94; includes HBASE-5616 , HBASE-5693 , HBASE-6327 , HBASE-7253 (Revision 1417559) Result = SUCCESS stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #612 (See https://builds.apache.org/job/HBase-0.94/612/)
        HBASE-7253 Backport Compaction Tool to 0.94; REVERT (Revision 1417740)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #612 (See https://builds.apache.org/job/HBase-0.94/612/ ) HBASE-7253 Backport Compaction Tool to 0.94; REVERT (Revision 1417740) Result = FAILURE stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #611 (See https://builds.apache.org/job/HBase-0.94/611/)
        HBASE-7253 Backport Compaction Tool to 0.94; includes HBASE-5616, HBASE-5693, HBASE-6327, HBASE-7253 (Revision 1417559)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #611 (See https://builds.apache.org/job/HBase-0.94/611/ ) HBASE-7253 Backport Compaction Tool to 0.94; includes HBASE-5616 , HBASE-5693 , HBASE-6327 , HBASE-7253 (Revision 1417559) Result = FAILURE stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
        Hide
        stack added a comment -

        Committed to 0.94 as part of HBASE-7253

        Show
        stack added a comment - Committed to 0.94 as part of HBASE-7253
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #284 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/284/)
        HBASE-7253 CompactionTool (Revision 1416657)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce
        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #284 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/284/ ) HBASE-7253 CompactionTool (Revision 1416657) Result = FAILURE stack : Files : /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        Hide
        stack added a comment -

        Matteo Bertozzi Thanks for fixing up the release note.

        Show
        stack added a comment - Matteo Bertozzi Thanks for fixing up the release note.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #3588 (See https://builds.apache.org/job/HBase-TRUNK/3588/)
        HBASE-7253 CompactionTool (Revision 1416657)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce
        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #3588 (See https://builds.apache.org/job/HBase-TRUNK/3588/ ) HBASE-7253 CompactionTool (Revision 1416657) Result = FAILURE stack : Files : /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/mapreduce/JobUtil.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
        Hide
        stack added a comment -

        Committed to trunk. Thanks for the patch Matteo. Make new issue for 0.94 since it'll be a little different. Good stuff.

        Show
        stack added a comment - Committed to trunk. Thanks for the patch Matteo. Make new issue for 0.94 since it'll be a little different. Good stuff.
        Hide
        Matteo Bertozzi added a comment -

        v1 patch with fixes to missing options explanation.

        Show
        Matteo Bertozzi added a comment - v1 patch with fixes to missing options explanation.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12555653/HBASE-7253-v0.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        -1 javadoc. The javadoc tool appears to have generated 101 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12555653/HBASE-7253-v0.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 101 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3432//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12555653/HBASE-7253-v0.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        -1 javadoc. The javadoc tool appears to have generated 101 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12555653/HBASE-7253-v0.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 101 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3428//console This message is automatically generated.
        Hide
        stack added a comment -

        It looks great Matteo. You have actually used this tool in production already, is that not true: i.e. it works (smile).

        Would suggest you add to the usage explanation of -compactOnce and -mapred args. I'd image -mapred means run it as a mapreduce job and if this flag is not passed, then we will run it locally. Not sure of -compactOnce. Would suggest you also add the usage as the release note on this issue so folks can learn about this new facility and that its easy to use.

        Good stuff Matteo.

        Show
        stack added a comment - It looks great Matteo. You have actually used this tool in production already, is that not true: i.e. it works (smile). Would suggest you add to the usage explanation of -compactOnce and -mapred args. I'd image -mapred means run it as a mapreduce job and if this flag is not passed, then we will run it locally. Not sure of -compactOnce. Would suggest you also add the usage as the release note on this issue so folks can learn about this new facility and that its easy to use. Good stuff Matteo.

          People

          • Assignee:
            Matteo Bertozzi
            Reporter:
            Matteo Bertozzi
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development