HBase
  1. HBase
  2. HBASE-10118

Major compact keeps deletes with future timestamps

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      hbase.hstore.time.to.purge.deletes has been changed; if it is not set, or set to 0, all delete markers including those with future timestamp are purged during the later major compaction. Otherwise, a delete marker is kept until the major compaction after marker's timestamp + this setting.
      Show
      hbase.hstore.time.to.purge.deletes has been changed; if it is not set, or set to 0, all delete markers including those with future timestamp are purged during the later major compaction. Otherwise, a delete marker is kept until the major compaction after marker's timestamp + this setting.

      Description

      Hello!

      During migration from HBase 0.90.6 to 0.94.6 we found changed behaviour in how major compact handles delete markers with timestamps in the future. Before HBASE-4721 major compact purged deletes regardless of their timestamp. Newer versions keep them in HFile until timestamp not reached.

      I guess this happened due to new check in ScanQueryMatcher EnvironmentEdgeManager.currentTimeMillis() - timestamp) <= timeToPurgeDeletes.

      This can be worked around by specifying large negative value in hbase.hstore.time.to.purge.deletes option, but, unfortunately, negative values are pulled up to zero by Math.max in HStore.java.

      Maybe, we are trying to do something weird by specifing delete timestamp in future, but HBASE-4721 definitely breaks old behaviour we rely on.

      Steps to reproduce this:

      put 'test', 'delmeRow', 'delme:something', 'hello'
      flush 'test'
      delete 'test', 'delmeRow', 'delme:something', 1394161431061
      flush 'test'
      major_compact 'test'
      

      Before major_compact we have two hfiles with the following:

      first:
      K: delmeRow/delme:something/1384161431061/Put/vlen=5/ts=0
      
      second:
      K: delmeRow/delme:something/1394161431061/DeleteColumn/vlen=0/ts=0
      

      After major compact we get the following:

      K: delmeRow/delme:something/1394161431061/DeleteColumn/vlen=0/ts=0
      

      In our installation, we resolved this by removing Math.max and setting hbase.hstore.time.to.purge.deletes to Integer.MIN_VALUE, which purges delete markers, and it looks like a solution. But, maybe, there are better approach.

      1. HBASE-10118-trunk-v3.diff
        6 kB
        Liu Shaohui
      2. HBASE-10118-trunk-v2.diff
        5 kB
        Liu Shaohui
      3. HBASE-10118-trunk-v1.diff
        4 kB
        Liu Shaohui
      4. HBASE-10118-0.94-v1.diff
        4 kB
        Liu Shaohui

        Activity

        Hide
        Jean-Marc Spaggiari added a comment -

        Hum. Maybe we should make this optional? Like, hbase.hstore.time.to.purge.future.deletes=true or false?

        Show
        Jean-Marc Spaggiari added a comment - Hum. Maybe we should make this optional? Like, hbase.hstore.time.to.purge.future.deletes=true or false?
        Hide
        Sergey Shelukhin added a comment -

        Yeah there's no good justification either way... strange behavior of custom TSes during compaction is currently a given, e.g. if you put custom TS=1, delete TS=3, put TS=2 the result depends on compaction timing, if compaction happens after delete 2nd put will survive potentially forever, and if it happens after 2nd put it will be dropped.
        +1 on configurable

        Show
        Sergey Shelukhin added a comment - Yeah there's no good justification either way... strange behavior of custom TSes during compaction is currently a given, e.g. if you put custom TS=1, delete TS=3, put TS=2 the result depends on compaction timing, if compaction happens after delete 2nd put will survive potentially forever, and if it happens after 2nd put it will be dropped. +1 on configurable
        Hide
        Lars Hofhansl added a comment -

        When the special delete TTL is not set (or set to 0) it should not have any effect.

        Show
        Lars Hofhansl added a comment - When the special delete TTL is not set (or set to 0) it should not have any effect.
        Hide
        Liu Shaohui added a comment -

        Agree with Lars Hofhansl
        When the special delete TTL is not set (or set to 0) , it should not have any effect.

        Jean-Marc Spaggiari Sergey Shelukhin
        If we add an option hbase.hstore.time.to.purge.future.deletes.
        it will bring conflicts when user set the delete TTL and hbase.hstore.time.to.purge.future.deletes = true, because we don't know when the future delete kvs are inserted and when to delete these future kvs.

        So i think if the special delete TTL is not set, we keep the behavior as before and future deleted kvs are purged during major compaction.

        And when the special delete TTL is set, the future deleted kvs are kept util
        kv.timestap() + delete TTL.

        Show
        Liu Shaohui added a comment - Agree with Lars Hofhansl When the special delete TTL is not set (or set to 0) , it should not have any effect. Jean-Marc Spaggiari Sergey Shelukhin If we add an option hbase.hstore.time.to.purge.future.deletes. it will bring conflicts when user set the delete TTL and hbase.hstore.time.to.purge.future.deletes = true, because we don't know when the future delete kvs are inserted and when to delete these future kvs. So i think if the special delete TTL is not set, we keep the behavior as before and future deleted kvs are purged during major compaction. And when the special delete TTL is set, the future deleted kvs are kept util kv.timestap() + delete TTL.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12636677/HBASE-10118-trunk-v1.diff
        against trunk revision .
        ATTACHMENT ID: 12636677

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636677/HBASE-10118-trunk-v1.diff against trunk revision . ATTACHMENT ID: 12636677 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9089//console This message is automatically generated.
        Hide
        Honghua Feng added a comment -

        So i think if the special delete TTL is not set, we keep the behavior as before and future deleted kvs are purged during major compaction. And when the special delete TTL is set, the future deleted kvs are kept until kv.timestap() + delete TTL.

        Delete TTL means 'time to purge deletes', right?

        Make sense and patch looks good to me. +1

        Show
        Honghua Feng added a comment - So i think if the special delete TTL is not set, we keep the behavior as before and future deleted kvs are purged during major compaction. And when the special delete TTL is set, the future deleted kvs are kept until kv.timestap() + delete TTL. Delete TTL means 'time to purge deletes', right? Make sense and patch looks good to me. +1
        Hide
        Ted Yu added a comment -

        w.r.t. the new test, please close HTable at the end of the test.

        +     // waiting for the major compaction completed
        

        Typo: 'completed' -> 'to complete'
        See the following code in TestNamespaceUpgrade for another approach of waiting for compaction to finish:

              TEST_UTIL.getHBaseAdmin().majorCompact(newTableName);
              TEST_UTIL.waitFor(30000, new Waiter.Predicate<IOException>() {
                @Override
                public boolean evaluate() throws IOException {
                  try {
                    return TEST_UTIL.getHBaseAdmin().getCompactionState(newTableName) ==
                        AdminProtos.GetRegionInfoResponse.CompactionState.NONE;
                  } catch (InterruptedException e) {
                    throw new IOException(e);
                  }
                }
              });
        
        Show
        Ted Yu added a comment - w.r.t. the new test, please close HTable at the end of the test. + // waiting for the major compaction completed Typo: 'completed' -> 'to complete' See the following code in TestNamespaceUpgrade for another approach of waiting for compaction to finish: TEST_UTIL.getHBaseAdmin().majorCompact(newTableName); TEST_UTIL.waitFor(30000, new Waiter.Predicate<IOException>() { @Override public boolean evaluate() throws IOException { try { return TEST_UTIL.getHBaseAdmin().getCompactionState(newTableName) == AdminProtos.GetRegionInfoResponse.CompactionState.NONE; } catch (InterruptedException e) { throw new IOException(e); } } });
        Hide
        Liu Shaohui added a comment -

        Thanks for Ted Yu's careful review.

        Show
        Liu Shaohui added a comment - Thanks for Ted Yu 's careful review.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12636855/HBASE-10118-trunk-v2.diff
        against trunk revision .
        ATTACHMENT ID: 12636855

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 6 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636855/HBASE-10118-trunk-v2.diff against trunk revision . ATTACHMENT ID: 12636855 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. -1 javadoc . The javadoc tool appears to have generated 6 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9095//console This message is automatically generated.
        Hide
        Liu Shaohui added a comment -

        After a look at the failed build, the javadoc and findbugs warnings, and the failed test have no relations with this patch.

        Show
        Liu Shaohui added a comment - After a look at the failed build, the javadoc and findbugs warnings, and the failed test have no relations with this patch.
        Hide
        Liu Shaohui added a comment -

        Anyone help to push this issue? Ted Yu Lars Hofhansl

        Show
        Liu Shaohui added a comment - Anyone help to push this issue? Ted Yu Lars Hofhansl
        Hide
        Sergey Shelukhin added a comment -

        +1... can you document new behavior for config setting somewhere

        Show
        Sergey Shelukhin added a comment - +1... can you document new behavior for config setting somewhere
        Hide
        Lars Hofhansl added a comment -

        +1 (including 0.94, this is a real bug)

        Just add a release note here.

        Show
        Lars Hofhansl added a comment - +1 (including 0.94, this is a real bug) Just add a release note here.
        Hide
        Liu Shaohui added a comment -

        Add some documents.

        Sergey Shelukhin
        Could you help to review the document? Some expressions maybe be not accurate.

        Show
        Liu Shaohui added a comment - Add some documents. Sergey Shelukhin Could you help to review the document? Some expressions maybe be not accurate.
        Hide
        Liu Shaohui added a comment -

        Update patch format

        Show
        Liu Shaohui added a comment - Update patch format
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12638201/HBASE-10118-trunk-v3.diff
        against trunk revision .
        ATTACHMENT ID: 12638201

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:

        -1 core zombie tests. There are 1 zombie test(s): at org.apache.hadoop.hbase.mapreduce.TestTableMapReduceBase.testMultiRegionTable(TestTableMapReduceBase.java:96)

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12638201/HBASE-10118-trunk-v3.diff against trunk revision . ATTACHMENT ID: 12638201 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.mapreduce.TestTableMapReduceBase.testMultiRegionTable(TestTableMapReduceBase.java:96) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9174//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12638209/HBASE-10118-trunk-v3.diff
        against trunk revision .
        ATTACHMENT ID: 12638209

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12638209/HBASE-10118-trunk-v3.diff against trunk revision . ATTACHMENT ID: 12638209 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9175//console This message is automatically generated.
        Hide
        Liu Shaohui added a comment -

        Someone help to review the added document in the patch and push this issue?

        Show
        Liu Shaohui added a comment - Someone help to review the added document in the patch and push this issue?
        Hide
        Sergey Shelukhin added a comment -

        There's typo in scenarios; I can commit this later today and fix on commit

        Show
        Sergey Shelukhin added a comment - There's typo in scenarios; I can commit this later today and fix on commit
        Hide
        Sergey Shelukhin added a comment -

        Ok, I committed to trunk. 98 and 96 are coming next. Lars Hofhansl/Liu Shaohui can you guys make a 94 patch

        Show
        Sergey Shelukhin added a comment - Ok, I committed to trunk. 98 and 96 are coming next. Lars Hofhansl / Liu Shaohui can you guys make a 94 patch
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5061 (See https://builds.apache.org/job/HBase-TRUNK/5061/)
        HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584383)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        • /hbase/trunk/src/main/docbkx/book.xml
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5061 (See https://builds.apache.org/job/HBase-TRUNK/5061/ ) HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584383) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/trunk/src/main/docbkx/book.xml
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96 #376 (See https://builds.apache.org/job/hbase-0.96/376/)
        HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584392)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        • /hbase/branches/0.96/src/main/docbkx/book.xml
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96 #376 (See https://builds.apache.org/job/hbase-0.96/376/ ) HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584392) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/branches/0.96/src/main/docbkx/book.xml
        Hide
        Hudson added a comment -

        ABORTED: Integrated in HBase-0.98-on-Hadoop-1.1 #247 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/247/)
        HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584388)

        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        • /hbase/branches/0.98/src/main/docbkx/book.xml
        Show
        Hudson added a comment - ABORTED: Integrated in HBase-0.98-on-Hadoop-1.1 #247 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/247/ ) HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584388) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/branches/0.98/src/main/docbkx/book.xml
        Hide
        Hudson added a comment -

        ABORTED: Integrated in HBase-0.98 #263 (See https://builds.apache.org/job/HBase-0.98/263/)
        HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584388)

        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        • /hbase/branches/0.98/src/main/docbkx/book.xml
        Show
        Hudson added a comment - ABORTED: Integrated in HBase-0.98 #263 (See https://builds.apache.org/job/HBase-0.98/263/ ) HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584388) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/branches/0.98/src/main/docbkx/book.xml
        Hide
        Liu Shaohui added a comment -

        patch for 0.94

        Show
        Liu Shaohui added a comment - patch for 0.94
        Hide
        Liu Shaohui added a comment -

        Thanks for Sergey Shelukhin's review

        Show
        Liu Shaohui added a comment - Thanks for Sergey Shelukhin 's review
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12638620/HBASE-10118-0.94-v1.diff
        against trunk revision .
        ATTACHMENT ID: 12638620

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9198//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12638620/HBASE-10118-0.94-v1.diff against trunk revision . ATTACHMENT ID: 12638620 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9198//console This message is automatically generated.
        Hide
        Lars Hofhansl added a comment -

        Committed to 0.94. Closing.

        Thanks for the patch Liu Shaohui.

        Show
        Lars Hofhansl added a comment - Committed to 0.94. Closing. Thanks for the patch Liu Shaohui .
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94-security #456 (See https://builds.apache.org/job/HBase-0.94-security/456/)
        HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514)

        • /hbase/branches/0.94/src/docbkx/book.xml
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94-security #456 (See https://builds.apache.org/job/HBase-0.94-security/456/ ) HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514) /hbase/branches/0.94/src/docbkx/book.xml /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94-JDK7 #103 (See https://builds.apache.org/job/HBase-0.94-JDK7/103/)
        HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514)

        • /hbase/branches/0.94/src/docbkx/book.xml
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94-JDK7 #103 (See https://builds.apache.org/job/HBase-0.94-JDK7/103/ ) HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514) /hbase/branches/0.94/src/docbkx/book.xml /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #64 (See https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/64/)
        HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514)

        • /hbase/branches/0.94/src/docbkx/book.xml
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #64 (See https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/64/ ) HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514) /hbase/branches/0.94/src/docbkx/book.xml /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94 #1337 (See https://builds.apache.org/job/HBase-0.94/1337/)
        HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514)

        • /hbase/branches/0.94/src/docbkx/book.xml
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94 #1337 (See https://builds.apache.org/job/HBase-0.94/1337/ ) HBASE-10118 Major compact keeps deletes with future timestamps. (Liu Shaohui) (larsh: rev 1584514) /hbase/branches/0.94/src/docbkx/book.xml /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96-hadoop2 #259 (See https://builds.apache.org/job/hbase-0.96-hadoop2/259/)
        HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584392)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
        • /hbase/branches/0.96/src/main/docbkx/book.xml
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96-hadoop2 #259 (See https://builds.apache.org/job/hbase-0.96-hadoop2/259/ ) HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev 1584392) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/branches/0.96/src/main/docbkx/book.xml

          People

          • Assignee:
            Liu Shaohui
            Reporter:
            Max Lapan
          • Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development