HBase
  1. HBase
  2. HBASE-6059

Replaying recovered edits would make deleted data exist again

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When we replay recovered edits, we used the minSeqId of Store, It may cause deleted data appeared again.

      Let's see how it happens. Suppose the region with two families(cf1,cf2)

      1.put one data to the region (put r1,cf1:q1,v1)

      2.move the region from server A to server B.

      3.delete the data put by step 1(delete r1)

      4.flush this region.

      5.make major compaction for this region

      6.move the region from server B to server A.

      7.Abort server A

      8.After the region is online, we could get the deleted data(r1,cf1:q1,v1)
      (When we replay recovered edits, we used the minSeqId of Store, because cf2 has no store files, so its seqId is 0, so the edit log of put data will be replayed to the region)

      1. HBASE-6059v5.patch
        19 kB
        chunhui shen
      2. HBASE-6059v4.patch
        18 kB
        chunhui shen
      3. HBASE-6059v3.patch
        16 kB
        chunhui shen
      4. HBASE-6059v2.patch
        17 kB
        chunhui shen
      5. HBASE-6059-testcase.patch
        6 kB
        chunhui shen
      6. HBASE-6059.patch
        16 kB
        chunhui shen
      7. 6059v7-94.patch
        19 kB
        chunhui shen
      8. 6059v7.txt
        19 kB
        stack
      9. 6059v7.txt
        19 kB
        Ted Yu
      10. 6059v6.txt
        19 kB
        Ted Yu

        Issue Links

          Activity

          Hide
          chunhui shen added a comment -

          I have written the test case to reproduce the issue

          Show
          chunhui shen added a comment - I have written the test case to reproduce the issue
          Hide
          chunhui shen added a comment -

          In the solution patch, I use Map<byte[], Long> maxSeqIdInStores to save each store's maxSeqId,
          So, when replaying edit logs, we skip the edits for different stores accoring to its own maxSeqId

          Show
          chunhui shen added a comment - In the solution patch, I use Map<byte[], Long> maxSeqIdInStores to save each store's maxSeqId, So, when replaying edit logs, we skip the edits for different stores accoring to its own maxSeqId
          Hide
          ramkrishna.s.vasudevan added a comment - - edited

          @Chunhui
          This is a damn good one. But still i find one problem is there in this. A similar type of problem that you have reported. Pls correct me if am wrong.
          In the same test case in the place where you are deleting the row 'r1' if i delete the row 'r2' also

          {edit}
          In the same test case in the place where you are deleting the row 'r1' if i delete the row 'r' also{edit}
          del = new Delete(Bytes.toBytes("r"));
              htable.delete(del);
              resultScanner = htable.getScanner(new Scan());
              count = 0;
              while (resultScanner.next() != null) {
                count++;
              }
          

          Now my seq id from the store files will be 0 only as nothing to get after major compaction. So still the same problem is occuring. I tried to simulate this with the same test case that you added.
          May be we need someother way to know that the edit has been deleted out by a major compaction? Because as i see this problem that without major compaction there is no issue at all.

          Show
          ramkrishna.s.vasudevan added a comment - - edited @Chunhui This is a damn good one. But still i find one problem is there in this. A similar type of problem that you have reported. Pls correct me if am wrong. In the same test case in the place where you are deleting the row 'r1' if i delete the row 'r2' also {edit} In the same test case in the place where you are deleting the row 'r1' if i delete the row 'r' also{edit} del = new Delete(Bytes.toBytes( "r" )); htable.delete(del); resultScanner = htable.getScanner( new Scan()); count = 0; while (resultScanner.next() != null ) { count++; } Now my seq id from the store files will be 0 only as nothing to get after major compaction. So still the same problem is occuring. I tried to simulate this with the same test case that you added. May be we need someother way to know that the edit has been deleted out by a major compaction? Because as i see this problem that without major compaction there is no issue at all.
          Hide
          ramkrishna.s.vasudevan added a comment -

          On major compaction if all the entries in the store file is deleted we don't write any new store file and hence this problem happens. To address this can we still have one empty store file but with its meta data alone atleast to get what was the seq id compacted that resulted in that store file.
          This will help us to overcome the problem mentioned above? Pls provide your suggestions.

          Show
          ramkrishna.s.vasudevan added a comment - On major compaction if all the entries in the store file is deleted we don't write any new store file and hence this problem happens. To address this can we still have one empty store file but with its meta data alone atleast to get what was the seq id compacted that resulted in that store file. This will help us to overcome the problem mentioned above? Pls provide your suggestions.
          Hide
          chunhui shen added a comment -

          @ram
          Yes, I have also considered that all the entries in the store file is deleted and we don't write any new store file.
          But, could we generate one empty store file with its meta data alone? Let me do a try first.

          Show
          chunhui shen added a comment - @ram Yes, I have also considered that all the entries in the store file is deleted and we don't write any new store file. But, could we generate one empty store file with its meta data alone? Let me do a try first.
          Hide
          ramkrishna.s.vasudevan added a comment -

          @Chunhui
          I tried this

                if(writer == null && majorCompaction){
                  writer = store.createWriterInTmp(maxKeyCount, compactionCompression,
                      true);
                }
                if (writer != null) {
                  writer.appendMetadata(maxId, majorCompaction);
                  writer.close();
                }
          

          It actually allows me . But again not sure if all the testcases can run and any other scenario is possible with this. Just a wild try and it worked.
          Also i noted one thing TestStore.testEmptyStoreFile(). JFYI.
          Good on you Chunhui.

          Show
          ramkrishna.s.vasudevan added a comment - @Chunhui I tried this if (writer == null && majorCompaction){ writer = store.createWriterInTmp(maxKeyCount, compactionCompression, true ); } if (writer != null ) { writer.appendMetadata(maxId, majorCompaction); writer.close(); } It actually allows me . But again not sure if all the testcases can run and any other scenario is possible with this. Just a wild try and it worked. Also i noted one thing TestStore.testEmptyStoreFile(). JFYI. Good on you Chunhui.
          Hide
          Ted Yu added a comment -

          If majorCompaction is false, we still need to check !kvs.isEmpty(), right ?

          Show
          Ted Yu added a comment - If majorCompaction is false, we still need to check !kvs.isEmpty(), right ?
          Hide
          chunhui shen added a comment -

          @ram
          I think create empty store file is available and easy to solve the problem. Of course, we should pass the testcase.

          If empty store file is not available, I think we could retain one or more kvs(such as, a delete type) in the majorcompaction.

          Show
          chunhui shen added a comment - @ram I think create empty store file is available and easy to solve the problem. Of course, we should pass the testcase. If empty store file is not available, I think we could retain one or more kvs(such as, a delete type) in the majorcompaction.
          Hide
          chunhui shen added a comment -

          If majorCompaction is false, we still need to check !kvs.isEmpty(), right?

          Yes, I think just about majorCompaction, minorCompaction will retain delete type, there is no problem.

          Show
          chunhui shen added a comment - If majorCompaction is false, we still need to check !kvs.isEmpty(), right? Yes, I think just about majorCompaction, minorCompaction will retain delete type, there is no problem.
          Hide
          ramkrishna.s.vasudevan added a comment -

          I think only major compaction could lead us to this problem which probabaly deletes it.
          Incase of TTL expiry of all the entries in a store file, can we have this scenario of empty StoreFile getting created on minor or major compaction? I think creating empty store file should be fine. Lets take others input also on this?

          Show
          ramkrishna.s.vasudevan added a comment - I think only major compaction could lead us to this problem which probabaly deletes it. Incase of TTL expiry of all the entries in a store file, can we have this scenario of empty StoreFile getting created on minor or major compaction? I think creating empty store file should be fine. Lets take others input also on this?
          Hide
          chunhui shen added a comment -

          @ram
          Yes, I forget to consider TTL.
          So we should create empty file if no kvs after minor or major compaction.

          Show
          chunhui shen added a comment - @ram Yes, I forget to consider TTL. So we should create empty file if no kvs after minor or major compaction.
          Hide
          chunhui shen added a comment -

          In the patchv2, fix the issue in the case: no kvs after compaction.(Using creating empty storefile mentioned by Ram )

          Show
          chunhui shen added a comment - In the patchv2, fix the issue in the case: no kvs after compaction.(Using creating empty storefile mentioned by Ram )
          Hide
          Ted Yu added a comment -

          Nice work.
          Minor comments:

          -    // Get minimum of the maxSeqId across all the store.
          +    // Get the maxSeqId for each store.
          

          The second line above seems redundant - same sentence appears later.

          -    if (files == null || files.isEmpty()) return seqid;
          +    if (files == null || files.isEmpty())
          +      return seqid;
          

          The above change is not necessary.

          +    for (Map.Entry<byte[], Long> maxSeqIdInStore : maxSeqIdInStores.entrySet()) {
          +      msg = msg + "; store=" + Bytes.toString(maxSeqIdInStore.getKey())
          +          + ",minSequenceid=" + maxSeqIdInStore.getValue();
          

          Do we really need the above loop (there could be many stores, making the log very long) ?

          +      if (serverInfo != null && serverInfo.equals(destServer.getServerName()))
          +        break;
          

          break can be moved to the same line as if.

          Show
          Ted Yu added a comment - Nice work. Minor comments: - // Get minimum of the maxSeqId across all the store. + // Get the maxSeqId for each store. The second line above seems redundant - same sentence appears later. - if (files == null || files.isEmpty()) return seqid; + if (files == null || files.isEmpty()) + return seqid; The above change is not necessary. + for (Map.Entry< byte [], Long > maxSeqIdInStore : maxSeqIdInStores.entrySet()) { + msg = msg + "; store=" + Bytes.toString(maxSeqIdInStore.getKey()) + + ",minSequenceid=" + maxSeqIdInStore.getValue(); Do we really need the above loop (there could be many stores, making the log very long) ? + if (serverInfo != null && serverInfo.equals(destServer.getServerName())) + break ; break can be moved to the same line as if.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12528557/HBASE-6059v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 35 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.TestColumnSeeking

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1957//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1957//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1957//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528557/HBASE-6059v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 35 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestColumnSeeking Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1957//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1957//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1957//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Patch v3 with Ted's comment,

          And I run the TestColumnSeeking, it passed

          Show
          chunhui shen added a comment - Patch v3 with Ted's comment, And I run the TestColumnSeeking, it passed
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12528568/HBASE-6059v3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.replication.TestReplication
          org.apache.hadoop.hbase.client.TestShell
          org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
          org.apache.hadoop.hbase.regionserver.TestStore
          org.apache.hadoop.hbase.replication.TestMasterReplication

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1958//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1958//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1958//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528568/HBASE-6059v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.client.TestShell org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.regionserver.TestStore org.apache.hadoop.hbase.replication.TestMasterReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1958//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1958//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1958//console This message is automatically generated.
          Hide
          ramkrishna.s.vasudevan added a comment -

          @Chunhui
          Can you run the entire testsuite? HadoopQA will not run all the testcases. Similarly for HBASE-6065.

          Show
          ramkrishna.s.vasudevan added a comment - @Chunhui Can you run the entire testsuite? HadoopQA will not run all the testcases. Similarly for HBASE-6065 .
          Hide
          chunhui shen added a comment -

          @ram
          Sorry, we don't have the test environment for trunk.
          I think TestStore is failed by empty storefil, do we modify the testcase or the patch?

          Show
          chunhui shen added a comment - @ram Sorry, we don't have the test environment for trunk. I think TestStore is failed by empty storefil, do we modify the testcase or the patch?
          Hide
          Ted Yu added a comment -

          I think test case should be modified.

          Show
          Ted Yu added a comment - I think test case should be modified.
          Hide
          chunhui shen added a comment -

          Patch v4 modified TestStore#testDeleteExpiredStoreFiles since we could create empty store file now.

          Show
          chunhui shen added a comment - Patch v4 modified TestStore#testDeleteExpiredStoreFiles since we could create empty store file now.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12528693/HBASE-6059v4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.replication.TestReplication
          org.apache.hadoop.hbase.client.TestShell
          org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
          org.apache.hadoop.hbase.replication.TestMasterReplication
          org.apache.hadoop.hbase.master.TestSplitLogManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1962//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1962//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1962//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528693/HBASE-6059v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.client.TestShell org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.replication.TestMasterReplication org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1962//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1962//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1962//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Find the bug which cause TestShell failed.

          Store#rowAtOrBeforeFromStoreFile,
          we should do it considering empty store file now, else it will throw NPW

          Store#rowAtOrBeforeFromStoreFile
          private void rowAtOrBeforeFromStoreFile(final StoreFile f,
                                                    final GetClosestRowBeforeTracker state)
                throws IOException {
              StoreFile.Reader r = f.getReader();
              if (r == null) {
                LOG.warn("StoreFile " + f + " has a null Reader");
                return;
              }
               }
          +    if (r.getEntries() == 0) {
          +      LOG.warn("StoreFile " + f + " is a empty store file");
          +      return;
          +    }
              // TODO: Cache these keys rather than make each time?
              byte [] fk = r.getFirstKey();
          

          Mmodify it in the patchV5 and passed the TestShell now.

          Show
          chunhui shen added a comment - Find the bug which cause TestShell failed. Store#rowAtOrBeforeFromStoreFile, we should do it considering empty store file now, else it will throw NPW Store#rowAtOrBeforeFromStoreFile private void rowAtOrBeforeFromStoreFile( final StoreFile f, final GetClosestRowBeforeTracker state) throws IOException { StoreFile.Reader r = f.getReader(); if (r == null ) { LOG.warn( "StoreFile " + f + " has a null Reader" ); return ; } } + if (r.getEntries() == 0) { + LOG.warn( "StoreFile " + f + " is a empty store file" ); + return ; + } // TODO: Cache these keys rather than make each time? byte [] fk = r.getFirstKey(); Mmodify it in the patchV5 and passed the TestShell now.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12528713/HBASE-6059v5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.replication.TestReplication
          org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
          org.apache.hadoop.hbase.replication.TestMasterReplication

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1963//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1963//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1963//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528713/HBASE-6059v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.replication.TestMasterReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1963//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1963//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1963//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          Patch v6 modifies the comment in TestStore.java

          Show
          Ted Yu added a comment - Patch v6 modifies the comment in TestStore.java
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12528774/6059v6.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.replication.TestReplication
          org.apache.hadoop.hbase.master.TestSplitLogManager
          org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
          org.apache.hadoop.hbase.replication.TestMasterReplication

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1967//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1967//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1967//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528774/6059v6.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 34 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.master.TestSplitLogManager org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.replication.TestMasterReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1967//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1967//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1967//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          I ran TestSplitLogManager with patch v6 and it passed.

          Show
          Ted Yu added a comment - I ran TestSplitLogManager with patch v6 and it passed.
          Hide
          ramkrishna.s.vasudevan added a comment -

          @Stack
          Could you take a look at this solution and and patch?
          @Ted
          Is this ok to be committed? My concern was with creating an empty store file now.

          Show
          ramkrishna.s.vasudevan added a comment - @Stack Could you take a look at this solution and and patch? @Ted Is this ok to be committed? My concern was with creating an empty store file now.
          Hide
          Ted Yu added a comment -

          I would listen to opinion from people who are more familiar with store files about the current solution.

          Show
          Ted Yu added a comment - I would listen to opinion from people who are more familiar with store files about the current solution.
          Hide
          ramkrishna.s.vasudevan added a comment -

          @Lars
          Can you have a look at this?

          Show
          ramkrishna.s.vasudevan added a comment - @Lars Can you have a look at this?
          Hide
          stack added a comment -

          +1

          Great work @Chunhui. Nice find. Nice tests too.

          Show
          stack added a comment - +1 Great work @Chunhui. Nice find. Nice tests too.
          Hide
          Ted Yu added a comment -

          Patch v7 is rebased on trunk.

          Show
          Ted Yu added a comment - Patch v7 is rebased on trunk.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12530272/6059v7.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.master.TestAssignmentManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2063//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2063//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530272/6059v7.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestAssignmentManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2063//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2063//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12530290/6059v7.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2066//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2066//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530290/6059v7.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2066//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2066//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12530313/6059v7.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.master.TestSplitLogManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2067//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2067//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530313/6059v7.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2067//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2067//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          I run the failed test org.apache.hadoop.hbase.master.TestSplitLogManager.testOrphanTaskAcquisition on local PC, it passed.

          Show
          chunhui shen added a comment - I run the failed test org.apache.hadoop.hbase.master.TestSplitLogManager.testOrphanTaskAcquisition on local PC, it passed.
          Hide
          stack added a comment -

          Applied to trunk. Thanks for the patch Chunhui and for all who helped get it in (Ram and Ted).

          Show
          stack added a comment - Applied to trunk. Thanks for the patch Chunhui and for all who helped get it in (Ram and Ted).
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2961 (See https://builds.apache.org/job/HBase-TRUNK/2961/)
          HBASE-6059 Replaying recovered edits would make deleted data exist again (Revision 1344554)

          Result = FAILURE
          stack :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2961 (See https://builds.apache.org/job/HBase-TRUNK/2961/ ) HBASE-6059 Replaying recovered edits would make deleted data exist again (Revision 1344554) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #34 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/34/)
          HBASE-6059 Replaying recovered edits would make deleted data exist again (Revision 1344554)

          Result = FAILURE
          stack :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #34 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/34/ ) HBASE-6059 Replaying recovered edits would make deleted data exist again (Revision 1344554) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Hide
          Lars Hofhansl added a comment -

          Should we have this in 0.94.1?

          Show
          Lars Hofhansl added a comment - Should we have this in 0.94.1?
          Hide
          ramkrishna.s.vasudevan added a comment -

          +1 for getting it in 0.94.

          Show
          ramkrishna.s.vasudevan added a comment - +1 for getting it in 0.94.
          Hide
          Lars Hofhansl added a comment -

          Wanna make a patch Ram? I am technically on vacation.

          Show
          Lars Hofhansl added a comment - Wanna make a patch Ram? I am technically on vacation.
          Hide
          ramkrishna.s.vasudevan added a comment -

          Ok..I can make it over the weekend..

          Show
          ramkrishna.s.vasudevan added a comment - Ok..I can make it over the weekend..
          Hide
          chunhui shen added a comment -

          Updating patch for 0.94

          Show
          chunhui shen added a comment - Updating patch for 0.94
          Hide
          chunhui shen added a comment -

          @Lars @ram
          I has done for 0.94~~

          Show
          chunhui shen added a comment - @Lars @ram I has done for 0.94~~
          Hide
          ramkrishna.s.vasudevan added a comment -

          @Chunhui
          Thanks a lot.. If Lars can't commit it, I can review and commit it to 0.94.
          @Lars
          Is it fine?

          Show
          ramkrishna.s.vasudevan added a comment - @Chunhui Thanks a lot.. If Lars can't commit it, I can review and commit it to 0.94. @Lars Is it fine?
          Hide
          Lars Hofhansl added a comment -

          I don't grok the patch in all detail, but looks good, and same as trunk patch. So +1.
          @Stack: Maybe you can have a safety look...?

          Show
          Lars Hofhansl added a comment - I don't grok the patch in all detail, but looks good, and same as trunk patch. So +1. @Stack: Maybe you can have a safety look...?
          Hide
          ramkrishna.s.vasudevan added a comment -

          Discussed with Lars too.. We can backport this to 0.94.

          Show
          ramkrishna.s.vasudevan added a comment - Discussed with Lars too.. We can backport this to 0.94.
          Hide
          Lars Hofhansl added a comment -

          Stack had opened HBASE-6628 for the backport.
          Let's do the 0.94 work there.

          Show
          Lars Hofhansl added a comment - Stack had opened HBASE-6628 for the backport. Let's do the 0.94 work there.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #145 (See https://builds.apache.org/job/HBase-0.94-security/145/)
          HBASE-7210 Backport HBASE-6059 to 0.94 (Ted Yu) (Revision 1483992)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #145 (See https://builds.apache.org/job/HBase-0.94-security/145/ ) HBASE-7210 Backport HBASE-6059 to 0.94 (Ted Yu) (Revision 1483992) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #988 (See https://builds.apache.org/job/HBase-0.94/988/)
          HBASE-7210 Backport HBASE-6059 to 0.94 (Ted Yu) (Revision 1483992)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #988 (See https://builds.apache.org/job/HBase-0.94/988/ ) HBASE-7210 Backport HBASE-6059 to 0.94 (Ted Yu) (Revision 1483992) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
          Hide
          stack added a comment -

          Marking closed.

          Show
          stack added a comment - Marking closed.

            People

            • Assignee:
              chunhui shen
              Reporter:
              chunhui shen
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development