HBase
  1. HBase
  2. HBASE-11218

Data loss in HBase standalone mode

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fix for local filesystem data loss in standalone and pseudo-distributed mode.

      Description

      Data loss in HBase standalone mode.
      How to produce it

      1. Start HBase standalone mode.
      2. Create a table using hbase shell.
      3. Scan '.META.' and you will find data in meta table
      4. Kill the HBase process with -9 option
      5. Start the HBase agaion
      6. Scan '.META.' and you will find nothing in meta table.

      There are three main reasons.

      1. FSDataOutputStream.sync should call flush() if the underlying wrapped stream is not Syncable. See HADOOP-8861
      2. writeChecksum is ture in default LocalFileSystem and the ChecksumFSOutputSummer will buffer the data, which make the waledits are not written to os's filesystem with sync method immediately, and those edits will be lost in regionserver's failover.
      3. The MiniZooKeeperCluster deletes the old zk data at startup which maye cause data loss in meta table. The failover procedure is: split pre root regionserver's hlog -> assign root -> split pre meta regionserver's hlog -> assign meta -> split all other regionservers' hlogs -> assign other regions. If there is no data in zookeeper, we will get null for root regionserver and then assign root table. Some data in root table maybe be lost for some root's WalEdits have not been splited and replayed. So does the Meta table.

      I finished the patch for 0.94 and am working on the patch for trunk. Suggestions are welcomed.

      1. 11218v2.098.txt
        7 kB
        stack
      2. HBASE-11218-trunk-v1.diff
        7 kB
        Liu Shaohui
      3. HBASE-11218-trunk-v2.diff
        7 kB
        Liu Shaohui

        Activity

        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #321 (See https://builds.apache.org/job/HBase-0.98/321/)
        HBASE-11272 Backport HBASE-11218 (Data loss in HBase standalone mode) to 0.98 (apurtell: rev 13c7ff669532e18ebf5261734b98c51a4ad69c6b)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #321 (See https://builds.apache.org/job/HBase-0.98/321/ ) HBASE-11272 Backport HBASE-11218 (Data loss in HBase standalone mode) to 0.98 (apurtell: rev 13c7ff669532e18ebf5261734b98c51a4ad69c6b) hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #302 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/302/)
        HBASE-11272 Backport HBASE-11218 (Data loss in HBase standalone mode) to 0.98 (apurtell: rev 13c7ff669532e18ebf5261734b98c51a4ad69c6b)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #302 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/302/ ) HBASE-11272 Backport HBASE-11218 (Data loss in HBase standalone mode) to 0.98 (apurtell: rev 13c7ff669532e18ebf5261734b98c51a4ad69c6b) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        Hide
        Andrew Purtell added a comment -

        Further work can be found on HBASE-11272

        Show
        Andrew Purtell added a comment - Further work can be found on HBASE-11272
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #301 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/301/)
        HBASE-11218 Data loss in HBase standalone mode; REVERT (stack: rev 111ebaaea3bc70888ac27c09d0c8433f66df13e0)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #301 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/301/ ) HBASE-11218 Data loss in HBase standalone mode; REVERT (stack: rev 111ebaaea3bc70888ac27c09d0c8433f66df13e0) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #320 (See https://builds.apache.org/job/HBase-0.98/320/)
        HBASE-11218 Data loss in HBase standalone mode; REVERT (stack: rev 111ebaaea3bc70888ac27c09d0c8433f66df13e0)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #320 (See https://builds.apache.org/job/HBase-0.98/320/ ) HBASE-11218 Data loss in HBase standalone mode; REVERT (stack: rev 111ebaaea3bc70888ac27c09d0c8433f66df13e0) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        Hide
        Andrew Purtell added a comment -

        Let me look what might be needed to put back into 0.98 tomorrow

        Show
        Andrew Purtell added a comment - Let me look what might be needed to put back into 0.98 tomorrow
        Hide
        stack added a comment -

        Reverted from 0.98.

        Show
        stack added a comment - Reverted from 0.98.
        Hide
        Ted Yu added a comment -

        Was the following compilation error (on hadoop-1) related to this JIRA ?

        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hbase-server: Compilation failure
        [ERROR] /home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java:[87,8] cannot find symbol
        [ERROR] symbol  : method setWriteChecksum(boolean)
        [ERROR] location: class org.apache.hadoop.fs.FileSystem
        
        Show
        Ted Yu added a comment - Was the following compilation error (on hadoop-1) related to this JIRA ? [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile ( default -compile) on project hbase-server: Compilation failure [ERROR] /home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java:[87,8] cannot find symbol [ERROR] symbol : method setWriteChecksum( boolean ) [ERROR] location: class org.apache.hadoop.fs.FileSystem
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #319 (See https://builds.apache.org/job/HBase-0.98/319/)
        HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) (stack: rev 86ab435b8cd4d77ad4b90cd43fd5acd8579b60a4)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #319 (See https://builds.apache.org/job/HBase-0.98/319/ ) HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) (stack: rev 86ab435b8cd4d77ad4b90cd43fd5acd8579b60a4) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        Hide
        Andrew Purtell added a comment -

        Thanks for applying to 0.98

        Show
        Andrew Purtell added a comment - Thanks for applying to 0.98
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #5149 (See https://builds.apache.org/job/HBase-TRUNK/5149/)
        HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) (stack: rev 78f7cd450fe7ad5ed4b4b9634c7499e65968476f)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentListener.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) – DOC ADDENDUM (stack: rev b4a2d607a30344c37ed361cc8ffefe9159df01f0)
        • src/main/docbkx/configuration.xml
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #5149 (See https://builds.apache.org/job/HBase-TRUNK/5149/ ) HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) (stack: rev 78f7cd450fe7ad5ed4b4b9634c7499e65968476f) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentListener.java hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) – DOC ADDENDUM (stack: rev b4a2d607a30344c37ed361cc8ffefe9159df01f0) src/main/docbkx/configuration.xml
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #300 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/300/)
        HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) (stack: rev 86ab435b8cd4d77ad4b90cd43fd5acd8579b60a4)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #300 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/300/ ) HBASE-11218 Data loss in HBase standalone mode (Liu Shaohui) (stack: rev 86ab435b8cd4d77ad4b90cd43fd5acd8579b60a4) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
        stack made changes -
        Attachment 11218v2.098.txt [ 12647176 ]
        Hide
        stack added a comment -

        Patch I applied to 0.98

        Show
        stack added a comment - Patch I applied to 0.98
        Hide
        stack added a comment -

        Committed to master. Let me commit to 0.98 too. Fixed the javadoc warning in trunk commit (was not from this patch). Added addendum updating doc on local fs dataloss to say we don't lose data in 0.98.4 and 1.0.0.

        Thanks for the patch Liu Shaohui

        Show
        stack added a comment - Committed to master. Let me commit to 0.98 too. Fixed the javadoc warning in trunk commit (was not from this patch). Added addendum updating doc on local fs dataloss to say we don't lose data in 0.98.4 and 1.0.0. Thanks for the patch Liu Shaohui
        stack made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Release Note Fix for local filesystem data loss in standalone and pseudo-distributed mode.
        Resolution Fixed [ 1 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12647120/HBASE-11218-trunk-v2.diff
        against trunk revision .
        ATTACHMENT ID: 12647120

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 12 new or modified tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        -1 release audit. The applied patch generated 32 release audit warnings (more than the trunk's current 0 warnings).

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12647120/HBASE-11218-trunk-v2.diff against trunk revision . ATTACHMENT ID: 12647120 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 12 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 1 warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 32 release audit warnings (more than the trunk's current 0 warnings). +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9621//console This message is automatically generated.
        Liu Shaohui made changes -
        Attachment HBASE-11218-trunk-v2.diff [ 12647120 ]
        Hide
        Liu Shaohui added a comment -

        Disable the checksum of localFileSystem in the pseudo-distributed mode deployments.

        Thanks for Enis Soztutar and stack 's review.

        Show
        Liu Shaohui added a comment - Disable the checksum of localFileSystem in the pseudo-distributed mode deployments. Thanks for Enis Soztutar and stack 's review.
        Hide
        Liu Shaohui added a comment -

        Enis Soztutar
        Sorry, It's my mistake.
        Agree that we should fix this issue in the pseudo-distributed mode deployments.

        Show
        Liu Shaohui added a comment - Enis Soztutar Sorry, It's my mistake. Agree that we should fix this issue in the pseudo-distributed mode deployments.
        Hide
        Enis Soztutar added a comment -

        I think the data loss in pseudo distributed mode caused by hardware failure is accepted, data loss caused by the HBase's implement is not accepted and we should fix it.

        I think there is a misunderstanding here. I was referring to the same local file system issue. The fix in the patch will only fix it for local mode deployments, but not pseudo-distributed mode deployments.

        Show
        Enis Soztutar added a comment - I think the data loss in pseudo distributed mode caused by hardware failure is accepted, data loss caused by the HBase's implement is not accepted and we should fix it. I think there is a misunderstanding here. I was referring to the same local file system issue. The fix in the patch will only fix it for local mode deployments, but not pseudo-distributed mode deployments.
        Hide
        stack added a comment -

        Liu Shaohui Could you do your same trick on the RS side too if local fs (as Enis suggests above)?

        Thanks for doing the test compares. I will commit this unless objection later today. We can look into fixing pseudo on local fs in a new issue?

        Show
        stack added a comment - Liu Shaohui Could you do your same trick on the RS side too if local fs (as Enis suggests above)? Thanks for doing the test compares. I will commit this unless objection later today. We can look into fixing pseudo on local fs in a new issue?
        Hide
        Liu Shaohui added a comment -

        Enis Soztutar

        in pseudo distributed mode, where there is an actual RS, but still using the local fs, there still will be data loss with this patch from my understanding

        Yes. but I think the data loss in pseudo distributed mode caused by hardware failure is accepted, data loss caused by the HBase's implement is not accepted and we should fix it.

        Show
        Liu Shaohui added a comment - Enis Soztutar in pseudo distributed mode, where there is an actual RS, but still using the local fs, there still will be data loss with this patch from my understanding Yes. but I think the data loss in pseudo distributed mode caused by hardware failure is accepted, data loss caused by the HBase's implement is not accepted and we should fix it.
        Hide
        Liu Shaohui added a comment -

        Nicolas Liochon
        The comparison done in my dev machine is following

        case trunk HBASE-11218
        small tests 6m32.725s 6m34.124s
        medium tests 91m23.238s 93m6.703s
        large tests 154m58.474s 150m42.585s

        The test time increase is very little.

        Show
        Liu Shaohui added a comment - Nicolas Liochon The comparison done in my dev machine is following case trunk HBASE-11218 small tests 6m32.725s 6m34.124s medium tests 91m23.238s 93m6.703s large tests 154m58.474s 150m42.585s The test time increase is very little.
        Hide
        stack added a comment -

        I like the idea of adding the config to the hbase fs.

        Show
        stack added a comment - I like the idea of adding the config to the hbase fs.
        Hide
        Enis Soztutar added a comment -

        in pseudo distributed mode, where there is an actual RS, but still using the local fs, there still will be data loss with this patch from my understanding. Is that correct? Should we configure the local fs in HRegionFileSystem?

        Show
        Enis Soztutar added a comment - in pseudo distributed mode, where there is an actual RS, but still using the local fs, there still will be data loss with this patch from my understanding. Is that correct? Should we configure the local fs in HRegionFileSystem?
        Hide
        Nicolas Liochon added a comment -

        to be clearer I'm +1 if there is no test time increase, or if it's limited. 10% would be ok...

        Show
        Nicolas Liochon added a comment - to be clearer I'm +1 if there is no test time increase, or if it's limited. 10% would be ok...
        Hide
        stack added a comment -

        If only we had a robot that could run the patch against trunk.... so we could compare before and after.

        Show
        stack added a comment - If only we had a robot that could run the patch against trunk.... so we could compare before and after.
        Hide
        Nicolas Liochon added a comment -

        I'm afraid of the added syncs... But I understand the need.

        Show
        Nicolas Liochon added a comment - I'm afraid of the added syncs... But I understand the need.
        Hide
        stack added a comment -

        Patch looks good to me. I like the way the patch skirts the issue that causes us dataloss. Why you think slowdown Nicolas Liochon?

        Show
        stack added a comment - Patch looks good to me. I like the way the patch skirts the issue that causes us dataloss. Why you think slowdown Nicolas Liochon ?
        Jean-Marc Spaggiari made changes -
        Comment [ [~nkeywal]
        {quote}
        I'm curious, why is it an issue for you?
        {quote}
        We use the standalone hbase in our onebox test env. In this env, we start the application and other services (eg, hbase, zookeeper) that it depends on in a machine, , then do many end-to-end logical and failure tests. So it is important to keep data durability in some level. Data loss in standalone mode give a not good impression of HBase to out users.

        {quote}
        On the patch, will this slow down the tests?
        {quote}
        I will do a comparison and show the data later.
        ]
        Hide
        Liu Shaohui added a comment -

        Nicolas Liochon

        I'm curious, why is it an issue for you?

        We use the standalone hbase in our onebox test env. In this env, we start the application and other services (eg, hbase, zookeeper) that it depends on in a machine, , then do many end-to-end logical and failure tests. So it is important to keep data durability in some level. Data loss in standalone mode give a not good impression of HBase to out users.

        On the patch, will this slow down the tests?

        I will do a comparison and show the data later.

        Show
        Liu Shaohui added a comment - Nicolas Liochon I'm curious, why is it an issue for you? We use the standalone hbase in our onebox test env. In this env, we start the application and other services (eg, hbase, zookeeper) that it depends on in a machine, , then do many end-to-end logical and failure tests. So it is important to keep data durability in some level. Data loss in standalone mode give a not good impression of HBase to out users. On the patch, will this slow down the tests? I will do a comparison and show the data later.
        Hide
        Nicolas Liochon added a comment -

        I'm curious, why is it an issue for you?
        On the patch, will this slow down the tests?

        Show
        Nicolas Liochon added a comment - I'm curious, why is it an issue for you? On the patch, will this slow down the tests?
        Liu Shaohui made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Liu Shaohui made changes -
        Attachment HBASE-11218-trunk-v1.diff [ 12646472 ]
        Hide
        Liu Shaohui added a comment -

        Patch for trunk.
        Some notes about the change in tests.
        When the hbase is restarted, the table state in those tests are disbabled for the test disable the table before. So i add enableTable(table) in TestOfflineMetaRebuildBase.
        For hbck does not the NOT_IN_META_OR_DEPLOYED for disabled tables and we can not enable it in the tests, so i remove the NOT_IN_META_OR_DEPLOYED error check in TestOfflineMetaRebuildOverlap and TestOfflineMetaRebuildOverlap.

        Show
        Liu Shaohui added a comment - Patch for trunk. Some notes about the change in tests. When the hbase is restarted, the table state in those tests are disbabled for the test disable the table before. So i add enableTable(table) in TestOfflineMetaRebuildBase. For hbck does not the NOT_IN_META_OR_DEPLOYED for disabled tables and we can not enable it in the tests, so i remove the NOT_IN_META_OR_DEPLOYED error check in TestOfflineMetaRebuildOverlap and TestOfflineMetaRebuildOverlap.
        Liu Shaohui made changes -
        Field Original Value New Value
        Assignee Liu Shaohui [ liushaohui ]
        Liu Shaohui created issue -

          People

          • Assignee:
            Liu Shaohui
            Reporter:
            Liu Shaohui
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development