HBase
  1. HBase
  2. HBASE-3696

No durability when running with LocalFileSystem

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: documentation, wal
    • Labels:
      None

      Description

      LocalFileSystem in Hadoop doesn't currently implement sync(), so when we're running in that case, we don't have any durability. This isn't a huge deal since it isn't a realistic deployment scenario, but it's probably worth documenting. It caused some confusion for a user when a table disappeared after killing a standalone instance that was hosting its data in the local FS.

        Issue Links

          Activity

          Hide
          stack added a comment -

          Resolving as implemented by HBASE-11218 (which should go in soon) at least for standalone. Data loss on fs has been doc'd also.

          Hopefully HBASE-11218 will do pseudo distributed mode too. If not, doc will need tweaking. Can do in new issue.

          Show
          stack added a comment - Resolving as implemented by HBASE-11218 (which should go in soon) at least for standalone. Data loss on fs has been doc'd also. Hopefully HBASE-11218 will do pseudo distributed mode too. If not, doc will need tweaking. Can do in new issue.
          Hide
          Misty Stanley-Jones added a comment -

          Can this be closed or can the DOC component be removed, based on the previous comment?

          Show
          Misty Stanley-Jones added a comment - Can this be closed or can the DOC component be removed, based on the previous comment?
          Hide
          Harsh J added a comment -

          Doc patch of mine got applied via HBASE-9379.

          Show
          Harsh J added a comment - Doc patch of mine got applied via HBASE-9379 .
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12600032/HBASE-3696.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12600032/HBASE-3696.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6917//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Is this added foot-note sufficient?

          Show
          Harsh J added a comment - Is this added foot-note sufficient?
          Hide
          Ian Varley added a comment -

          This confused me a bit today while trying to debug HBase replication running locally; you have to roll the HLog in order for ReplicationSource to pick up and replicate your edits. +1 to a fix (but, no idea what that'd be; an implementation of sync() for LocalFileSystem?)

          Show
          Ian Varley added a comment - This confused me a bit today while trying to debug HBase replication running locally; you have to roll the HLog in order for ReplicationSource to pick up and replicate your edits. +1 to a fix (but, no idea what that'd be; an implementation of sync() for LocalFileSystem?)
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1814 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1814/)
          HBASE-3696 isMajorCompaction() check triggers lots of listStatus DFS RPC calls from HBase

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1814 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1814/ ) HBASE-3696 isMajorCompaction() check triggers lots of listStatus DFS RPC calls from HBase

            People

            • Assignee:
              Unassigned
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development