HBase
  1. HBase
  2. HBASE-5676

Note that dfs.support.append does not have to be enabled post 1.x

    Details

    • Type: Task Task
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.95.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In Hadoop 1.x (HADOOP-8230) we are going to enable durable sync by default, and remove the dfs.support.append option. What this means for you:

      • HBase will work out of the box on Hadoop 1.x, no need to tell people to re-configure
      • You no longer have to enable append (which can result in data loss) to enable HBase support
      1. HBASE-5676.patch
        10 kB
        David S. Wang

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          The "X" btw is TBD, you can follow the jira to see what version this lands in.

          Show
          Eli Collins added a comment - The "X" btw is TBD, you can follow the jira to see what version this lands in.
          Hide
          Eli Collins added a comment -

          HADOOP-8230 was committed for the Hadoop 1.1 release.

          Show
          Eli Collins added a comment - HADOOP-8230 was committed for the Hadoop 1.1 release.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4999/
          -----------------------------------------------------------

          Review request for hbase.

          Summary
          -------

          Change append checks to sync and hflush checks, in keeping with the spirit of the original code.

          Should only be applied to 0.96, as only 0.96 will support Hadoop 1.x+ according to HBASE-5536.

          This addresses bug HBASE-5676.
          https://issues.apache.org/jira/browse/HBASE-5676

          Diffs


          src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon 6bc2cf2
          src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java a0aaaf4
          src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java 302a3b1
          src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 3d35d3e
          src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java 3ea0a3b
          src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java cb7efc3
          src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 339a120

          Diff: https://reviews.apache.org/r/4999/diff

          Testing
          -------

          Passed local -PrunAllTests. Passed test-patch.sh:

          +1 overall.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          Thanks,

          David

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4999/ ----------------------------------------------------------- Review request for hbase. Summary ------- Change append checks to sync and hflush checks, in keeping with the spirit of the original code. Should only be applied to 0.96, as only 0.96 will support Hadoop 1.x+ according to HBASE-5536 . This addresses bug HBASE-5676 . https://issues.apache.org/jira/browse/HBASE-5676 Diffs src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon 6bc2cf2 src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java a0aaaf4 src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java 302a3b1 src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 3d35d3e src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java 3ea0a3b src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java cb7efc3 src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 339a120 Diff: https://reviews.apache.org/r/4999/diff Testing ------- Passed local -PrunAllTests. Passed test-patch.sh: +1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. Thanks, David
          Hide
          David S. Wang added a comment -
          Show
          David S. Wang added a comment - Review is at: https://reviews.apache.org/r/4999/
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4999/#review7524
          -----------------------------------------------------------

          Are you sure this is right? If you're using HBase against Hadoop 1.0, for example, the syncFs() call exists, but sync isn't enabled by default, so this would give you an incorrect result, I'd think

          • Todd

          On 2012-05-03 21:22:19, David Wang wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4999/

          -----------------------------------------------------------

          (Updated 2012-05-03 21:22:19)

          Review request for hbase.

          Summary

          -------

          Change append checks to sync and hflush checks, in keeping with the spirit of the original code.

          Should only be applied to 0.96, as only 0.96 will support Hadoop 1.x+ according to HBASE-5536.

          This addresses bug HBASE-5676.

          https://issues.apache.org/jira/browse/HBASE-5676

          Diffs

          -----

          src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon 6bc2cf2

          src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java a0aaaf4

          src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java 302a3b1

          src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 3d35d3e

          src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java 3ea0a3b

          src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java cb7efc3

          src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 339a120

          Diff: https://reviews.apache.org/r/4999/diff

          Testing

          -------

          Passed local -PrunAllTests. Passed test-patch.sh:

          +1 overall.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          Thanks,

          David

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4999/#review7524 ----------------------------------------------------------- Are you sure this is right? If you're using HBase against Hadoop 1.0, for example, the syncFs() call exists, but sync isn't enabled by default, so this would give you an incorrect result, I'd think Todd On 2012-05-03 21:22:19, David Wang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4999/ ----------------------------------------------------------- (Updated 2012-05-03 21:22:19) Review request for hbase. Summary ------- Change append checks to sync and hflush checks, in keeping with the spirit of the original code. Should only be applied to 0.96, as only 0.96 will support Hadoop 1.x+ according to HBASE-5536 . This addresses bug HBASE-5676 . https://issues.apache.org/jira/browse/HBASE-5676 Diffs ----- src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon 6bc2cf2 src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java a0aaaf4 src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java 302a3b1 src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 3d35d3e src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java 3ea0a3b src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java cb7efc3 src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 339a120 Diff: https://reviews.apache.org/r/4999/diff Testing ------- Passed local -PrunAllTests. Passed test-patch.sh: +1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. Thanks, David
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525512/HBASE-5676.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1751//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1751//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1751//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525512/HBASE-5676.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1751//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1751//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1751//console This message is automatically generated.
          Hide
          David S. Wang added a comment -

          Sounds like we should wait for the discussion on HADOOP-8230 to reach a conclusion before I modify the patch. I'll monitor HADOOP-8230 and post another patch when there's a consensus.

          Show
          David S. Wang added a comment - Sounds like we should wait for the discussion on HADOOP-8230 to reach a conclusion before I modify the patch. I'll monitor HADOOP-8230 and post another patch when there's a consensus.

            People

            • Assignee:
              David S. Wang
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development