Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.4.0
    • Component/s: Deployment
    • Labels:
      None

      Description

      According to the HBase book () :

      HBase will lose data unless it is running on an HDFS that has a durable sync implementation. Hadoop 0.20.2, Hadoop 0.20.203.0, and Hadoop 0.20.204.0 DO NOT have this attribute. Currently only Hadoop versions 0.20.205.x or any release in excess of this version – this includes hadoop 1.0.0 – have a working, durable sync [6]. Sync has to be explicitly enabled by setting dfs.support.append equal to true on both the client side – in hbase-site.xml – and on the serverside in hdfs-site.xml (The sync facility HBase needs is a subset of the append code path).

      http://hbase.apache.org/book.html#hadoop

      1. BIGTOP-552.patch
        2 kB
        Michael Morello

        Activity

        Hide
        Michael Morello added a comment -

        This patch :

        • explicitly enable durable sync in hbase-site.xml
        • enable durable sync in HDFS if variable $hdfs_support_append is set to "true"
        • enable shortcut reader feature for HBase if variable $hdfs_shortcut_reader_user is et to "hbase"
        Show
        Michael Morello added a comment - This patch : explicitly enable durable sync in hbase-site.xml enable durable sync in HDFS if variable $hdfs_support_append is set to "true" enable shortcut reader feature for HBase if variable $hdfs_shortcut_reader_user is et to "hbase"
        Hide
        Roman Shaposhnik added a comment -

        Michael, the patch looks good and I think it can be committed. Thanks a million for taking time to submit it!

        Now, I think we can apply it as is to branch-0.3. As for trunk, Ptr and I have been talking about moving everything to the use of extlookup lately. I'll leave it up to him to either apply the patch as is or change it to use extlookup.

        Show
        Roman Shaposhnik added a comment - Michael, the patch looks good and I think it can be committed. Thanks a million for taking time to submit it! Now, I think we can apply it as is to branch-0.3. As for trunk, Ptr and I have been talking about moving everything to the use of extlookup lately. I'll leave it up to him to either apply the patch as is or change it to use extlookup.
        Hide
        Patrick Taylor Ramsey added a comment -

        Hi, Michael. Thanks for setting this up. I'd prefer that in general in templates we both check that variables are defined and that they're non-empty, but it looks like we call has_variable? only all over the place so I'll clean that up in a separate patch. +1 and committed.

        Show
        Patrick Taylor Ramsey added a comment - Hi, Michael. Thanks for setting this up. I'd prefer that in general in templates we both check that variables are defined and that they're non-empty, but it looks like we call has_variable? only all over the place so I'll clean that up in a separate patch. +1 and committed.
        Hide
        Patrick Taylor Ramsey added a comment -

        Committed to trunk as well. Thanks again, Michael.

        Show
        Patrick Taylor Ramsey added a comment - Committed to trunk as well. Thanks again, Michael.
        Hide
        Patrick Taylor Ramsey added a comment -

        This patch is now in branch-0.3, as it was intended to be last Friday. Apologies for the oversight, and thanks again for taking the time to contribute.

        Show
        Patrick Taylor Ramsey added a comment - This patch is now in branch-0.3, as it was intended to be last Friday. Apologies for the oversight, and thanks again for taking the time to contribute.

          People

          • Assignee:
            Patrick Taylor Ramsey
            Reporter:
            Michael Morello
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development