Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3120

Enable hsync and hflush by default

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0-alpha
    • None
    • None
    • Reviewed

    Description

      The work on branch-20-append was to support sync, for durable HBase WALs, not append. The branch-20-append implementation is known to be buggy. There's been confusion about this, we often answer queries on the list like this. Unfortunately, the way to enable correct sync on branch-1 for HBase is to set dfs.support.append to true in your config, which has the side effect of enabling append (which we don't want to do).

      For v1.x let's:

      1. Always enable the sync path (currently only enabled if dfs.support.append is set)
      2. Remove the dfs.support.append configuration option. Let's keep the code paths though in case we ever fix append on branch-1, in which case we can add the config option back

      For 2.x let's

      1. Always enable the hsync/hflush path
      2. The dfs.support.appends only enables the append specific paths (since the hsync/hflush paths are now always on). Append will still default to being enabled so there is no net effect by default.

      Attachments

        1. hdfs-3120.txt
          21 kB
          Eli Collins
        2. hdfs-3120.txt
          18 kB
          Eli Collins

        Issue Links

          Activity

            People

              eli Eli Collins
              eli2 Eli Collins
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: