Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13788

Update EC documentation about rack fault tolerance

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      From http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html:

      For rack fault-tolerance, it is also important to have at least as many racks as the configured EC stripe width. For EC policy RS (6,3), this means minimally 9 racks, and ideally 10 or 11 to handle planned and unplanned outages. For clusters with fewer racks than the stripe width, HDFS cannot maintain rack fault-tolerance, but will still attempt to spread a striped file across multiple nodes to preserve node-level fault-tolerance.

      Theoretical minimum is 3 racks, and ideally 9 or more, so the document should be updated.

      (I didn't check timestamps, but this is probably due to BlockPlacementPolicyRackFaultTolerant isn't completely done when HDFS-9088 introduced this doc. Later there's also examples in TestErasureCodingMultipleRacks to test this explicitly.)

        Attachments

        1. HDFS-13788.001.patch
          2 kB
          Kitti Nanasi
        2. HDFS-13788.002.patch
          2 kB
          Kitti Nanasi

          Activity

            People

            • Assignee:
              knanasi Kitti Nanasi
              Reporter:
              xiaochen Xiao Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: