Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-9040

Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-7285
    • None
    • None
    • Reviewed

    Description

      The general idea is to simplify error handling logic.

      Proposal 1:
      A BlockGroupDataStreamer to communicate with NN to allocate/update block, and StripedDataStreamer s only have to stream blocks to DNs.

      Proposal 2:
      See below the comment from jingzhao.

      Attachments

        1. HDFS-9040.00.patch
          71 kB
          Walter Su
        2. HDFS-9040.001.wip.patch
          112 kB
          Jing Zhao
        3. HDFS-9040.02.bgstreamer.patch
          74 kB
          Walter Su
        4. HDFS-9040-HDFS-7285.002.patch
          115 kB
          Jing Zhao
        5. HDFS-9040-HDFS-7285.003.patch
          118 kB
          Jing Zhao
        6. HDFS-9040-HDFS-7285.004.patch
          120 kB
          Jing Zhao
        7. HDFS-9040-HDFS-7285.005.patch
          123 kB
          Jing Zhao
        8. HDFS-9040-HDFS-7285.005.patch
          120 kB
          Jing Zhao
        9. HDFS-9040-HDFS-7285.006.patch
          123 kB
          Jing Zhao
        10. HDFS-9040-HDFS-7285.007.patch
          123 kB
          Jing Zhao

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9079 Erasure coding: preallocate multiple generation stamps and serialize updates from data streamers

          • Major - Major loss of function.
          • Patch Available
          is duplicated by

          Sub-task - The sub-task of the issue HDFS-8383 Tolerate multiple failures in DFSStripedOutputStream

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HDFS-9098 Erasure coding: emulate race conditions among striped streamers in write pipeline

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              jingzhao Jing Zhao
              walter.k.su Walter Su
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: