Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-7369

Erasure coding: distribute recovery work for striped blocks to DataNode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-7285
    • None
    • None

    Description

      This JIRA updates NameNode to handle background / offline recovery of erasure coded blocks. It includes 2 parts:

      1. Extend UnderReplicatedBlocks to recognize EC blocks and insert them to appropriate priority levels.
      2. Update ReplicationMonitor to distinguish block codec tasks and send a new DataNode command.

      Attachments

        1. HDFS-7369-000-part1.patch
          3 kB
          Zhe Zhang
        2. HDFS-7369-000-part2.patch
          14 kB
          Zhe Zhang
        3. HDFS-7369-001.patch
          33 kB
          Zhe Zhang
        4. HDFS-7369-002.patch
          41 kB
          Zhe Zhang
        5. HDFS-7369-003.patch
          42 kB
          Zhe Zhang
        6. HDFS-7369-004.patch
          46 kB
          Zhe Zhang

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue HDFS-7689 Add periodic checker to find the corrupted EC blocks/files

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HDFS-7344 [umbrella] Erasure Coding worker and support in DataNode

          • Major - Major loss of function.
          • Resolved
          requires

          Sub-task - The sub-task of the issue HDFS-7716 Erasure Coding: extend BlockInfo to handle EC info

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-7912 Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              zhz Zhe Zhang
              zhz Zhe Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: