Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9657

Schedule EC tasks at proper time to reduce the impact of recovery traffic

    XMLWordPrintableJSON

Details

    Description

      The EC recover tasks consume a lot of network bandwidth and disk I/O. Recovering a corrupt block requires transferring 6 blocks , hence creating a 6X overhead in network bandwidth and disk I/O. When a datanode fails , the recovery of the whole blocks on this datanode may use up the network bandwith. We need to start a recovery task at a proper time in order to give less impact to the system.

      Attachments

        1. HDFS-9657-001.patch
          9 kB
          Li Bo
        2. HDFS-9657-002.patch
          14 kB
          Li Bo

        Issue Links

          Activity

            People

              Unassigned Unassigned
              libo-intel Li Bo
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: