Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7836 BlockManager Scalability Improvements
  3. HDFS-7923

The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.8.0, 3.0.0-alpha1
    • None
    • None

    Description

      The DataNodes should rate-limit their full block reports. They can do this by first sending a heartbeat message to the NN with an optional boolean set which requests permission to send a full block report. If the NN responds with another optional boolean set, the DN will send an FBR... if not, it will wait until later. This can be done compatibly with optional fields.

      Attachments

        1. HDFS-7923.000.patch
          41 kB
          Charles Lamb
        2. HDFS-7923.001.patch
          41 kB
          Charles Lamb
        3. HDFS-7923.002.patch
          46 kB
          Charles Lamb
        4. HDFS-7923.003.patch
          74 kB
          Colin McCabe
        5. HDFS-7923.004.patch
          80 kB
          Colin McCabe
        6. HDFS-7923.006.patch
          77 kB
          Colin McCabe
        7. HDFS-7923.007.patch
          77 kB
          Colin McCabe

        Issue Links

          Activity

            People

              cmccabe Colin McCabe
              cmccabe Colin McCabe
              Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: