Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-14162

Balancer should work with ObserverNode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0, 3.3.0, 3.1.4, 3.2.2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Balancer provides a substantial RPC load on NameNode. It would be good to divert Balancer RPCs getBlocks(), etc. to ObserverNode. The main problem is that Balancer uses NamenodeProtocol, while ORPP currently supports only ClientProtocol.

        Attachments

        1. testBalancerWithObserver.patch
          6 kB
          Konstantin Shvachko
        2. HDFS-14162-HDFS-12943.wip0.patch
          19 kB
          Erik Krogen
        3. HDFS-14162.000.patch
          22 kB
          Erik Krogen
        4. testBalancerWithObserver-3.patch
          14 kB
          Konstantin Shvachko
        5. HDFS-14162.001.patch
          28 kB
          Erik Krogen
        6. HDFS-14162.002.patch
          32 kB
          Erik Krogen
        7. ReflectionBenchmark.java
          1 kB
          Erik Krogen
        8. HDFS-14162.003.patch
          33 kB
          Erik Krogen
        9. HDFS-14162.004.patch
          33 kB
          Erik Krogen
        10. HDFS-14162-branch-2.004.patch
          32 kB
          Erik Krogen
        11. HDFS-14162-branch-2.005.patch
          35 kB
          Erik Krogen

          Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-15032 Balancer crashes when it fails to contact an unavailable NN via ObserverReadProxyProvider

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13183 Standby NameNode process getBlocks request to reduce Active load

          • Major - Major loss of function.
          • Patch Available
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14245 Class cast error in GetGroups with ObserverReadProxyProvider

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-14979 [Observer Node] Balancer should submit getBlocks to Observer Node when possible

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                shv Konstantin Shvachko
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: