Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-14162

Balancer should work with ObserverNode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0, 3.3.0, 3.1.4, 3.2.2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Balancer provides a substantial RPC load on NameNode. It would be good to divert Balancer RPCs getBlocks(), etc. to ObserverNode. The main problem is that Balancer uses NamenodeProtocol, while ORPP currently supports only ClientProtocol.

        Attachments

        1. HDFS-14162.000.patch
          22 kB
          Erik Krogen
        2. HDFS-14162.001.patch
          28 kB
          Erik Krogen
        3. HDFS-14162.002.patch
          32 kB
          Erik Krogen
        4. HDFS-14162.003.patch
          33 kB
          Erik Krogen
        5. HDFS-14162.004.patch
          33 kB
          Erik Krogen
        6. HDFS-14162-branch-2.004.patch
          32 kB
          Erik Krogen
        7. HDFS-14162-branch-2.005.patch
          35 kB
          Erik Krogen
        8. HDFS-14162-HDFS-12943.wip0.patch
          19 kB
          Erik Krogen
        9. ReflectionBenchmark.java
          1 kB
          Erik Krogen
        10. testBalancerWithObserver.patch
          6 kB
          Konstantin Shvachko
        11. testBalancerWithObserver-3.patch
          14 kB
          Konstantin Shvachko

          Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-15032 Balancer crashes when it fails to contact an unavailable NN via ObserverReadProxyProvider

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-13183 Standby NameNode process getBlocks request to reduce Active load

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14245 Class cast error in GetGroups with ObserverReadProxyProvider

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-14979 [Observer Node] Balancer should submit getBlocks to Observer Node when possible

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                shv Konstantin Shvachko
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: