Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-14162

Balancer should work with ObserverNode

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0, 3.3.0, 3.1.4, 3.2.2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Balancer provides a substantial RPC load on NameNode. It would be good to divert Balancer RPCs getBlocks(), etc. to ObserverNode. The main problem is that Balancer uses NamenodeProtocol, while ORPP currently supports only ClientProtocol.

        Attachments

        1. HDFS-14162.000.patch
          22 kB
          Erik Krogen
        2. HDFS-14162.001.patch
          28 kB
          Erik Krogen
        3. HDFS-14162.002.patch
          32 kB
          Erik Krogen
        4. HDFS-14162.003.patch
          33 kB
          Erik Krogen
        5. HDFS-14162.004.patch
          33 kB
          Erik Krogen
        6. HDFS-14162-branch-2.004.patch
          32 kB
          Erik Krogen
        7. HDFS-14162-branch-2.005.patch
          35 kB
          Erik Krogen
        8. HDFS-14162-HDFS-12943.wip0.patch
          19 kB
          Erik Krogen
        9. ReflectionBenchmark.java
          1 kB
          Erik Krogen
        10. testBalancerWithObserver.patch
          6 kB
          Konstantin Shvachko
        11. testBalancerWithObserver-3.patch
          14 kB
          Konstantin Shvachko

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. HDFS-15032 Balancer crashes when it fails to contact an unavailable NN via ObserverReadProxyProvider

        • Major - Major loss of function.
        • Resolved
        is related to

        New Feature - A new feature of the product, which has yet to be developed. HDFS-13183 Standby NameNode process getBlocks request to reduce Active load

        • Major - Major loss of function.
        • Resolved
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-14245 Class cast error in GetGroups with ObserverReadProxyProvider

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-14979 [Observer Node] Balancer should submit getBlocks to Observer Node when possible

        • Major - Major loss of function.
        • Resolved

          Activity
          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              xkrogen Erik Krogen
              Reporter:
              shv Konstantin Shvachko

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment