Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11384

Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      When running balancer on hadoop cluster which have more than 3000 Datanodes will cause NameNode's rpc.CallQueueLength spike. We observed this situation could cause Hbase cluster failure due to RegionServer's WAL timeout.

      Attachments

        1. balancer.week.png
          50 kB
          yunjiong zhao
        2. balancer.day.png
          58 kB
          yunjiong zhao
        3. HDFS-11384.001.patch
          4 kB
          yunjiong zhao
        4. HDFS-11384.002.patch
          5 kB
          yunjiong zhao
        5. HDFS-11384.003.patch
          5 kB
          Konstantin Shvachko
        6. HDFS-11384.004.patch
          5 kB
          Konstantin Shvachko
        7. HDFS-11384.005.patch
          5 kB
          Konstantin Shvachko
        8. HDFS-11384.006.patch
          13 kB
          Konstantin Shvachko
        9. HDFS-11384-007.patch
          13 kB
          Konstantin Shvachko
        10. HDFS-11384.008.patch
          15 kB
          Konstantin Shvachko
        11. HDFS-11384.009.patch
          14 kB
          Konstantin Shvachko
        12. HDFS-11384.010.patch
          16 kB
          Konstantin Shvachko
        13. HDFS-11384.011.patch
          16 kB
          Konstantin Shvachko
        14. HDFS-11384-branch-2.8.011.patch
          13 kB
          Konstantin Shvachko
        15. HDFS-11384-branch-2.7.011.patch
          13 kB
          Konstantin Shvachko

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. HDFS-12347 TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-14973 Balancer getBlocks RPC dispersal does not function properly

        • Major - Major loss of function.
        • Resolved
        is related to

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-8825 Enhancements to Balancer

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shv Konstantin Shvachko
            zhaoyunjiong yunjiong zhao
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment