Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-16678

RBF supports disable getNodeUsage() in RBFMetrics

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • rbf
    • Reviewed

    Description

      In our prod environment, we try to collect RBF metrics every 15s through jmx_exporter. And we found that collection task often failed.

      After tracing and found that the collection task is blocked at getNodeUsage() in RBFMetrics, because it will collection all datanode's usage from downstream nameservices. This is a very expensive and almost useless operation. Because in most scenarios, each NameSerivce contains almost the same DNs. We can get the data usage's from any one nameservices, not from RBF.

      So I feel that RBF should supports disable getNodeUsage() in RBFMetrics.

      Attachments

        Issue Links

          Activity

            People

              xuzq_zander ZanderXu
              xuzq_zander ZanderXu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h