Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11301

[optionally] update jmx cache to drop old metrics

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0, 2.6.5
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      MetricsSourceAdapter::updateJmxCache() skips updating the info cache if no new metric is added since last time:

            int oldCacheSize = attrCache.size();
            int newCacheSize = updateAttrCache();
            if (oldCacheSize < newCacheSize) {
              updateInfoCache();
            }
      

      This behavior is not desirable in some applications. For example nntop (HDFS-6982) reports the top users via jmx. The list is updated after each report. The previously reported top users hence should be removed from the cache upon each report request.

      In our production run of nntop we made a change to ignore the size check and always perform updateInfoCache. I am planning to submit a patch including this change. The feature can be enabled by a configuration parameter.

        Attachments

        1. HADOOP-11301.v04.patch
          4 kB
          Maysam Yabandeh
        2. HADOOP-11301.v03.patch
          4 kB
          Maysam Yabandeh
        3. HADOOP-11301.v02.patch
          4 kB
          Maysam Yabandeh
        4. HADOOP-11301.v01.patch
          7 kB
          Maysam Yabandeh

          Issue Links

            Activity

              People

              • Assignee:
                maysamyabandeh Maysam Yabandeh
                Reporter:
                maysamyabandeh Maysam Yabandeh
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: