Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13521

SampleQuantile does not perform well under load

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.3.0
    • None
    • metrics
    • None

    Description

      After adding quantile collection to one of our clusters we saw much higher latency for RPCs. This was traced down to the quantile collection. Samples are being buffered and inserted in groups of 500. After the buffered samples are inserted, the entire set of samples for this time period (600 seconds at the longest for us) is "compressed".

      All operations for RPC metrics are synchronized. Usually this isn't an issue but it seems that this compression operation is taking a significant amount of time.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HDFS-10873 Add histograms for FSNamesystemLock Metrics

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-10475 Adding metrics for long FSNamesystem read and write locks

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              mwagner Mark Wagner
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: