Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24994

Add hedgedReadOpsInCurThread metric

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha-1, 1.7.0, 2.4.0
    • Component/s: metrics
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Expose Hadoop hedgedReadOpsInCurThread metric to HBase.
      This metric counts the number of times the hedged reads service executor rejected a read task, falling back to the current thread.
      This will help determine the proper size of the thread pool (dfs.client.hedged.read.threadpool.size).
      Show
      Expose Hadoop hedgedReadOpsInCurThread metric to HBase. This metric counts the number of times the hedged reads service executor rejected a read task, falling back to the current thread. This will help determine the proper size of the thread pool (dfs.client.hedged.read.threadpool.size).

      Description

      When adding Hedged Reads metrics at https://issues.apache.org/jira/browse/HBASE-12220, hedgedReadOpsInCurThread was not included.

      I wonder if there was any reason to not include them. If not, I will provide pull request to include them soon.

       

      Hedged reads use same thread pool for the original read:
      https://github.com/apache/hadoop/blob/0b8464d75227fcee2c6e7f2410377b3d53d3d5f8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L1349-L1357
      When the executor is full and rejects the task, it will be executed in current thread, increasing this metric:
      https://github.com/apache/hadoop/blob/0b8464d75227fcee2c6e7f2410377b3d53d3d5f8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java#L2913-L2918

      This metric is very useful to determine if the hedged reads thread pool (dfs.client.hedged.read.threadpool.size) is undersized.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                akiraluca Javier Akira Luca de Tena
                Reporter:
                akiraluca Javier Akira Luca de Tena
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: