Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3650

Use MutableQuantiles to provide latency histograms for various operations

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None

      Description

      MutableQuantiles provide accurate estimation of various percentiles for a stream of data. Many existing metrics reported by a MutableRate would also benefit from having these percentiles; lets add MutableQuantiles where we think it'd be useful.

      1. hdfs-3650-1.patch
        30 kB
        Andrew Wang
      2. hdfs-3650-2.patch
        34 kB
        Andrew Wang
      3. hdfs-3650-3.patch
        34 kB
        Andrew Wang

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          Setting the target version instead of the "fix version." Please only set the fix version once it's been committed.

          Show
          Aaron T. Myers added a comment - Setting the target version instead of the "fix version." Please only set the fix version once it's been committed.
          Hide
          Andrew Wang added a comment -

          Patch that adds percentile latency for the following metrics:

          Datanode

          • SendDataPacketTransferNanos
          • SendDataPacketBlockedOnNetworkNanos
          • FlushNanos
          • FsyncNanos
          • PacketAckRoundTripTimeNanos

          Namenode

          • Syncs
          • BlockReport

          I added a new configuration option (dfs.metrics.percentiles.intervals) that takes comma-delimited integers, enabling multiple rollover intervals for the percentile metrics. All metrics will roll at these intervals. Percentile monitoring is also off by default.

          Show
          Andrew Wang added a comment - Patch that adds percentile latency for the following metrics: Datanode SendDataPacketTransferNanos SendDataPacketBlockedOnNetworkNanos FlushNanos FsyncNanos PacketAckRoundTripTimeNanos Namenode Syncs BlockReport I added a new configuration option (dfs.metrics.percentiles.intervals) that takes comma-delimited integers, enabling multiple rollover intervals for the percentile metrics. All metrics will roll at these intervals. Percentile monitoring is also off by default.
          Hide
          Aaron T. Myers added a comment -

          Marking PA for Andrew so that test-patch runs.

          Show
          Aaron T. Myers added a comment - Marking PA for Andrew so that test-patch runs.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12536721/hdfs-3650-1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
          org.apache.hadoop.io.file.tfile.TestTFileByteArrays
          org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
          org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
          org.apache.hadoop.hdfs.TestReplication
          org.apache.hadoop.hdfs.TestPersistBlocks

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2836//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2836//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536721/hdfs-3650-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl org.apache.hadoop.io.file.tfile.TestTFileByteArrays org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestPersistBlocks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2836//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2836//console This message is automatically generated.
          Hide
          Andrew Wang added a comment -

          I think the testcase failures are unrelated, even though there are a lot of them.

          TestZKFailoverController, TestTFile* (HADOOP-8537), TestNameNodeMetrics (HDFS-2434) are reported flaky.

          TestMetricsSystemImpl, TestReplication, TestPersistBlocks, I ran locally and passed.

          Show
          Andrew Wang added a comment - I think the testcase failures are unrelated, even though there are a lot of them. TestZKFailoverController, TestTFile* ( HADOOP-8537 ), TestNameNodeMetrics ( HDFS-2434 ) are reported flaky. TestMetricsSystemImpl, TestReplication, TestPersistBlocks, I ran locally and passed.
          Hide
          Aaron T. Myers added a comment -

          Hey Andrew, the patch seems to no longer apply to trunk. Would you mind rebasing it and uploading again? Sorry for the delay.

          Show
          Aaron T. Myers added a comment - Hey Andrew, the patch seems to no longer apply to trunk. Would you mind rebasing it and uploading again? Sorry for the delay.
          Hide
          Andrew Wang added a comment -

          Freshly rebased for trunk.

          Show
          Andrew Wang added a comment - Freshly rebased for trunk.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12537710/hdfs-3650-1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestFileAppend4
          org.apache.hadoop.hdfs.TestDatanodeBlockScanner

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2894//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2894//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537710/hdfs-3650-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.hdfs.TestDatanodeBlockScanner +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2894//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2894//console This message is automatically generated.
          Hide
          Andrew Wang added a comment -

          These two test failures look like HADOOP-8596 and HDFS-3660, I think unrelated.

          Show
          Andrew Wang added a comment - These two test failures look like HADOOP-8596 and HDFS-3660 , I think unrelated.
          Hide
          Aaron T. Myers added a comment -

          Patch looks great, Andrew. Just a few little comments:

          1. Instead of doing Configuration#get(...) and then doing the comma separating yourself, you can use Configuration#getTrimmedStringCollection, which will do the comma handling for you. For that matter, it might be nice to add a getIntegerCollection method to the Configuration class, to also handle the integer parsing.
          2. I find the variable name "splitted" rather unfortunate. How about "splitValues" ?
          3. There are a few spurious whitespace changes in TestDataNodeMetrics.
          4. You should add an entry in hdfs-default.xml for the new dfs.metrics.percentiles.intervals.key, even if it has an empty value, so that you can add a description of what it does, and the format of what it should be set to.
          5. I find the loop try/catch of AssertionError in TestDataNodeMetrics#testRoundTripAckPercentilesMetric kind of unfortunate. How about instead you get the list of DNs involved in the write pipeline via DFSOutputStream#getPipeline when writing the file, and then always assert the quantile gauges on the actual appropriate DN?
          6. If assertQuantileGauges are identical between TestDataNodeMetrics and TestNameNodeMetrics, how about refactoring those methods? Perhaps as a static method in DFSTestUtil?
          7. Could also stand to refactor the two new tests in TestNameNodeMetrics, since they appear identical, save for two values.
          Show
          Aaron T. Myers added a comment - Patch looks great, Andrew. Just a few little comments: Instead of doing Configuration#get(...) and then doing the comma separating yourself, you can use Configuration#getTrimmedStringCollection, which will do the comma handling for you. For that matter, it might be nice to add a getIntegerCollection method to the Configuration class, to also handle the integer parsing. I find the variable name "splitted" rather unfortunate. How about "splitValues" ? There are a few spurious whitespace changes in TestDataNodeMetrics. You should add an entry in hdfs-default.xml for the new dfs.metrics.percentiles.intervals.key, even if it has an empty value, so that you can add a description of what it does, and the format of what it should be set to. I find the loop try/catch of AssertionError in TestDataNodeMetrics#testRoundTripAckPercentilesMetric kind of unfortunate. How about instead you get the list of DNs involved in the write pipeline via DFSOutputStream#getPipeline when writing the file, and then always assert the quantile gauges on the actual appropriate DN? If assertQuantileGauges are identical between TestDataNodeMetrics and TestNameNodeMetrics, how about refactoring those methods? Perhaps as a static method in DFSTestUtil? Could also stand to refactor the two new tests in TestNameNodeMetrics, since they appear identical, save for two values.
          Hide
          Andrew Wang added a comment -

          Thanks for the review Aaron. Tried to address all your comments.

          I folded the new DN asserts into existing tests in TestDataNodeMetrics, and combined the two new NN tests into a single one since they were almost identical.

          I moved into #assertQuantileGauges in MetricsAsserts, since it was just copy pasted before.

          Show
          Andrew Wang added a comment - Thanks for the review Aaron. Tried to address all your comments. I folded the new DN asserts into existing tests in TestDataNodeMetrics , and combined the two new NN tests into a single one since they were almost identical. I moved into #assertQuantileGauges in MetricsAsserts , since it was just copy pasted before.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12538065/hdfs-3650-2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestDFSShell

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2911//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2911//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538065/hdfs-3650-2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestDFSShell +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2911//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2911//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          The latest patch looks great to me, Andrew. One tiny nit: in testRoundTripAckMetric in one place you do a "assertTrue(x != null)". Instead of this, you can just do "assertNotNull".

          +1 once this is addressed.

          Show
          Aaron T. Myers added a comment - The latest patch looks great to me, Andrew. One tiny nit: in testRoundTripAckMetric in one place you do a "assertTrue(x != null)". Instead of this, you can just do "assertNotNull ". +1 once this is addressed.
          Hide
          Andrew Wang added a comment -

          Nit addressed, and removed an unused import. Thanks!

          Show
          Andrew Wang added a comment - Nit addressed, and removed an unused import. Thanks!
          Hide
          Aaron T. Myers added a comment -

          I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Andrew.

          Show
          Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Andrew.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2589 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2589/)
          HDFS-3650. Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. (Revision 1366246)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366246
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/metrics/NameNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2589 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2589/ ) HDFS-3650 . Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. (Revision 1366246) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366246 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/metrics/NameNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2525 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2525/)
          HDFS-3650. Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. (Revision 1366246)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366246
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/metrics/NameNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2525 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2525/ ) HDFS-3650 . Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. (Revision 1366246) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366246 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/metrics/NameNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2545 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2545/)
          HDFS-3650. Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. (Revision 1366246)

          Result = FAILURE
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366246
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/metrics/NameNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2545 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2545/ ) HDFS-3650 . Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. (Revision 1366246) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366246 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/metrics/NameNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java

            People

            • Assignee:
              Andrew Wang
              Reporter:
              Andrew Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development