Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12916

Allow RPC scheduler/callqueue backoff using response times

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: ipc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently back off policy from HADOOP-10597 is hard coded to base on whether call queue is full. This ticket is open to allow flexible back off policies such as moving average of response time in RPC calls of different priorities.

      1. HADOOP-12916.00.patch
        75 kB
        Xiaoyu Yao
      2. HADOOP-12916.01.patch
        76 kB
        Xiaoyu Yao
      3. HADOOP-12916.02.patch
        78 kB
        Xiaoyu Yao
      4. HADOOP-12916.03.patch
        79 kB
        Xiaoyu Yao
      5. HADOOP-12916.04.patch
        79 kB
        Xiaoyu Yao
      6. HADOOP-12916.05.patch
        80 kB
        Xiaoyu Yao
      7. HADOOP-12916.06.patch
        82 kB
        Xiaoyu Yao
      8. HADOOP-12916.07.patch
        81 kB
        Xiaoyu Yao
      9. HADOOP-12916.08.patch
        85 kB
        Xiaoyu Yao

        Issue Links

          Activity

          Hide
          xyao Xiaoyu Yao added a comment -

          Attach a patch that
          1) Separate the scheduler interface and implementation from FCQ
          2) Allow DecayRpcScheduler to collect window based per priority level response time metrics.
          3) Allow optional backoff based on per priority level response time target based backoff.
          4) Additional unit test for response time target based backoff.
          5) Additional unit test to ensure backward compatibility for current FCQ deployment

          With this refactor, the FCQ is purely a weighted priority call queue and the DecayRpcScheduler is a fairness scheduler based on the callers' call count. The weight priority call queue can be plugged with other priority based scheduler for Qos.

          Show
          xyao Xiaoyu Yao added a comment - Attach a patch that 1) Separate the scheduler interface and implementation from FCQ 2) Allow DecayRpcScheduler to collect window based per priority level response time metrics. 3) Allow optional backoff based on per priority level response time target based backoff. 4) Additional unit test for response time target based backoff. 5) Additional unit test to ensure backward compatibility for current FCQ deployment With this refactor, the FCQ is purely a weighted priority call queue and the DecayRpcScheduler is a fairness scheduler based on the callers' call count. The weight priority call queue can be plugged with other priority based scheduler for Qos.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 6m 40s trunk passed
          +1 compile 6m 8s trunk passed with JDK v1.8.0_74
          +1 compile 6m 44s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 34s trunk passed
          +1 javadoc 0m 51s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 53s the patch passed with JDK v1.8.0_74
          -1 javac 10m 17s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738)
          +1 javac 5m 53s the patch passed
          +1 compile 6m 44s the patch passed with JDK v1.7.0_95
          -1 javac 17m 1s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734)
          +1 javac 6m 44s the patch passed
          -1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 29 new + 548 unchanged - 19 fixed = 577 total (was 567)
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 1m 52s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 0m 51s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
          +1 unit 6m 50s hadoop-common in the patch passed with JDK v1.8.0_74.
          +1 unit 7m 13s hadoop-common in the patch passed with JDK v1.7.0_95.
          -1 asflicense 0m 22s Patch generated 1 ASF License warnings.
          58m 58s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Unread field:DecayRpcScheduler.java:[line 636]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794012/HADOOP-12916.00.patch
          JIRA Issue HADOOP-12916
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f12f9447d205 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dc951e6
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt
          javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 40s trunk passed +1 compile 6m 8s trunk passed with JDK v1.8.0_74 +1 compile 6m 44s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 34s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 53s the patch passed with JDK v1.8.0_74 -1 javac 10m 17s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738) +1 javac 5m 53s the patch passed +1 compile 6m 44s the patch passed with JDK v1.7.0_95 -1 javac 17m 1s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734) +1 javac 6m 44s the patch passed -1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 29 new + 548 unchanged - 19 fixed = 577 total (was 567) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 52s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 0m 51s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 +1 unit 6m 50s hadoop-common in the patch passed with JDK v1.8.0_74. +1 unit 7m 13s hadoop-common in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 22s Patch generated 1 ASF License warnings. 58m 58s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Unread field:DecayRpcScheduler.java: [line 636] Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794012/HADOOP-12916.00.patch JIRA Issue HADOOP-12916 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f12f9447d205 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dc951e6 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8867/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Thanks Xiaoyu Yao. Some comments on the patch.

          • In CallQueueManager.createScheduler,
            • createScheduler should be static.
            • Should priorityLevels be passed in the second try-catch?
          • Remove public from getPriorityLevel.
          • In parseNumLevels, print ns in the warning.
          • backOffByResponseTimeEnabled should be declared with a lower case boolean.
          • The following are not used. Let's remove them.
            • RpcScheduler.getPriorityLevels()
            • getAverageResponseTime(int level)
            • setScheduler
            • MetricsProxy.numLevels
          • DefaultRpcScheduler needs license header.
          • averageResponseTimeInLastWindow and callCountInLastWindow need to be synchronized.
          • Let's remove the unused import from Schedulable.
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Thanks Xiaoyu Yao . Some comments on the patch. In CallQueueManager.createScheduler, createScheduler should be static. Should priorityLevels be passed in the second try-catch? Remove public from getPriorityLevel. In parseNumLevels, print ns in the warning. backOffByResponseTimeEnabled should be declared with a lower case boolean. The following are not used. Let's remove them. RpcScheduler.getPriorityLevels() getAverageResponseTime(int level) setScheduler MetricsProxy.numLevels DefaultRpcScheduler needs license header. averageResponseTimeInLastWindow and callCountInLastWindow need to be synchronized. Let's remove the unused import from Schedulable.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks for Tsz Wo Nicholas Sze for the review. I've update the patch that addresses your comments and the checkstyle/findbugs issues from Jenkins.

          "Should priorityLevels be passed in the second try/catch in CallQueueManager#createScheduler?"

          We already pass the int.class which is the priority level in the second try/catch.

          Show
          xyao Xiaoyu Yao added a comment - Thanks for Tsz Wo Nicholas Sze for the review. I've update the patch that addresses your comments and the checkstyle/findbugs issues from Jenkins. "Should priorityLevels be passed in the second try/catch in CallQueueManager#createScheduler?" We already pass the int.class which is the priority level in the second try/catch.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          I mean ctor.newInstance() below.

          +      Constructor<T> ctor = theClass.getDeclaredConstructor(int.class);
          +      return ctor.newInstance();
          
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - I mean ctor.newInstance() below. + Constructor<T> ctor = theClass.getDeclaredConstructor( int .class); + return ctor.newInstance();
          Hide
          xyao Xiaoyu Yao added a comment -

          Got it. Update the patch with the priorityLevel passed for the second constructor.

          Show
          xyao Xiaoyu Yao added a comment - Got it. Update the patch with the priorityLevel passed for the second constructor.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 6m 29s trunk passed
          +1 compile 5m 48s trunk passed with JDK v1.8.0_74
          +1 compile 6m 50s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 54s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 40s the patch passed
          +1 compile 5m 51s the patch passed with JDK v1.8.0_74
          -1 javac 10m 19s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738)
          +1 javac 5m 51s the patch passed
          +1 compile 6m 41s the patch passed with JDK v1.7.0_95
          -1 javac 17m 0s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734)
          +1 javac 6m 41s the patch passed
          -1 checkstyle 0m 27s hadoop-common-project/hadoop-common: patch generated 4 new + 544 unchanged - 23 fixed = 548 total (was 567)
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 52s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95
          -1 unit 6m 26s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 6m 46s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 22s Patch generated 2 ASF License warnings.
          57m 43s



          Reason Tests
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794304/HADOOP-12916.01.patch
          JIRA Issue HADOOP-12916
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 85b5cde6afe1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 33239c9
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt
          javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 29s trunk passed +1 compile 5m 48s trunk passed with JDK v1.8.0_74 +1 compile 6m 50s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 35s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 5m 51s the patch passed with JDK v1.8.0_74 -1 javac 10m 19s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738) +1 javac 5m 51s the patch passed +1 compile 6m 41s the patch passed with JDK v1.7.0_95 -1 javac 17m 0s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734) +1 javac 6m 41s the patch passed -1 checkstyle 0m 27s hadoop-common-project/hadoop-common: patch generated 4 new + 544 unchanged - 23 fixed = 548 total (was 567) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95 -1 unit 6m 26s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 6m 46s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 22s Patch generated 2 ASF License warnings. 57m 43s Reason Tests JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794304/HADOOP-12916.01.patch JIRA Issue HADOOP-12916 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 85b5cde6afe1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 33239c9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8875/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 6m 34s trunk passed
          +1 compile 5m 59s trunk passed with JDK v1.8.0_74
          +1 compile 6m 42s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 34s trunk passed
          +1 javadoc 0m 52s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 54s the patch passed with JDK v1.8.0_74
          -1 javac 10m 18s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738)
          +1 javac 5m 54s the patch passed
          +1 compile 6m 43s the patch passed with JDK v1.7.0_95
          -1 javac 17m 1s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734)
          +1 javac 6m 43s the patch passed
          -1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 4 new + 544 unchanged - 23 fixed = 548 total (was 567)
          +1 mvnsite 0m 55s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 49s the patch passed
          +1 javadoc 0m 52s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
          -1 unit 6m 35s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 6m 44s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 22s Patch generated 2 ASF License warnings.
          57m 54s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.ha.TestZKFailoverController
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794304/HADOOP-12916.01.patch
          JIRA Issue HADOOP-12916
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux dcceceb248af 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 33239c9
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt
          javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 34s trunk passed +1 compile 5m 59s trunk passed with JDK v1.8.0_74 +1 compile 6m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 34s trunk passed +1 javadoc 0m 52s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 54s the patch passed with JDK v1.8.0_74 -1 javac 10m 18s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738) +1 javac 5m 54s the patch passed +1 compile 6m 43s the patch passed with JDK v1.7.0_95 -1 javac 17m 1s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734) +1 javac 6m 43s the patch passed -1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 4 new + 544 unchanged - 23 fixed = 548 total (was 567) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 -1 unit 6m 35s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 6m 44s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 22s Patch generated 2 ASF License warnings. 57m 54s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.ha.TestZKFailoverController JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794304/HADOOP-12916.01.patch JIRA Issue HADOOP-12916 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dcceceb248af 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 33239c9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8877/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Update the patch that cleans up the javac and checkstyle issues.

          Show
          xyao Xiaoyu Yao added a comment - Update the patch that cleans up the javac and checkstyle issues.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 8m 20s trunk passed
          +1 compile 9m 15s trunk passed with JDK v1.8.0_74
          +1 compile 8m 13s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 1m 5s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 43s trunk passed
          +1 javadoc 1m 6s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 11s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 49s the patch passed
          +1 compile 9m 9s the patch passed with JDK v1.8.0_74
          +1 javac 9m 9s the patch passed
          +1 compile 8m 8s the patch passed with JDK v1.7.0_95
          +1 javac 8m 8s the patch passed
          -1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 2 new + 523 unchanged - 44 fixed = 525 total (was 567)
          +1 mvnsite 1m 4s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 59s the patch passed
          +1 javadoc 1m 8s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 12s the patch passed with JDK v1.7.0_95
          -1 unit 10m 7s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 9m 36s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 24s Patch generated 2 ASF License warnings.
          77m 25s



          Reason Tests
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794368/HADOOP-12916.02.patch
          JIRA Issue HADOOP-12916
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 07f2e1d70d14 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 33239c9
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 8m 20s trunk passed +1 compile 9m 15s trunk passed with JDK v1.8.0_74 +1 compile 8m 13s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 43s trunk passed +1 javadoc 1m 6s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 11s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 49s the patch passed +1 compile 9m 9s the patch passed with JDK v1.8.0_74 +1 javac 9m 9s the patch passed +1 compile 8m 8s the patch passed with JDK v1.7.0_95 +1 javac 8m 8s the patch passed -1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 2 new + 523 unchanged - 44 fixed = 525 total (was 567) +1 mvnsite 1m 4s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 59s the patch passed +1 javadoc 1m 8s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 12s the patch passed with JDK v1.7.0_95 -1 unit 10m 7s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 9m 36s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 24s Patch generated 2 ASF License warnings. 77m 25s Reason Tests JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794368/HADOOP-12916.02.patch JIRA Issue HADOOP-12916 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 07f2e1d70d14 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 33239c9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8880/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Thanks for the update. I think averageResponseTimeInLastWindow and callCountInLastWindow probably need to be declared with AtomicLongArray and AtomicDoubleArray since we also need to synchronize individual values in the arrays. Patch looks good other than that.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Thanks for the update. I think averageResponseTimeInLastWindow and callCountInLastWindow probably need to be declared with AtomicLongArray and AtomicDoubleArray since we also need to synchronize individual values in the arrays. Patch looks good other than that.
          Hide
          xyao Xiaoyu Yao added a comment -

          Attached a new patch that uses AtomLongArray/AtomicDoubleArray for previous window response time/count. Also include some minor clean up for check style issue.

          Show
          xyao Xiaoyu Yao added a comment - Attached a new patch that uses AtomLongArray/AtomicDoubleArray for previous window response time/count. Also include some minor clean up for check style issue.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          The new patch looks good. One last comment:

          • Since averageResponseTimeInLastWindow and callCountInLastWindow are atomic arrays, we can remove synchronized (this) when accessing them.
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - The new patch looks good. One last comment: Since averageResponseTimeInLastWindow and callCountInLastWindow are atomic arrays, we can remove synchronized (this) when accessing them.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Tsz Wo Nicholas Sze! Attach patch 04 to remove the redundant synchronization when accessing atomic arrays.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Tsz Wo Nicholas Sze ! Attach patch 04 to remove the redundant synchronization when accessing atomic arrays.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          +1 the new patch looks good.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - +1 the new patch looks good.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 7m 4s trunk passed
          +1 compile 6m 29s trunk passed with JDK v1.8.0_74
          +1 compile 7m 18s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 59s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 40s trunk passed
          +1 javadoc 1m 0s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 8s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 44s the patch passed
          +1 compile 6m 36s the patch passed with JDK v1.8.0_74
          +1 javac 6m 36s the patch passed
          +1 compile 6m 42s the patch passed with JDK v1.7.0_95
          +1 javac 6m 42s the patch passed
          +1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568)
          +1 mvnsite 0m 55s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 54s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
          -1 unit 6m 40s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 7m 2s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 23s Patch generated 2 ASF License warnings.
          61m 6s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.net.TestClusterTopology
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794822/HADOOP-12916.03.patch
          JIRA Issue HADOOP-12916
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c4ae4f3b558e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e7ed05e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 7m 4s trunk passed +1 compile 6m 29s trunk passed with JDK v1.8.0_74 +1 compile 7m 18s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 1m 0s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 8s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 44s the patch passed +1 compile 6m 36s the patch passed with JDK v1.8.0_74 +1 javac 6m 36s the patch passed +1 compile 6m 42s the patch passed with JDK v1.7.0_95 +1 javac 6m 42s the patch passed +1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 54s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 -1 unit 6m 40s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 7m 2s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 23s Patch generated 2 ASF License warnings. 61m 6s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.net.TestClusterTopology JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794822/HADOOP-12916.03.patch JIRA Issue HADOOP-12916 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c4ae4f3b558e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e7ed05e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8896/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 6m 43s trunk passed
          +1 compile 6m 13s trunk passed with JDK v1.8.0_74
          +1 compile 6m 55s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 59s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 36s trunk passed
          +1 javadoc 0m 56s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 6s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 6m 11s the patch passed with JDK v1.8.0_74
          +1 javac 6m 11s the patch passed
          +1 compile 6m 50s the patch passed with JDK v1.7.0_95
          +1 javac 6m 50s the patch passed
          +1 checkstyle 0m 27s hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568)
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 52s the patch passed
          +1 javadoc 0m 55s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 6s the patch passed with JDK v1.7.0_95
          -1 unit 6m 41s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 6m 59s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 21s Patch generated 2 ASF License warnings.
          59m 38s



          Reason Tests
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794848/HADOOP-12916.04.patch
          JIRA Issue HADOOP-12916
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5f5716499f03 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 57ead18
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 43s trunk passed +1 compile 6m 13s trunk passed with JDK v1.8.0_74 +1 compile 6m 55s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 36s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 6s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 6m 11s the patch passed with JDK v1.8.0_74 +1 javac 6m 11s the patch passed +1 compile 6m 50s the patch passed with JDK v1.7.0_95 +1 javac 6m 50s the patch passed +1 checkstyle 0m 27s hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 6s the patch passed with JDK v1.7.0_95 -1 unit 6m 41s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 6m 59s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 21s Patch generated 2 ASF License warnings. 59m 38s Reason Tests JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794848/HADOOP-12916.04.patch JIRA Issue HADOOP-12916 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5f5716499f03 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 57ead18 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8899/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user xiaoyuyao opened a pull request:

          https://github.com/apache/hadoop/pull/86

          HADOOP-12916

          Create a github PR for code review of v04 patch on the JIRA.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/xiaoyuyao/hadoop HADOOP-12916

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/86.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #86


          commit 93cf7377980a69f8b5b22453419a95b58a19c2aa
          Author: Xiaoyu Yao <xyao@apache.org>
          Date: 2016-03-26T02:45:48Z

          HADOOP-12916.patch.04


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user xiaoyuyao opened a pull request: https://github.com/apache/hadoop/pull/86 HADOOP-12916 Create a github PR for code review of v04 patch on the JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xiaoyuyao/hadoop HADOOP-12916 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/86.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #86 commit 93cf7377980a69f8b5b22453419a95b58a19c2aa Author: Xiaoyu Yao <xyao@apache.org> Date: 2016-03-26T02:45:48Z HADOOP-12916 .patch.04
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user arp7 commented on a diff in the pull request:

          https://github.com/apache/hadoop/pull/86#discussion_r57606373

          — Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java —
          @@ -1636,6 +1640,15 @@ public long getTimeDuration(String name, long defaultValue, TimeUnit unit)

          { return unit.convert(Long.parseLong(vStr), vUnit.unit()); }

          + public long[] getTimeDurations(String name, TimeUnit unit) {
          — End diff –

          Consider returning a List.

          Show
          githubbot ASF GitHub Bot added a comment - Github user arp7 commented on a diff in the pull request: https://github.com/apache/hadoop/pull/86#discussion_r57606373 — Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java — @@ -1636,6 +1640,15 @@ public long getTimeDuration(String name, long defaultValue, TimeUnit unit) { return unit.convert(Long.parseLong(vStr), vUnit.unit()); } + public long[] getTimeDurations(String name, TimeUnit unit) { — End diff – Consider returning a List.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user arp7 commented on a diff in the pull request:

          https://github.com/apache/hadoop/pull/86#discussion_r57606818

          — Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java —
          @@ -49,25 +57,68 @@
          private final AtomicReference<BlockingQueue<E>> putRef;
          private final AtomicReference<BlockingQueue<E>> takeRef;

          + private RpcScheduler scheduler;
          +
          public CallQueueManager(Class<? extends BlockingQueue<E>> backingClass,
          + Class<? extends RpcScheduler> schedulerClass,
          boolean clientBackOffEnabled, int maxQueueSize, String namespace,
          Configuration conf)

          { + int priorityLevels = parseNumLevels(namespace, conf); + this.scheduler = createScheduler(schedulerClass, priorityLevels, + namespace, conf); BlockingQueue<E> bq = createCallQueueInstance(backingClass, - maxQueueSize, namespace, conf); + priorityLevels, maxQueueSize, namespace, conf); this.clientBackOffEnabled = clientBackOffEnabled; this.putRef = new AtomicReference<BlockingQueue<E>>(bq); this.takeRef = new AtomicReference<BlockingQueue<E>>(bq); LOG.info("Using callQueue " + backingClass); }

          + private static <T extends RpcScheduler> T createScheduler(
          + Class<T> theClass, int priorityLevels, String ns, Configuration conf) {
          + // Used for custom, configurable scheduler
          + try

          { + Constructor<T> ctor = theClass.getDeclaredConstructor(int.class, + String.class, Configuration.class); + return ctor.newInstance(priorityLevels, ns, conf); + }

          catch (RuntimeException e) {
          — End diff –

          See HDFS-9478 which is fixing exception handling when constructing callqueue instances. We could use a similar fix for createScheduler.

          Show
          githubbot ASF GitHub Bot added a comment - Github user arp7 commented on a diff in the pull request: https://github.com/apache/hadoop/pull/86#discussion_r57606818 — Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java — @@ -49,25 +57,68 @@ private final AtomicReference<BlockingQueue<E>> putRef; private final AtomicReference<BlockingQueue<E>> takeRef; + private RpcScheduler scheduler; + public CallQueueManager(Class<? extends BlockingQueue<E>> backingClass, + Class<? extends RpcScheduler> schedulerClass, boolean clientBackOffEnabled, int maxQueueSize, String namespace, Configuration conf) { + int priorityLevels = parseNumLevels(namespace, conf); + this.scheduler = createScheduler(schedulerClass, priorityLevels, + namespace, conf); BlockingQueue<E> bq = createCallQueueInstance(backingClass, - maxQueueSize, namespace, conf); + priorityLevels, maxQueueSize, namespace, conf); this.clientBackOffEnabled = clientBackOffEnabled; this.putRef = new AtomicReference<BlockingQueue<E>>(bq); this.takeRef = new AtomicReference<BlockingQueue<E>>(bq); LOG.info("Using callQueue " + backingClass); } + private static <T extends RpcScheduler> T createScheduler( + Class<T> theClass, int priorityLevels, String ns, Configuration conf) { + // Used for custom, configurable scheduler + try { + Constructor<T> ctor = theClass.getDeclaredConstructor(int.class, + String.class, Configuration.class); + return ctor.newInstance(priorityLevels, ns, conf); + } catch (RuntimeException e) { — End diff – See HDFS-9478 which is fixing exception handling when constructing callqueue instances. We could use a similar fix for createScheduler.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user xiaoyuyao commented on a diff in the pull request:

          https://github.com/apache/hadoop/pull/86#discussion_r57619429

          — Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java —
          @@ -1636,6 +1640,15 @@ public long getTimeDuration(String name, long defaultValue, TimeUnit unit)

          { return unit.convert(Long.parseLong(vStr), vUnit.unit()); }

          + public long[] getTimeDurations(String name, TimeUnit unit) {
          — End diff –

          Good point. I will address that in the next patch.

          Show
          githubbot ASF GitHub Bot added a comment - Github user xiaoyuyao commented on a diff in the pull request: https://github.com/apache/hadoop/pull/86#discussion_r57619429 — Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java — @@ -1636,6 +1640,15 @@ public long getTimeDuration(String name, long defaultValue, TimeUnit unit) { return unit.convert(Long.parseLong(vStr), vUnit.unit()); } + public long[] getTimeDurations(String name, TimeUnit unit) { — End diff – Good point. I will address that in the next patch.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Arpit Agarwal and Tsz Wo Nicholas Sze for the review.
          Update the patch that
          1) Fixing exception handling when constructing the scheduler instance.
          2) Keep the default decay factor and decay windows size to the same as the original FCQ.
          3) Enable decay on the average response time for scheduler decision.

          I don't change the Configuration#getTimeDurations() to return a List because it incurs extra box/unbox overhead. A long array should be good for many cases.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Arpit Agarwal and Tsz Wo Nicholas Sze for the review. Update the patch that 1) Fixing exception handling when constructing the scheduler instance. 2) Keep the default decay factor and decay windows size to the same as the original FCQ. 3) Enable decay on the average response time for scheduler decision. I don't change the Configuration#getTimeDurations() to return a List because it incurs extra box/unbox overhead. A long array should be good for many cases.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 6m 54s trunk passed
          +1 compile 6m 10s trunk passed with JDK v1.8.0_77
          +1 compile 6m 48s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 33s trunk passed
          +1 javadoc 0m 56s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 40s the patch passed
          +1 compile 6m 10s the patch passed with JDK v1.8.0_77
          +1 javac 6m 10s the patch passed
          +1 compile 6m 48s the patch passed with JDK v1.7.0_95
          +1 javac 6m 48s the patch passed
          +1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 0 new + 522 unchanged - 45 fixed = 522 total (was 567)
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 1s Patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 55s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
          -1 unit 7m 8s hadoop-common in the patch failed with JDK v1.8.0_77.
          -1 unit 7m 10s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 23s Patch generated 3 ASF License warnings.
          60m 1s



          Reason Tests
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12916
          GITHUB PR https://github.com/apache/hadoop/pull/86
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4be9d0f495bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8018280
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 54s trunk passed +1 compile 6m 10s trunk passed with JDK v1.8.0_77 +1 compile 6m 48s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 33s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 6m 10s the patch passed with JDK v1.8.0_77 +1 javac 6m 10s the patch passed +1 compile 6m 48s the patch passed with JDK v1.7.0_95 +1 javac 6m 48s the patch passed +1 checkstyle 0m 26s hadoop-common-project/hadoop-common: patch generated 0 new + 522 unchanged - 45 fixed = 522 total (was 567) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 1s Patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 -1 unit 7m 8s hadoop-common in the patch failed with JDK v1.8.0_77. -1 unit 7m 10s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 23s Patch generated 3 ASF License warnings. 60m 1s Reason Tests JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12916 GITHUB PR https://github.com/apache/hadoop/pull/86 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4be9d0f495bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8018280 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8952/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -
          +        double decayedAvgRespTime = (responseTimeAvgInLastWindow.get(i) > 0.0) ?
          +            decayFactor * responseTimeAvgInLastWindow.get(i) +
          +            (1 - decayFactor) * averageResponseTime : averageResponseTime;
          

          For the decayed case, should the formula be decayFactor*responseTimeAvgInLastWindow.get( i) + averageResponseTime, i.e. no (1 - decayFactor) in the second term?

          BTW, the if-statement can be rewritten as below to make it shorter.

                final double lastAvg = responseTimeAvgInLastWindow.get(i);
                if (enableDecay && lastAvg > 0) {
                  final double decayed =  decayFactor * lastAvg + averageResponseTime;
                  responseTimeAvgInLastWindow.set(i, decayed);
                } else {
                  responseTimeAvgInLastWindow.set(i, averageResponseTime);
                }
          
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - + double decayedAvgRespTime = (responseTimeAvgInLastWindow.get(i) > 0.0) ? + decayFactor * responseTimeAvgInLastWindow.get(i) + + (1 - decayFactor) * averageResponseTime : averageResponseTime; For the decayed case, should the formula be decayFactor*responseTimeAvgInLastWindow.get( i) + averageResponseTime , i.e. no (1 - decayFactor) in the second term? BTW, the if-statement can be rewritten as below to make it shorter. final double lastAvg = responseTimeAvgInLastWindow.get(i); if (enableDecay && lastAvg > 0) { final double decayed = decayFactor * lastAvg + averageResponseTime; responseTimeAvgInLastWindow.set(i, decayed); } else { responseTimeAvgInLastWindow.set(i, averageResponseTime); }
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Tsz Wo Nicholas Sze for the review. I've updated the patch based on your suggestion. Also rebase the patch to trunk with additional unit test added for the scheduler constructor exception handling.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Tsz Wo Nicholas Sze for the review. I've updated the patch based on your suggestion. Also rebase the patch to trunk with additional unit test added for the scheduler constructor exception handling.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 11s HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-12916
          GITHUB PR https://github.com/apache/hadoop/pull/86
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8967/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 11s HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-12916 GITHUB PR https://github.com/apache/hadoop/pull/86 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8967/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 5s HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-12916
          GITHUB PR https://github.com/apache/hadoop/pull/86
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8968/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-12916 GITHUB PR https://github.com/apache/hadoop/pull/86 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8968/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 11s HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-12916
          GITHUB PR https://github.com/apache/hadoop/pull/86
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8970/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 11s HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-12916 GITHUB PR https://github.com/apache/hadoop/pull/86 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8970/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          The patch applies to my local trunk branch without problem. Not sure why Jenkins failed. Reattach..

          Show
          xyao Xiaoyu Yao added a comment - The patch applies to my local trunk branch without problem. Not sure why Jenkins failed. Reattach..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          -1 mvninstall 11m 47s root in trunk failed.
          +1 compile 17m 18s trunk passed with JDK v1.8.0_74
          +1 compile 11m 20s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 42s trunk passed
          +1 mvnsite 1m 27s trunk passed
          +1 mvneclipse 0m 22s trunk passed
          +1 findbugs 2m 19s trunk passed
          +1 javadoc 1m 34s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 31s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 58s the patch passed
          +1 compile 11m 29s the patch passed with JDK v1.8.0_74
          +1 javac 11m 29s the patch passed
          +1 compile 9m 59s the patch passed with JDK v1.7.0_95
          +1 javac 9m 59s the patch passed
          +1 checkstyle 0m 39s hadoop-common-project/hadoop-common: patch generated 0 new + 522 unchanged - 45 fixed = 522 total (was 567)
          +1 mvnsite 1m 18s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 20s the patch passed
          +1 javadoc 1m 19s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 27s the patch passed with JDK v1.7.0_95
          -1 unit 10m 22s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 9m 55s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 31s Patch generated 1 ASF License warnings.
          101m 3s



          Reason Tests
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12916
          GITHUB PR https://github.com/apache/hadoop/pull/86
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0d7712a39e4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 60e4116
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/branch-mvninstall-root.txt
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. -1 mvninstall 11m 47s root in trunk failed. +1 compile 17m 18s trunk passed with JDK v1.8.0_74 +1 compile 11m 20s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 42s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 22s trunk passed +1 findbugs 2m 19s trunk passed +1 javadoc 1m 34s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 31s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 58s the patch passed +1 compile 11m 29s the patch passed with JDK v1.8.0_74 +1 javac 11m 29s the patch passed +1 compile 9m 59s the patch passed with JDK v1.7.0_95 +1 javac 9m 59s the patch passed +1 checkstyle 0m 39s hadoop-common-project/hadoop-common: patch generated 0 new + 522 unchanged - 45 fixed = 522 total (was 567) +1 mvnsite 1m 18s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 20s the patch passed +1 javadoc 1m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 27s the patch passed with JDK v1.7.0_95 -1 unit 10m 22s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 9m 55s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 31s Patch generated 1 ASF License warnings. 101m 3s Reason Tests JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12916 GITHUB PR https://github.com/apache/hadoop/pull/86 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0d7712a39e4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 60e4116 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8973/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Attach a new patch v08 to get a clean Jenkins run.
          Delta of changes: Adding a new unit test ensuring DecayRpcScheduler works without FCQ such as LinkedBlockingQueue.

          Show
          xyao Xiaoyu Yao added a comment - Attach a new patch v08 to get a clean Jenkins run. Delta of changes: Adding a new unit test ensuring DecayRpcScheduler works without FCQ such as LinkedBlockingQueue.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          +1 the new patch looks good.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - +1 the new patch looks good.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 6m 27s trunk passed
          +1 compile 5m 48s trunk passed with JDK v1.8.0_77
          +1 compile 6m 43s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 34s trunk passed
          +1 javadoc 0m 54s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 59s the patch passed with JDK v1.8.0_77
          +1 javac 5m 59s the patch passed
          +1 compile 7m 48s the patch passed with JDK v1.7.0_95
          +1 javac 7m 48s the patch passed
          +1 checkstyle 0m 28s hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568)
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 7s the patch passed
          +1 javadoc 1m 3s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 10s the patch passed with JDK v1.7.0_95
          -1 unit 8m 5s hadoop-common in the patch failed with JDK v1.8.0_77.
          -1 unit 8m 2s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 23s Patch generated 2 ASF License warnings.
          62m 35s



          Reason Tests
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12916
          GITHUB PR https://github.com/apache/hadoop/pull/86
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7e61c97c762d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 32c0c3e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 27s trunk passed +1 compile 5m 48s trunk passed with JDK v1.8.0_77 +1 compile 6m 43s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 34s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 59s the patch passed with JDK v1.8.0_77 +1 javac 5m 59s the patch passed +1 compile 7m 48s the patch passed with JDK v1.7.0_95 +1 javac 7m 48s the patch passed +1 checkstyle 0m 28s hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 7s the patch passed +1 javadoc 1m 3s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 10s the patch passed with JDK v1.7.0_95 -1 unit 8m 5s hadoop-common in the patch failed with JDK v1.8.0_77. -1 unit 8m 2s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 23s Patch generated 2 ASF License warnings. 62m 35s Reason Tests JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12916 GITHUB PR https://github.com/apache/hadoop/pull/86 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7e61c97c762d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 32c0c3e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8982/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9534/)
          HADOOP-12916. Allow RPC scheduler/callqueue backoff using response (xyao: rev d95c6eb32cec7768ac418fb467b1198ccf3cf0dc)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestDecayRpcScheduler.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DecayRpcSchedulerMXBean.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestFairCallQueue.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DefaultRpcScheduler.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIdentityProviders.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestCallQueueManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DecayRpcScheduler.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RpcScheduler.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/FairCallQueue.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Schedulable.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WritableRpcEngine.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9534/ ) HADOOP-12916 . Allow RPC scheduler/callqueue backoff using response (xyao: rev d95c6eb32cec7768ac418fb467b1198ccf3cf0dc) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestDecayRpcScheduler.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DecayRpcSchedulerMXBean.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestFairCallQueue.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DefaultRpcScheduler.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIdentityProviders.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestCallQueueManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DecayRpcScheduler.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RpcScheduler.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/FairCallQueue.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Schedulable.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WritableRpcEngine.java
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Tsz Wo Nicholas Sze and Arpit Agarwal for the review.
          I've committed the patch to trunk, branch-2 and branch-2.8 and will add a short release note on the new configuration keys and the deprecated keys. More detailed document will be added with HADOOP-12980.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Tsz Wo Nicholas Sze and Arpit Agarwal for the review. I've committed the patch to trunk, branch-2 and branch-2.8 and will add a short release note on the new configuration keys and the deprecated keys. More detailed document will be added with HADOOP-12980 .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          After this Jira in following test is failing in Trunk. Raised HDFS-10253 to track this.. Since this testcase failure from HDFS project,jenkins did not catch here

           Error Details 
          org.apache.hadoop.TestRefreshCallQueue$MockCallQueue could not be constructed.
          
           Stack Trace 
          java.lang.RuntimeException: org.apache.hadoop.TestRefreshCallQueue$MockCallQueue could not be constructed.
          	at org.apache.hadoop.ipc.CallQueueManager.createCallQueueInstance(CallQueueManager.java:164)
          	at org.apache.hadoop.ipc.CallQueueManager.<init>(CallQueueManager.java:70)
          	at org.apache.hadoop.ipc.Server.<init>(Server.java:2579)
          	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:958)
          	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:535)
          	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:510)
          
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - After this Jira in following test is failing in Trunk. Raised HDFS-10253 to track this.. Since this testcase failure from HDFS project,jenkins did not catch here Error Details org.apache.hadoop.TestRefreshCallQueue$MockCallQueue could not be constructed. Stack Trace java.lang.RuntimeException: org.apache.hadoop.TestRefreshCallQueue$MockCallQueue could not be constructed. at org.apache.hadoop.ipc.CallQueueManager.createCallQueueInstance(CallQueueManager.java:164) at org.apache.hadoop.ipc.CallQueueManager.<init>(CallQueueManager.java:70) at org.apache.hadoop.ipc.Server.<init>(Server.java:2579) at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:958) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:535) at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:510)
          Hide
          kihwal Kihwal Lee added a comment -

          For namenode RPC server, doesn't the rpc response time include the namesystem lock wait time? It may not work correctly for namenode then.

          Show
          kihwal Kihwal Lee added a comment - For namenode RPC server, doesn't the rpc response time include the namesystem lock wait time? It may not work correctly for namenode then.
          Hide
          xyao Xiaoyu Yao added a comment -

          Kihwal Lee, the end to end rpc response time includes the queueing time and the processing time. The namesystem lock wait time is counted as part of the the processing time. Can you elaborate on "It may not work correctly for namenode?". Thanks!

          Show
          xyao Xiaoyu Yao added a comment - Kihwal Lee , the end to end rpc response time includes the queueing time and the processing time. The namesystem lock wait time is counted as part of the the processing time. Can you elaborate on "It may not work correctly for namenode?". Thanks!

            People

            • Assignee:
              xyao Xiaoyu Yao
              Reporter:
              xyao Xiaoyu Yao
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development