Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13227

AsyncCallHandler should use an event driven architecture to handle async calls

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: io, ipc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA is to address Jing's comments in HADOOP-13226.

      1. c13227_20160602.patch
        24 kB
        Tsz Wo Nicholas Sze
      2. c13227_20160606.patch
        26 kB
        Tsz Wo Nicholas Sze
      3. c13227_20160607.patch
        26 kB
        Tsz Wo Nicholas Sze
      4. c13227_20160608.patch
        29 kB
        Tsz Wo Nicholas Sze
      5. c13227_20160608b.patch
        30 kB
        Tsz Wo Nicholas Sze
      6. c13227_20160616.patch
        31 kB
        Tsz Wo Nicholas Sze
      7. c13227_20160621.patch
        30 kB
        Tsz Wo Nicholas Sze
      8. c13227_20160624.patch
        30 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          c13227_20160602.patch: uses an event driven architecture.

          After the patch, the running time of TestAsyncHDFSWithHA is decreased from around a minute to around 6 seconds in my machine since the unnecessary waits are eliminated.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - c13227_20160602.patch: uses an event driven architecture. After the patch, the running time of TestAsyncHDFSWithHA is decreased from around a minute to around 6 seconds in my machine since the unnecessary waits are eliminated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 29s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 18s Maven dependency ordering for branch
          +1 mvninstall 6m 44s trunk passed
          +1 compile 6m 43s trunk passed
          +1 checkstyle 1m 30s trunk passed
          +1 mvnsite 1m 51s trunk passed
          +1 mvneclipse 0m 23s trunk passed
          +1 findbugs 3m 11s trunk passed
          +1 javadoc 2m 1s trunk passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 35s the patch passed
          +1 compile 6m 45s the patch passed
          +1 javac 6m 45s the patch passed
          -1 checkstyle 1m 25s root: The patch generated 5 new + 213 unchanged - 1 fixed = 218 total (was 214)
          +1 mvnsite 1m 53s the patch passed
          +1 mvneclipse 0m 22s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 31s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 2m 2s the patch passed
          +1 unit 8m 13s hadoop-common in the patch passed.
          -1 unit 59m 35s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          109m 55s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce() calls Thread.sleep() with a lock held At RetryInvocationHandler.java:lock held At RetryInvocationHandler.java:[line 107]
          Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807818/c13227_20160602.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 44459a3170c4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 97e2449
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 18s Maven dependency ordering for branch +1 mvninstall 6m 44s trunk passed +1 compile 6m 43s trunk passed +1 checkstyle 1m 30s trunk passed +1 mvnsite 1m 51s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 3m 11s trunk passed +1 javadoc 2m 1s trunk passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 35s the patch passed +1 compile 6m 45s the patch passed +1 javac 6m 45s the patch passed -1 checkstyle 1m 25s root: The patch generated 5 new + 213 unchanged - 1 fixed = 218 total (was 214) +1 mvnsite 1m 53s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 31s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 2m 2s the patch passed +1 unit 8m 13s hadoop-common in the patch passed. -1 unit 59m 35s hadoop-hdfs in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 109m 55s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce() calls Thread.sleep() with a lock held At RetryInvocationHandler.java:lock held At RetryInvocationHandler.java: [line 107] Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807818/c13227_20160602.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 44459a3170c4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 97e2449 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9658/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks a lot for the great work, Tsz Wo Nicholas Sze! The patch looks good to me. Some minor comments:

          1. In RetryInfo#newRetryInfo, looks like failover, fail, and retry are mutual exclusive? Then can we simplify RetryInfo and only keep one RetryAction there?
          2. In AsyncCallHandler, the queue will be accessed by many threads. Therefore maybe we should consider directly using ConcurrentLinkedQueue which utilizes an efficient non-block algorithm.
          3. In checkCalls, do you think we can avoid the poll+offer operations for a not-done-yet call?
          Show
          jingzhao Jing Zhao added a comment - Thanks a lot for the great work, Tsz Wo Nicholas Sze ! The patch looks good to me. Some minor comments: In RetryInfo#newRetryInfo, looks like failover, fail, and retry are mutual exclusive? Then can we simplify RetryInfo and only keep one RetryAction there? In AsyncCallHandler, the queue will be accessed by many threads. Therefore maybe we should consider directly using ConcurrentLinkedQueue which utilizes an efficient non-block algorithm. In checkCalls , do you think we can avoid the poll+offer operations for a not-done-yet call?
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > In RetryInfo#newRetryInfo, looks like failover, fail, and retry are mutual exclusive? ...

          It is correct that the end result is mutually exclusive. However, we need to loop all of the actions in order to determine which one to keep. Indeed, we may combine the failover and fail in RetryInfo to a single action. Let me change it.

          > ... consider directly using ConcurrentLinkedQueue which utilizes an efficient non-block algorithm.

          It is a good idea.

          > In checkCalls, do you think we can avoid the poll+offer operations for a not-done-yet call?

          I think it is hard to avoid. We don't want to read the non-head elements from the queue since it is an O( n) operation, where n is the size of the queue. Poll and offer indeed are cheap for linked queue. Let me know if you have an idea to avoid poll+offer.

          Here is a new patch:

          c13227_20160606.patch

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > In RetryInfo#newRetryInfo, looks like failover, fail, and retry are mutual exclusive? ... It is correct that the end result is mutually exclusive. However, we need to loop all of the actions in order to determine which one to keep. Indeed, we may combine the failover and fail in RetryInfo to a single action. Let me change it. > ... consider directly using ConcurrentLinkedQueue which utilizes an efficient non-block algorithm. It is a good idea. > In checkCalls, do you think we can avoid the poll+offer operations for a not-done-yet call? I think it is hard to avoid. We don't want to read the non-head elements from the queue since it is an O( n) operation, where n is the size of the queue. Poll and offer indeed are cheap for linked queue. Let me know if you have an idea to avoid poll+offer. Here is a new patch: c13227_20160606.patch
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks a lot for updating the patch, Nicholas!

          We don't want to read the non-head elements from the queue since it is an O( n) operation, where n is the size of the queue.

          Can we use ConcurrentLinkedQueue's iterator for the scanning here? We will not get any ConcurrentModificationException, also only the monitor thread removes elements from the queue, all other threads only add elements into the queue.

          Show
          jingzhao Jing Zhao added a comment - Thanks a lot for updating the patch, Nicholas! We don't want to read the non-head elements from the queue since it is an O( n) operation, where n is the size of the queue. Can we use ConcurrentLinkedQueue's iterator for the scanning here? We will not get any ConcurrentModificationException , also only the monitor thread removes elements from the queue, all other threads only add elements into the queue.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 29s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 36s Maven dependency ordering for branch
          +1 mvninstall 7m 52s trunk passed
          +1 compile 7m 51s trunk passed
          +1 checkstyle 1m 22s trunk passed
          +1 mvnsite 1m 51s trunk passed
          +1 mvneclipse 0m 23s trunk passed
          +1 findbugs 3m 4s trunk passed
          +1 javadoc 2m 7s trunk passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 28s the patch passed
          +1 compile 6m 44s the patch passed
          +1 javac 6m 44s the patch passed
          -1 checkstyle 1m 24s root: The patch generated 5 new + 213 unchanged - 1 fixed = 218 total (was 214)
          +1 mvnsite 1m 46s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 31s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 2m 0s the patch passed
          -1 unit 16m 58s hadoop-common in the patch failed.
          -1 unit 70m 36s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          131m 54s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce() calls Thread.sleep() with a lock held At RetryInvocationHandler.java:lock held At RetryInvocationHandler.java:[line 107]
          Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
          Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808462/c13227_20160606.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 258f00ed56a8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 35f255b
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 36s Maven dependency ordering for branch +1 mvninstall 7m 52s trunk passed +1 compile 7m 51s trunk passed +1 checkstyle 1m 22s trunk passed +1 mvnsite 1m 51s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 3m 4s trunk passed +1 javadoc 2m 7s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 28s the patch passed +1 compile 6m 44s the patch passed +1 javac 6m 44s the patch passed -1 checkstyle 1m 24s root: The patch generated 5 new + 213 unchanged - 1 fixed = 218 total (was 214) +1 mvnsite 1m 46s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 31s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 2m 0s the patch passed -1 unit 16m 58s hadoop-common in the patch failed. -1 unit 70m 36s hadoop-hdfs in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 131m 54s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce() calls Thread.sleep() with a lock held At RetryInvocationHandler.java:lock held At RetryInvocationHandler.java: [line 107] Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808462/c13227_20160606.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 258f00ed56a8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 35f255b Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9667/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > Can we use ConcurrentLinkedQueue's iterator for the scanning here? We will not get any ConcurrentModificationException ...

          Honestly, I didn't know that it won't throw ConcurrentModificationException in this case. I learned something here. Thanks a lot! Here is a new patch.

          c13227_20160607.patch

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > Can we use ConcurrentLinkedQueue's iterator for the scanning here? We will not get any ConcurrentModificationException ... Honestly, I didn't know that it won't throw ConcurrentModificationException in this case. I learned something here. Thanks a lot! Here is a new patch. c13227_20160607.patch
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 31s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 36s Maven dependency ordering for branch
          +1 mvninstall 6m 41s trunk passed
          +1 compile 6m 13s trunk passed
          +1 checkstyle 1m 20s trunk passed
          +1 mvnsite 1m 45s trunk passed
          +1 mvneclipse 0m 22s trunk passed
          +1 findbugs 2m 55s trunk passed
          +1 javadoc 1m 58s trunk passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 30s the patch passed
          +1 compile 7m 45s the patch passed
          +1 javac 7m 45s the patch passed
          -1 checkstyle 1m 29s root: The patch generated 1 new + 213 unchanged - 1 fixed = 214 total (was 214)
          +1 mvnsite 1m 46s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 59s the patch passed
          +1 javadoc 2m 14s the patch passed
          -1 unit 7m 55s hadoop-common in the patch failed.
          -1 unit 77m 28s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          128m 23s



          Reason Tests
          Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808526/c13227_20160607.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux eae3da16e471 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6de9213
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 36s Maven dependency ordering for branch +1 mvninstall 6m 41s trunk passed +1 compile 6m 13s trunk passed +1 checkstyle 1m 20s trunk passed +1 mvnsite 1m 45s trunk passed +1 mvneclipse 0m 22s trunk passed +1 findbugs 2m 55s trunk passed +1 javadoc 1m 58s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 7m 45s the patch passed +1 javac 7m 45s the patch passed -1 checkstyle 1m 29s root: The patch generated 1 new + 213 unchanged - 1 fixed = 214 total (was 214) +1 mvnsite 1m 46s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 59s the patch passed +1 javadoc 2m 14s the patch passed -1 unit 7m 55s hadoop-common in the patch failed. -1 unit 77m 28s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 128m 23s Reason Tests Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808526/c13227_20160607.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux eae3da16e471 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6de9213 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9673/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          The latest patch looks good to me. The only minor comment:

                final RetryAction a = failover != null? failover
                    : retry == null? fail: null;
          

          Here a can be assigned with retry itself if retry is not failover/null? Other than this +1.

          Show
          jingzhao Jing Zhao added a comment - The latest patch looks good to me. The only minor comment: final RetryAction a = failover != null ? failover : retry == null ? fail: null ; Here a can be assigned with retry itself if retry is not failover/null? Other than this +1.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          c13227_20160608.patch:

          • uses RetryDecision ordering to further simplify newRetryInfo.
          • checkCalls should check if the queue is empty at the end.
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - c13227_20160608.patch: uses RetryDecision ordering to further simplify newRetryInfo. checkCalls should check if the queue is empty at the end.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          c13227_20160608b.patch: checkEmpty() should be right after remove()

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - c13227_20160608b.patch: checkEmpty() should be right after remove()
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 28s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 6m 28s trunk passed
          +1 compile 6m 41s trunk passed
          +1 checkstyle 1m 26s trunk passed
          +1 mvnsite 1m 50s trunk passed
          +1 mvneclipse 0m 22s trunk passed
          +1 findbugs 3m 8s trunk passed
          +1 javadoc 2m 16s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 2m 4s the patch passed
          +1 compile 9m 10s the patch passed
          +1 javac 9m 10s the patch passed
          -1 checkstyle 1m 45s root: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220)
          +1 mvnsite 2m 16s the patch passed
          +1 mvneclipse 0m 26s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 4m 3s the patch passed
          +1 javadoc 2m 29s the patch passed
          -1 unit 9m 17s hadoop-common in the patch failed.
          -1 unit 71m 17s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          127m 13s



          Reason Tests
          Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.ha.TestZKFailoverController
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestDataNodeUUID



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808819/c13227_20160608.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux eb80c47f0174 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 76f0800
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 6m 28s trunk passed +1 compile 6m 41s trunk passed +1 checkstyle 1m 26s trunk passed +1 mvnsite 1m 50s trunk passed +1 mvneclipse 0m 22s trunk passed +1 findbugs 3m 8s trunk passed +1 javadoc 2m 16s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 4s the patch passed +1 compile 9m 10s the patch passed +1 javac 9m 10s the patch passed -1 checkstyle 1m 45s root: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220) +1 mvnsite 2m 16s the patch passed +1 mvneclipse 0m 26s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 3s the patch passed +1 javadoc 2m 29s the patch passed -1 unit 9m 17s hadoop-common in the patch failed. -1 unit 71m 17s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 127m 13s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.ha.TestZKFailoverController   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeUUID Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808819/c13227_20160608.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux eb80c47f0174 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 76f0800 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9683/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 32s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 31s Maven dependency ordering for branch
          +1 mvninstall 8m 40s trunk passed
          +1 compile 8m 57s trunk passed
          +1 checkstyle 1m 45s trunk passed
          +1 mvnsite 2m 10s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 3m 42s trunk passed
          +1 javadoc 2m 10s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 2m 3s the patch passed
          +1 compile 8m 51s the patch passed
          +1 javac 8m 51s the patch passed
          -1 checkstyle 1m 38s root: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220)
          +1 mvnsite 2m 13s the patch passed
          +1 mvneclipse 0m 26s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 4m 20s the patch passed
          +1 javadoc 2m 19s the patch passed
          +1 unit 9m 47s hadoop-common in the patch passed.
          -1 unit 65m 59s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          128m 3s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.balancer.TestBalancer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808822/c13227_20160608b.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7c9f44517612 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 76f0800
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 32s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 8m 40s trunk passed +1 compile 8m 57s trunk passed +1 checkstyle 1m 45s trunk passed +1 mvnsite 2m 10s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 42s trunk passed +1 javadoc 2m 10s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 3s the patch passed +1 compile 8m 51s the patch passed +1 javac 8m 51s the patch passed -1 checkstyle 1m 38s root: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220) +1 mvnsite 2m 13s the patch passed +1 mvneclipse 0m 26s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 20s the patch passed +1 javadoc 2m 19s the patch passed +1 unit 9m 47s hadoop-common in the patch passed. -1 unit 65m 59s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 128m 3s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.balancer.TestBalancer Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808822/c13227_20160608b.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7c9f44517612 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 76f0800 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9684/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          +1 on the latest patch.

          Show
          jingzhao Jing Zhao added a comment - +1 on the latest patch.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          c13227_20160616.patch: sync'ed with trunk.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - c13227_20160616.patch: sync'ed with trunk.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 30s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 8m 24s trunk passed
          +1 compile 9m 17s trunk passed
          +1 checkstyle 1m 34s trunk passed
          +1 mvnsite 2m 18s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 3m 53s trunk passed
          +1 javadoc 2m 16s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          -1 mvninstall 0m 25s hadoop-common in the patch failed.
          -1 mvninstall 0m 48s hadoop-hdfs in the patch failed.
          -1 compile 0m 44s root in the patch failed.
          -1 javac 0m 44s root in the patch failed.
          -1 checkstyle 1m 36s root: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220)
          -1 mvnsite 0m 34s hadoop-common in the patch failed.
          -1 mvnsite 0m 56s hadoop-hdfs in the patch failed.
          +1 mvneclipse 0m 26s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 20s hadoop-common in the patch failed.
          -1 findbugs 0m 27s hadoop-hdfs in the patch failed.
          +1 javadoc 2m 8s the patch passed
          -1 unit 0m 29s hadoop-common in the patch failed.
          -1 unit 0m 47s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          39m 51s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811015/c13227_20160616.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 09a267fbeb5b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2ca7344
          Default Java 1.8.0_91
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-compile-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 8m 24s trunk passed +1 compile 9m 17s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 2m 18s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 53s trunk passed +1 javadoc 2m 16s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch -1 mvninstall 0m 25s hadoop-common in the patch failed. -1 mvninstall 0m 48s hadoop-hdfs in the patch failed. -1 compile 0m 44s root in the patch failed. -1 javac 0m 44s root in the patch failed. -1 checkstyle 1m 36s root: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220) -1 mvnsite 0m 34s hadoop-common in the patch failed. -1 mvnsite 0m 56s hadoop-hdfs in the patch failed. +1 mvneclipse 0m 26s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 20s hadoop-common in the patch failed. -1 findbugs 0m 27s hadoop-hdfs in the patch failed. +1 javadoc 2m 8s the patch passed -1 unit 0m 29s hadoop-common in the patch failed. -1 unit 0m 47s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 39m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811015/c13227_20160616.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 09a267fbeb5b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2ca7344 Default Java 1.8.0_91 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-compile-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9795/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          c13227_20160621.patch: sync'ed with trunk.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - c13227_20160621.patch: sync'ed with trunk.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 30s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 7m 11s trunk passed
          +1 compile 7m 23s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 24s trunk passed
          +1 javadoc 0m 45s trunk passed
          -1 mvninstall 0m 22s hadoop-common in the patch failed.
          -1 compile 0m 42s root in the patch failed.
          -1 javac 0m 41s root in the patch failed.
          -0 checkstyle 0m 24s hadoop-common-project/hadoop-common: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220)
          -1 mvnsite 0m 23s hadoop-common in the patch failed.
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 15s hadoop-common in the patch failed.
          +1 javadoc 0m 42s the patch passed
          -1 unit 0m 22s hadoop-common in the patch failed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          22m 58s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812516/c13227_20160621.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8070cfd79004 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4ee3543
          Default Java 1.8.0_91
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-compile-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 7m 11s trunk passed +1 compile 7m 23s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 24s trunk passed +1 javadoc 0m 45s trunk passed -1 mvninstall 0m 22s hadoop-common in the patch failed. -1 compile 0m 42s root in the patch failed. -1 javac 0m 41s root in the patch failed. -0 checkstyle 0m 24s hadoop-common-project/hadoop-common: The patch generated 1 new + 219 unchanged - 1 fixed = 220 total (was 220) -1 mvnsite 0m 23s hadoop-common in the patch failed. +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 15s hadoop-common in the patch failed. +1 javadoc 0m 42s the patch passed -1 unit 0m 22s hadoop-common in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 22m 58s Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812516/c13227_20160621.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8070cfd79004 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4ee3543 Default Java 1.8.0_91 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-compile-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9853/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          c13227_20160624.patch: fixes a compilation problem.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - c13227_20160624.patch: fixes a compilation problem.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 6m 23s trunk passed
          +1 compile 6m 46s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 22s trunk passed
          +1 javadoc 0m 46s trunk passed
          +1 mvninstall 0m 40s the patch passed
          +1 compile 6m 45s the patch passed
          +1 javac 6m 45s the patch passed
          -0 checkstyle 0m 26s hadoop-common-project/hadoop-common: The patch generated 2 new + 220 unchanged - 1 fixed = 222 total (was 221)
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 26s the patch passed
          +1 javadoc 0m 46s the patch passed
          +1 unit 7m 32s hadoop-common in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          37m 36s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812981/c13227_20160624.patch
          JIRA Issue HADOOP-13227
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e4cbc0ea752c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0b9edf6
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9867/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9867/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9867/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 6m 23s trunk passed +1 compile 6m 46s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 22s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 40s the patch passed +1 compile 6m 45s the patch passed +1 javac 6m 45s the patch passed -0 checkstyle 0m 26s hadoop-common-project/hadoop-common: The patch generated 2 new + 220 unchanged - 1 fixed = 222 total (was 221) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 26s the patch passed +1 javadoc 0m 46s the patch passed +1 unit 7m 32s hadoop-common in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 37m 36s Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812981/c13227_20160624.patch JIRA Issue HADOOP-13227 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e4cbc0ea752c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0b9edf6 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9867/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9867/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9867/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          +1 on the latest patch.

          Show
          jingzhao Jing Zhao added a comment - +1 on the latest patch.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Thanks Jing for reviewing the patches.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Thanks Jing for reviewing the patches.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          I have committed this.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - I have committed this.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10019 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10019/)
          HADOOP-13227. AsyncCallHandler should use an event driven architecture (szetszwo: rev d328e667067743f723e332d92154da8e84e65742)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryPolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryInvocationHandler.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestAsyncIPC.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/retry/TestDefaultRetryPolicy.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/CallReturn.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/AsyncCallHandler.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10019 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10019/ ) HADOOP-13227 . AsyncCallHandler should use an event driven architecture (szetszwo: rev d328e667067743f723e332d92154da8e84e65742) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryPolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryInvocationHandler.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestAsyncIPC.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/retry/TestDefaultRetryPolicy.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/CallReturn.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/AsyncCallHandler.java
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user steveloughran opened a pull request:

          https://github.com/apache/hadoop/pull/163

          HADOOP-13227 outputstream specification

          WiP specification of output streams.

          Alongside the docs, this will include tests and tightening of behaviours of output streams which don't comply with the `java.io.OutputStream` specification or other parts of the specification (i.e. the extra methods are inconsistent with HDFS). Object stores are special; their behaviours will be documented and (unreconcilable) differences with filesystems handled in tests.

          Note that HDFS does not currently follow `java.io.OutputStream.write(int)`'s required behaviour: it will not fail on a write on a closed stream. This will have to be corrected

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/steveloughran/hadoop s3/HADOOP-13327-outputstream-spec

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/163.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #163


          commit b7ab6c6f2921eaa75f04e35133e56a58d56cf636
          Author: Steve Loughran <stevel@apache.org>
          Date: 2016-06-28T20:53:56Z

          HADOOP-13227 the initial output stream docs

          commit e07af211f2dbd85cd97d8a78c4f517bb4350d956
          Author: Steve Loughran <stevel@apache.org>
          Date: 2016-11-18T12:30:50Z

          HADOOP-13327 adding open/closed state to output stream, java.io.OutputStream defining actions when output streams are closed.

          Change-Id: Ia921e8d2f9af53e8e6655a806e79fcc1dba49d1d


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user steveloughran opened a pull request: https://github.com/apache/hadoop/pull/163 HADOOP-13227 outputstream specification WiP specification of output streams. Alongside the docs, this will include tests and tightening of behaviours of output streams which don't comply with the `java.io.OutputStream` specification or other parts of the specification (i.e. the extra methods are inconsistent with HDFS). Object stores are special; their behaviours will be documented and (unreconcilable) differences with filesystems handled in tests. Note that HDFS does not currently follow `java.io.OutputStream.write(int)`'s required behaviour: it will not fail on a write on a closed stream. This will have to be corrected You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveloughran/hadoop s3/ HADOOP-13327 -outputstream-spec Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/163.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #163 commit b7ab6c6f2921eaa75f04e35133e56a58d56cf636 Author: Steve Loughran <stevel@apache.org> Date: 2016-06-28T20:53:56Z HADOOP-13227 the initial output stream docs commit e07af211f2dbd85cd97d8a78c4f517bb4350d956 Author: Steve Loughran <stevel@apache.org> Date: 2016-11-18T12:30:50Z HADOOP-13327 adding open/closed state to output stream, java.io.OutputStream defining actions when output streams are closed. Change-Id: Ia921e8d2f9af53e8e6655a806e79fcc1dba49d1d

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development