Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: fs/s3, test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Some of the slow test cases contain tests that are now obsoleted by newer ones. For example, ITestS3ADeleteManyFiles has the test case testOpenCreate() which writes then reads files up 25 MB.

      Have a look at which of the s3a tests are taking time, review them to see if newer tests have superceded the slow ones; and cut them where appropriate.

      1. testrun.txt
        38 kB
        Steve Loughran
      2. HADOOP-13614-branch-2-008.patch
        52 kB
        Steve Loughran
      3. HADOOP-13614-branch-2-007.patch
        53 kB
        Steve Loughran
      4. HADOOP-13614-branch-2-006.patch
        38 kB
        Steve Loughran
      5. HADOOP-13614-branch-2-005.patch
        38 kB
        Chris Nauroth
      6. HADOOP-13614-branch-2-004.patch
        38 kB
        Steve Loughran
      7. HADOOP-13614-branch-2-002.patch
        9 kB
        Steve Loughran
      8. HADOOP-13614-branch-2-002.patch
        9 kB
        Steve Loughran
      9. HADOOP-13614-branch-2-001.patch
        2 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 001

          • cut 25 MB of upload test
          • shrink some of the encryption uploads. they were there primarily to see if claims that the stat of a file != the length on read if the file was encrypted. We haven't been able to replicate that; cutting down the file sizes will continue to check possible mismatch, but faster

          testing: S3 ireland

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 001 cut 25 MB of upload test shrink some of the encryption uploads. they were there primarily to see if claims that the stat of a file != the length on read if the file was encrypted. We haven't been able to replicate that; cutting down the file sizes will continue to check possible mismatch, but faster testing: S3 ireland
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 6m 26s branch-2 passed
          +1 compile 0m 15s branch-2 passed with JDK v1.8.0_101
          +1 compile 0m 18s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 0m 15s branch-2 passed
          +1 mvnsite 0m 24s branch-2 passed
          +1 mvneclipse 0m 14s branch-2 passed
          +1 findbugs 0m 32s branch-2 passed
          +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 16s branch-2 passed with JDK v1.7.0_111
          -1 mvninstall 0m 16s hadoop-aws in the patch failed.
          -1 compile 0m 13s hadoop-aws in the patch failed with JDK v1.8.0_101.
          -1 javac 0m 13s hadoop-aws in the patch failed with JDK v1.8.0_101.
          -1 compile 0m 16s hadoop-aws in the patch failed with JDK v1.7.0_111.
          -1 javac 0m 16s hadoop-aws in the patch failed with JDK v1.7.0_111.
          -0 checkstyle 0m 12s hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          -1 mvnsite 0m 17s hadoop-aws in the patch failed.
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 12s hadoop-aws in the patch failed.
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 14s the patch passed with JDK v1.7.0_111
          -1 unit 0m 16s hadoop-aws in the patch failed with JDK v1.7.0_111.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          13m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13614
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828602/HADOOP-13614-branch-2-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 693903be4057 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 853e1e6
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.8.0_101.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/testReport/
          modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 26s branch-2 passed +1 compile 0m 15s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 18s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 15s branch-2 passed +1 mvnsite 0m 24s branch-2 passed +1 mvneclipse 0m 14s branch-2 passed +1 findbugs 0m 32s branch-2 passed +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 16s branch-2 passed with JDK v1.7.0_111 -1 mvninstall 0m 16s hadoop-aws in the patch failed. -1 compile 0m 13s hadoop-aws in the patch failed with JDK v1.8.0_101. -1 javac 0m 13s hadoop-aws in the patch failed with JDK v1.8.0_101. -1 compile 0m 16s hadoop-aws in the patch failed with JDK v1.7.0_111. -1 javac 0m 16s hadoop-aws in the patch failed with JDK v1.7.0_111. -0 checkstyle 0m 12s hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) -1 mvnsite 0m 17s hadoop-aws in the patch failed. +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 12s hadoop-aws in the patch failed. +1 javadoc 0m 10s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_111 -1 unit 0m 16s hadoop-aws in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 13m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13614 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828602/HADOOP-13614-branch-2-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 693903be4057 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 853e1e6 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.8.0_101.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/testReport/ modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10517/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          The intent of this patch looks good to me. I expect it will be ready to go after updating ITestS3ADeleteFilesOneByOne to address the compilation error.

          Show
          cnauroth Chris Nauroth added a comment - The intent of this patch looks good to me. I expect it will be ready to go after updating ITestS3ADeleteFilesOneByOne to address the compilation error.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 002

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 002
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'm attaching a tet run against s3 ireland.

          one thing to note here is that lots of tests appear to be running twice and that as a result the test run is taking 51 min, when it used to take 15 minutes a few weeks back.

          This patch isn't the cause; I think that something is up with the POM, and that as more large scale tests have gone in, that problem has amplified. This test run is just an example of the problem I'm seeing, isolated from any other work

          Show
          stevel@apache.org Steve Loughran added a comment - I'm attaching a tet run against s3 ireland. one thing to note here is that lots of tests appear to be running twice and that as a result the test run is taking 51 min, when it used to take 15 minutes a few weeks back. This patch isn't the cause; I think that something is up with the POM, and that as more large scale tests have gone in, that problem has amplified. This test run is just an example of the problem I'm seeing, isolated from any other work
          Hide
          stevel@apache.org Steve Loughran added a comment -

          reattaching patch 002 so that it is the last file in the attachment list, then asking yetus to try it again

          Show
          stevel@apache.org Steve Loughran added a comment - reattaching patch 002 so that it is the last file in the attachment list, then asking yetus to try it again
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 3m 32s Docker failed to build yetus/hadoop:b59b8b7.



          Subsystem Report/Notes
          JIRA Issue HADOOP-13614
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829827/HADOOP-13614-branch-2-002.patch
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10569/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 3m 32s Docker failed to build yetus/hadoop:b59b8b7. Subsystem Report/Notes JIRA Issue HADOOP-13614 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829827/HADOOP-13614-branch-2-002.patch Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10569/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Patch 002 looks good to me. Something failed in the Docker build during the last pre-commit, so I submitted a new pre-commit run. That will show up here once a Jenkins host frees up:

          https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/

          one thing to note here is that lots of tests appear to be running twice...

          Thanks for sharing the test output. I see this is the command you ran:

          mvn -T 1C integration-test  -Pparallel-tests -DtestsThreadCount=4 
          

          There is an unfortunate Maven limitation between failsafe, use of automatically triggered profiles (e.g. tests-off vs. tests-on) and manual activation of profiles with the -P option. The only workaround I'm aware of is to switch to activation by presence of a property and avoid activation via -P. This is why during the integration test split, I added support for enabling parallel tests by -Dparallel-tests and updated documentation to change all occurrences of -Pparallel-tests to -Dparallel-tests.

          Show
          cnauroth Chris Nauroth added a comment - Patch 002 looks good to me. Something failed in the Docker build during the last pre-commit, so I submitted a new pre-commit run. That will show up here once a Jenkins host frees up: https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/ one thing to note here is that lots of tests appear to be running twice... Thanks for sharing the test output. I see this is the command you ran: mvn -T 1C integration-test -Pparallel-tests -DtestsThreadCount=4 There is an unfortunate Maven limitation between failsafe, use of automatically triggered profiles (e.g. tests-off vs. tests-on ) and manual activation of profiles with the -P option. The only workaround I'm aware of is to switch to activation by presence of a property and avoid activation via -P . This is why during the integration test split, I added support for enabling parallel tests by -Dparallel-tests and updated documentation to change all occurrences of -Pparallel-tests to -Dparallel-tests .
          Hide
          stevel@apache.org Steve Loughran added a comment -

          oh, ok. For the scale stuff changes in HADOOP-13560 I've had the profile -Pscale set a maven property, but there's special code in the test utils to recognise "unset" as an explicitly unset maven property (hence: use the config value, falling back to the default)

          Show
          stevel@apache.org Steve Loughran added a comment - oh, ok. For the scale stuff changes in HADOOP-13560 I've had the profile -Pscale set a maven property, but there's special code in the test utils to recognise "unset" as an explicitly unset maven property (hence: use the config value, falling back to the default)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 15m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 4m 16s Maven dependency ordering for branch
          +1 mvninstall 7m 22s branch-2 passed
          +1 compile 7m 0s branch-2 passed with JDK v1.8.0_101
          +1 compile 7m 27s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 37s branch-2 passed
          +1 mvnsite 1m 32s branch-2 passed
          +1 mvneclipse 1m 26s branch-2 passed
          +1 findbugs 2m 20s branch-2 passed
          +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 16s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 5m 53s the patch passed with JDK v1.8.0_101
          +1 javac 5m 53s the patch passed
          +1 compile 6m 44s the patch passed with JDK v1.7.0_111
          +1 javac 6m 44s the patch passed
          -0 checkstyle 1m 28s root: The patch generated 12 new + 23 unchanged - 0 fixed = 35 total (was 23)
          +1 mvnsite 1m 30s the patch passed
          +1 mvneclipse 0m 37s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 findbugs 2m 49s the patch passed
          +1 javadoc 1m 10s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 25s the patch passed with JDK v1.7.0_111
          +1 unit 8m 16s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          115m 48s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13614
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829827/HADOOP-13614-branch-2-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux baa538d4851c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 7484d0b
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 4m 16s Maven dependency ordering for branch +1 mvninstall 7m 22s branch-2 passed +1 compile 7m 0s branch-2 passed with JDK v1.8.0_101 +1 compile 7m 27s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 37s branch-2 passed +1 mvnsite 1m 32s branch-2 passed +1 mvneclipse 1m 26s branch-2 passed +1 findbugs 2m 20s branch-2 passed +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 16s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 5m 53s the patch passed with JDK v1.8.0_101 +1 javac 5m 53s the patch passed +1 compile 6m 44s the patch passed with JDK v1.7.0_111 +1 javac 6m 44s the patch passed -0 checkstyle 1m 28s root: The patch generated 12 new + 23 unchanged - 0 fixed = 35 total (was 23) +1 mvnsite 1m 30s the patch passed +1 mvneclipse 0m 37s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 2m 49s the patch passed +1 javadoc 1m 10s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 25s the patch passed with JDK v1.7.0_111 +1 unit 8m 16s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 115m 48s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13614 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829827/HADOOP-13614-branch-2-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux baa538d4851c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 7484d0b Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10596/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          The Checkstyle warnings are unused imports that can be cleaned up. After that's done, I think this will be ready to commit. The TestDNS failure is unrelated and tracked elsewhere.

          Show
          cnauroth Chris Nauroth added a comment - The Checkstyle warnings are unused imports that can be cleaned up. After that's done, I think this will be ready to commit. The TestDNS failure is unrelated and tracked elsewhere.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user steveloughran opened a pull request:

          https://github.com/apache/hadoop/pull/132

          HADOOP-13614 superfluous tests

          • where possible give all the s3a tests the superclass of AbstractS3ATestBase, clean up children.
          • ITestS3AFileSystemContract has path logic reworked so test can run in parallel
          • fixed up yarn test to work in parallel too, along with BlockingThreadPool and FastOutputStream
          • on the files changed, moved the copyright comment to being a simple comment rather than a javadoc one.

          This conflicts a bit with HADOOP-13560; this patch should go in first and then I'll address the diffs

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/steveloughran/hadoop s3/HADOOP-13614-superfluous-tests

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/132.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #132


          commit 0c62c670549a18c5cfdb44d940f36a98bf6a1168
          Author: Steve Loughran <stevel@apache.org>
          Date: 2016-09-15T08:37:05Z

          HADOOP-13614: cut some needless, slower S3A tests

          commit 3a4b5640814f0d0dc76461fa502cd354eef19bce
          Author: Steve Loughran <stevel@apache.org>
          Date: 2016-09-16T13:40:58Z

          HADOOP-13614; fix tests to compile, review all tests, clean slightly and move timing of upload/download times and bandwidth to contract tests

          commit 68b03bef79d25d71549da9ca72ebcdf7e1434938
          Author: Steve Loughran <stevel@apache.org>
          Date: 2016-09-30T17:28:14Z

          HADOOP-13614: patch 002.
          -where possible give all the s3a tests the superclass of AbstractS3ATestBase, clean up children.
          -ITestS3AFileSystemContract has path logic reworked so test can run in parallel
          -fixed up yarn test to work in parallel too, along with BlockingThreadPool and FastOutputStream
          -on the files changed, moved the copyright comment to being a simple comment rather than a javadoc one.

          This conflicts a bit with HADOOP-13560; this patch should go in first and then I'll address the diffs


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user steveloughran opened a pull request: https://github.com/apache/hadoop/pull/132 HADOOP-13614 superfluous tests where possible give all the s3a tests the superclass of AbstractS3ATestBase , clean up children. ITestS3AFileSystemContract has path logic reworked so test can run in parallel fixed up yarn test to work in parallel too, along with BlockingThreadPool and FastOutputStream on the files changed, moved the copyright comment to being a simple comment rather than a javadoc one. This conflicts a bit with HADOOP-13560 ; this patch should go in first and then I'll address the diffs You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveloughran/hadoop s3/ HADOOP-13614 -superfluous-tests Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/132.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #132 commit 0c62c670549a18c5cfdb44d940f36a98bf6a1168 Author: Steve Loughran <stevel@apache.org> Date: 2016-09-15T08:37:05Z HADOOP-13614 : cut some needless, slower S3A tests commit 3a4b5640814f0d0dc76461fa502cd354eef19bce Author: Steve Loughran <stevel@apache.org> Date: 2016-09-16T13:40:58Z HADOOP-13614 ; fix tests to compile, review all tests, clean slightly and move timing of upload/download times and bandwidth to contract tests commit 68b03bef79d25d71549da9ca72ebcdf7e1434938 Author: Steve Loughran <stevel@apache.org> Date: 2016-09-30T17:28:14Z HADOOP-13614 : patch 002. -where possible give all the s3a tests the superclass of AbstractS3ATestBase , clean up children. -ITestS3AFileSystemContract has path logic reworked so test can run in parallel -fixed up yarn test to work in parallel too, along with BlockingThreadPool and FastOutputStream -on the files changed, moved the copyright comment to being a simple comment rather than a javadoc one. This conflicts a bit with HADOOP-13560 ; this patch should go in first and then I'll address the diffs
          Hide
          stevel@apache.org Steve Loughran added a comment -

          latest patch a github PR. This does more as I decided I could make this the patch to fix up some of the test niggles that had been irritating us all, namely: different parent classes for tests, still too many not running in parallel.

          • where possible give all the s3a tests the superclass of AbstractS3ATestBase, clean up children.
          • ITestS3AFileSystemContract has path logic reworked so test can run in parallel
          • fixed up yarn test to work in parallel too, along with BlockingThreadPool and FastOutputStream
          • on the files changed, moved the copyright comment to being a simple comment rather than a javadoc one.

          This conflicts a bit with HADOOP-13560; this patch should go in first and then I'll address the diffs

          Tested: s3 ireland; if I skipthe s3n and s3 tests, it takes ~4:20 to do a test run now

          Show
          stevel@apache.org Steve Loughran added a comment - latest patch a github PR. This does more as I decided I could make this the patch to fix up some of the test niggles that had been irritating us all, namely: different parent classes for tests, still too many not running in parallel. where possible give all the s3a tests the superclass of AbstractS3ATestBase , clean up children. ITestS3AFileSystemContract has path logic reworked so test can run in parallel fixed up yarn test to work in parallel too, along with BlockingThreadPool and FastOutputStream on the files changed, moved the copyright comment to being a simple comment rather than a javadoc one. This conflicts a bit with HADOOP-13560 ; this patch should go in first and then I'll address the diffs Tested: s3 ireland; if I skipthe s3n and s3 tests, it takes ~4:20 to do a test run now
          Hide
          cnauroth Chris Nauroth added a comment -

          I haven't reviewed the entire patch yet, but I am consistently seeing test timeout failures from ITestS3ADeleteManyFiles and ITestS3ADeleteFilesOneByOne after applying it. The timeout value changed significantly with this patch.

          Old timeout inherited from S3AScaleTestBase:

            @Rule
            public Timeout testTimeout = new Timeout(30 * 60 * 1000);
          

          New timeout inherited from AbstractFSContractTestBase:

            public static final int DEFAULT_TEST_TIMEOUT = 180 * 1000;
          

          These tests consistently take longer than 3 minutes for me to run, testing against US-west-2. I think these tests will need to override AbstractFSContractTestBase#getTestTimeoutMillis() to restore the old timeout value.

          Show
          cnauroth Chris Nauroth added a comment - I haven't reviewed the entire patch yet, but I am consistently seeing test timeout failures from ITestS3ADeleteManyFiles and ITestS3ADeleteFilesOneByOne after applying it. The timeout value changed significantly with this patch. Old timeout inherited from S3AScaleTestBase : @Rule public Timeout testTimeout = new Timeout(30 * 60 * 1000); New timeout inherited from AbstractFSContractTestBase : public static final int DEFAULT_TEST_TIMEOUT = 180 * 1000; These tests consistently take longer than 3 minutes for me to run, testing against US-west-2. I think these tests will need to override AbstractFSContractTestBase#getTestTimeoutMillis() to restore the old timeout value.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          yes, lets do that. Interesting you've seen failures there and I haven't. I have had distcp fail on a multipart purge, but that is something addressed in HADOOP-13560, so I didn't replicate it.

          For those scale tests I've made the the timeout programmable via a system property that maven passes in —and the timeout is checked before running the big tests. nobody (else) wants to find a 5GB test run failed as maven/junit killed it after an hour

          Show
          stevel@apache.org Steve Loughran added a comment - yes, lets do that. Interesting you've seen failures there and I haven't. I have had distcp fail on a multipart purge, but that is something addressed in HADOOP-13560 , so I didn't replicate it. For those scale tests I've made the the timeout programmable via a system property that maven passes in —and the timeout is checked before running the big tests. nobody (else) wants to find a 5GB test run failed as maven/junit killed it after an hour
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 004; timeouts for all tests fixed to 10 min, scale tests (and distcp contract) set to a scale value of 30 min.

          Tests all worked against s3 ireland.
          Interesting that Chris saw timeouts against US east and I didn't see them against US ireland, makes me wonder if different endpoints have different latencies

          Show
          stevel@apache.org Steve Loughran added a comment - patch 004; timeouts for all tests fixed to 10 min, scale tests (and distcp contract) set to a scale value of 30 min. Tests all worked against s3 ireland. Interesting that Chris saw timeouts against US east and I didn't see them against US ireland, makes me wonder if different endpoints have different latencies
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 32s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 22 new or modified test files.
          0 mvndep 1m 11s Maven dependency ordering for branch
          +1 mvninstall 6m 44s branch-2 passed
          +1 compile 5m 51s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 37s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 25s branch-2 passed
          +1 mvnsite 1m 23s branch-2 passed
          +1 mvneclipse 0m 32s branch-2 passed
          +1 findbugs 2m 26s branch-2 passed
          +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 17s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 7s the patch passed
          +1 compile 5m 52s the patch passed with JDK v1.8.0_101
          -1 javac 5m 52s root-jdk1.8.0_101 with JDK v1.8.0_101 generated 1 new + 857 unchanged - 0 fixed = 858 total (was 857)
          +1 compile 6m 34s the patch passed with JDK v1.7.0_111
          -1 javac 6m 34s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 950 unchanged - 0 fixed = 951 total (was 950)
          -0 checkstyle 1m 27s root: The patch generated 4 new + 39 unchanged - 1 fixed = 43 total (was 40)
          +1 mvnsite 1m 29s the patch passed
          +1 mvneclipse 0m 38s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 2m 49s the patch passed
          +1 javadoc 1m 8s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 23s the patch passed with JDK v1.7.0_111
          +1 unit 8m 49s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          109m 11s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13614
          GITHUB PR https://github.com/apache/hadoop/pull/132
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 239fd7b9a0b2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 560e524
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 32s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 22 new or modified test files. 0 mvndep 1m 11s Maven dependency ordering for branch +1 mvninstall 6m 44s branch-2 passed +1 compile 5m 51s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 37s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 25s branch-2 passed +1 mvnsite 1m 23s branch-2 passed +1 mvneclipse 0m 32s branch-2 passed +1 findbugs 2m 26s branch-2 passed +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 17s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 7s the patch passed +1 compile 5m 52s the patch passed with JDK v1.8.0_101 -1 javac 5m 52s root-jdk1.8.0_101 with JDK v1.8.0_101 generated 1 new + 857 unchanged - 0 fixed = 858 total (was 857) +1 compile 6m 34s the patch passed with JDK v1.7.0_111 -1 javac 6m 34s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 950 unchanged - 0 fixed = 951 total (was 950) -0 checkstyle 1m 27s root: The patch generated 4 new + 39 unchanged - 1 fixed = 43 total (was 40) +1 mvnsite 1m 29s the patch passed +1 mvneclipse 0m 38s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 49s the patch passed +1 javadoc 1m 8s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 23s the patch passed with JDK v1.7.0_111 +1 unit 8m 49s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 109m 11s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13614 GITHUB PR https://github.com/apache/hadoop/pull/132 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 239fd7b9a0b2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 560e524 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10656/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, when I committed my HADOOP-13674 patch, I caused a conflict for your patch here. I rebased it for you, and I'm attaching it as revision 005. The only change since 004 is resolving some conflicts on import statements in ITestS3AFailureHandling. This version of the patch passes tests again US-west-2 for me, and I'm code reviewing in more detail now.

          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , when I committed my HADOOP-13674 patch, I caused a conflict for your patch here. I rebased it for you, and I'm attaching it as revision 005. The only change since 004 is resolving some conflicts on import statements in ITestS3AFailureHandling . This version of the patch passes tests again US-west-2 for me, and I'm code reviewing in more detail now.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 8s HADOOP-13614 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-13614
          GITHUB PR https://github.com/apache/hadoop/pull/132
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10659/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 8s HADOOP-13614 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-13614 GITHUB PR https://github.com/apache/hadoop/pull/132 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10659/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Oh, but pre-commit is going to the GitHub pull request. I guess you'll need to update it there, but that's easily done with the 005 file.

          Show
          cnauroth Chris Nauroth added a comment - Oh, but pre-commit is going to the GitHub pull request. I guess you'll need to update it there, but that's easily done with the 005 file.
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, thank you for the nice work here. It looks good to me. I have no other specific feedback. I would like to take one more review pass over it tomorrow morning (PDT) with a clearer head, but I think it's highly likely that I will approve after getting a clean pre-commit run. (As mentioned earlier, I think the GitHub pull request needs to be updated to match the revision 005 file to trigger pre-commit.)

          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , thank you for the nice work here. It looks good to me. I have no other specific feedback. I would like to take one more review pass over it tomorrow morning (PDT) with a clearer head, but I think it's highly likely that I will approve after getting a clean pre-commit run. (As mentioned earlier, I think the GitHub pull request needs to be updated to match the revision 005 file to trigger pre-commit.)
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch as rebased. Tested, S3a ireland. I was worried that a test had broken (ITestS3AContractCreate.testCreatedFileIsImmediatelyVisible, but that came down to the version of hadoop-common in mvn was one I'd built up in a different window to test your blobstore contract XML changes. False alarm

          Show
          stevel@apache.org Steve Loughran added a comment - patch as rebased. Tested, S3a ireland. I was worried that a test had broken ( ITestS3AContractCreate.testCreatedFileIsImmediatelyVisible , but that came down to the version of hadoop-common in mvn was one I'd built up in a different window to test your blobstore contract XML changes. False alarm
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 006 In sync with branch-2; testing against s3 ireland...apart from root dir deletion playing up, all is well

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 006 In sync with branch-2; testing against s3 ireland...apart from root dir deletion playing up, all is well
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 007

          • apply to branch-2
          • verify that all appears well.
          • Tune more tests, including ITestS3AMiniYarnCluster.

          A big change here is fixing up the scale tests to work as subclasses of the S3AScaleTestBase indirect subclasses of AbstractFSContractTestBase, because that sets up the test timeout rule. Rather than have a field of the same name and hope that its timeout gets picked up, I've tuned how timeouts get set up, so the subclasses do it. All well and good, except those subclasses are being called during the initialization of the base class, that is: before the subclasses are full inited. I don't ever like doing that, though it is working here.

          Tested S3a ireland, two (unrelated) failures

            ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:97->Assert.fail:88 Deletion of child entries failed, still have1
            s3a://hwdev-steve-ireland-new/fork-5
          
            ITestS3AFileContextStatistics>FCStatisticsBaseTest.testStatistics:102->verifyWrittenBytes:54 expected:<512> but was:<0>
          

          Time to run without scale tests, ~4 minutes, going to 6:30 with the scale tests @ default size in.

          Show
          stevel@apache.org Steve Loughran added a comment - patch 007 apply to branch-2 verify that all appears well. Tune more tests, including ITestS3AMiniYarnCluster. A big change here is fixing up the scale tests to work as subclasses of the S3AScaleTestBase indirect subclasses of AbstractFSContractTestBase, because that sets up the test timeout rule. Rather than have a field of the same name and hope that its timeout gets picked up, I've tuned how timeouts get set up, so the subclasses do it. All well and good, except those subclasses are being called during the initialization of the base class, that is: before the subclasses are full inited. I don't ever like doing that, though it is working here. Tested S3a ireland, two (unrelated) failures ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:97->Assert.fail:88 Deletion of child entries failed, still have1 s3a: //hwdev-steve-ireland- new /fork-5 ITestS3AFileContextStatistics>FCStatisticsBaseTest.testStatistics:102->verifyWrittenBytes:54 expected:<512> but was:<0> Time to run without scale tests, ~4 minutes, going to 6:30 with the scale tests @ default size in.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 25 new or modified test files.
          0 mvndep 1m 0s Maven dependency ordering for branch
          +1 mvninstall 7m 15s branch-2 passed
          +1 compile 6m 33s branch-2 passed with JDK v1.8.0_101
          +1 compile 7m 2s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 27s branch-2 passed
          +1 mvnsite 1m 30s branch-2 passed
          +1 mvneclipse 0m 31s branch-2 passed
          +1 findbugs 2m 24s branch-2 passed
          +1 javadoc 1m 7s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 19s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 17s Maven dependency ordering for patch
          -1 mvninstall 0m 18s hadoop-aws in the patch failed.
          -1 compile 6m 0s root in the patch failed with JDK v1.8.0_101.
          -1 javac 6m 0s root in the patch failed with JDK v1.8.0_101.
          -1 compile 6m 40s root in the patch failed with JDK v1.7.0_111.
          -1 javac 6m 40s root in the patch failed with JDK v1.7.0_111.
          -0 checkstyle 1m 47s root: The patch generated 10 new + 53 unchanged - 1 fixed = 63 total (was 54)
          -1 mvnsite 0m 27s hadoop-aws in the patch failed.
          +1 mvneclipse 0m 42s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          -1 findbugs 0m 23s hadoop-aws in the patch failed.
          +1 javadoc 1m 21s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 34s the patch passed with JDK v1.7.0_111
          +1 unit 9m 30s hadoop-common in the patch passed with JDK v1.7.0_111.
          -1 unit 0m 25s hadoop-aws in the patch failed with JDK v1.7.0_111.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          100m 19s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13614
          GITHUB PR https://github.com/apache/hadoop/pull/132
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux b8e905e5f360 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 5a37668
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 25 new or modified test files. 0 mvndep 1m 0s Maven dependency ordering for branch +1 mvninstall 7m 15s branch-2 passed +1 compile 6m 33s branch-2 passed with JDK v1.8.0_101 +1 compile 7m 2s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 27s branch-2 passed +1 mvnsite 1m 30s branch-2 passed +1 mvneclipse 0m 31s branch-2 passed +1 findbugs 2m 24s branch-2 passed +1 javadoc 1m 7s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 19s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 17s Maven dependency ordering for patch -1 mvninstall 0m 18s hadoop-aws in the patch failed. -1 compile 6m 0s root in the patch failed with JDK v1.8.0_101. -1 javac 6m 0s root in the patch failed with JDK v1.8.0_101. -1 compile 6m 40s root in the patch failed with JDK v1.7.0_111. -1 javac 6m 40s root in the patch failed with JDK v1.7.0_111. -0 checkstyle 1m 47s root: The patch generated 10 new + 53 unchanged - 1 fixed = 63 total (was 54) -1 mvnsite 0m 27s hadoop-aws in the patch failed. +1 mvneclipse 0m 42s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 0m 23s hadoop-aws in the patch failed. +1 javadoc 1m 21s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 34s the patch passed with JDK v1.7.0_111 +1 unit 9m 30s hadoop-common in the patch passed with JDK v1.7.0_111. -1 unit 0m 25s hadoop-aws in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 100m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13614 GITHUB PR https://github.com/apache/hadoop/pull/132 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b8e905e5f360 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 5a37668 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10839/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 008, rebase to branch-2, resub

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 008, rebase to branch-2, resub
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 7s HADOOP-13614 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-13614
          GITHUB PR https://github.com/apache/hadoop/pull/132
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10873/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s HADOOP-13614 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-13614 GITHUB PR https://github.com/apache/hadoop/pull/132 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10873/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          HADOOP-13755 replaces this with a -patch rather than PR submission.

          Show
          stevel@apache.org Steve Loughran added a comment - HADOOP-13755 replaces this with a -patch rather than PR submission.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 25 new or modified test files.
          0 mvndep 0m 58s Maven dependency ordering for branch
          +1 mvninstall 6m 44s branch-2 passed
          +1 compile 5m 57s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 39s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 30s branch-2 passed
          +1 mvnsite 1m 25s branch-2 passed
          +1 mvneclipse 0m 33s branch-2 passed
          +1 findbugs 2m 20s branch-2 passed
          +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 17s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          -1 mvninstall 0m 19s hadoop-aws in the patch failed.
          -1 compile 5m 43s root in the patch failed with JDK v1.8.0_101.
          -1 javac 5m 43s root in the patch failed with JDK v1.8.0_101.
          -1 compile 6m 34s root in the patch failed with JDK v1.7.0_111.
          -1 javac 6m 34s root in the patch failed with JDK v1.7.0_111.
          -0 checkstyle 1m 29s root: The patch generated 10 new + 53 unchanged - 1 fixed = 63 total (was 54)
          -1 mvnsite 0m 23s hadoop-aws in the patch failed.
          +1 mvneclipse 0m 37s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 0s The patch has no ill-formed XML file.
          -1 findbugs 0m 21s hadoop-aws in the patch failed.
          +1 javadoc 1m 9s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 21s the patch passed with JDK v1.7.0_111
          +1 unit 8m 40s hadoop-common in the patch passed with JDK v1.7.0_111.
          -1 unit 0m 25s hadoop-aws in the patch failed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          94m 20s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13614
          GITHUB PR https://github.com/apache/hadoop/pull/132
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 9676c7ace182 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 086577c
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 25 new or modified test files. 0 mvndep 0m 58s Maven dependency ordering for branch +1 mvninstall 6m 44s branch-2 passed +1 compile 5m 57s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 39s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 30s branch-2 passed +1 mvnsite 1m 25s branch-2 passed +1 mvneclipse 0m 33s branch-2 passed +1 findbugs 2m 20s branch-2 passed +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 17s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch -1 mvninstall 0m 19s hadoop-aws in the patch failed. -1 compile 5m 43s root in the patch failed with JDK v1.8.0_101. -1 javac 5m 43s root in the patch failed with JDK v1.8.0_101. -1 compile 6m 34s root in the patch failed with JDK v1.7.0_111. -1 javac 6m 34s root in the patch failed with JDK v1.7.0_111. -0 checkstyle 1m 29s root: The patch generated 10 new + 53 unchanged - 1 fixed = 63 total (was 54) -1 mvnsite 0m 23s hadoop-aws in the patch failed. +1 mvneclipse 0m 37s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 0s The patch has no ill-formed XML file. -1 findbugs 0m 21s hadoop-aws in the patch failed. +1 javadoc 1m 9s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 21s the patch passed with JDK v1.7.0_111 +1 unit 8m 40s hadoop-common in the patch passed with JDK v1.7.0_111. -1 unit 0m 25s hadoop-aws in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 94m 20s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13614 GITHUB PR https://github.com/apache/hadoop/pull/132 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 9676c7ace182 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 086577c Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10875/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          The current pull request seems to build fine for me locally, so I submitted another pre-commit run:

          https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/

          Show
          cnauroth Chris Nauroth added a comment - The current pull request seems to build fine for me locally, so I submitted another pre-commit run: https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 25 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 53s branch-2 passed
          +1 compile 6m 10s branch-2 passed with JDK v1.8.0_101
          +1 compile 7m 3s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 25s branch-2 passed
          +1 mvnsite 1m 26s branch-2 passed
          +1 mvneclipse 0m 32s branch-2 passed
          +1 findbugs 2m 33s branch-2 passed
          +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 20s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 17s Maven dependency ordering for patch
          -1 mvninstall 0m 19s hadoop-aws in the patch failed.
          -1 compile 6m 26s root in the patch failed with JDK v1.8.0_101.
          -1 javac 6m 26s root in the patch failed with JDK v1.8.0_101.
          -1 compile 7m 12s root in the patch failed with JDK v1.7.0_111.
          -1 javac 7m 12s root in the patch failed with JDK v1.7.0_111.
          -0 checkstyle 1m 36s root: The patch generated 10 new + 50 unchanged - 1 fixed = 60 total (was 51)
          -1 mvnsite 0m 25s hadoop-aws in the patch failed.
          +1 mvneclipse 0m 38s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          -1 findbugs 0m 20s hadoop-aws in the patch failed.
          +1 javadoc 1m 12s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 26s the patch passed with JDK v1.7.0_111
          +1 unit 10m 58s hadoop-common in the patch passed with JDK v1.7.0_111.
          -1 unit 0m 24s hadoop-aws in the patch failed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          99m 14s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13614
          GITHUB PR https://github.com/apache/hadoop/pull/132
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 27a4cded9e91 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 05d772e
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 25 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 53s branch-2 passed +1 compile 6m 10s branch-2 passed with JDK v1.8.0_101 +1 compile 7m 3s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 25s branch-2 passed +1 mvnsite 1m 26s branch-2 passed +1 mvneclipse 0m 32s branch-2 passed +1 findbugs 2m 33s branch-2 passed +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 20s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 17s Maven dependency ordering for patch -1 mvninstall 0m 19s hadoop-aws in the patch failed. -1 compile 6m 26s root in the patch failed with JDK v1.8.0_101. -1 javac 6m 26s root in the patch failed with JDK v1.8.0_101. -1 compile 7m 12s root in the patch failed with JDK v1.7.0_111. -1 javac 7m 12s root in the patch failed with JDK v1.7.0_111. -0 checkstyle 1m 36s root: The patch generated 10 new + 50 unchanged - 1 fixed = 60 total (was 51) -1 mvnsite 0m 25s hadoop-aws in the patch failed. +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. -1 findbugs 0m 20s hadoop-aws in the patch failed. +1 javadoc 1m 12s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 26s the patch passed with JDK v1.7.0_111 +1 unit 10m 58s hadoop-common in the patch passed with JDK v1.7.0_111. -1 unit 0m 24s hadoop-aws in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 99m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13614 GITHUB PR https://github.com/apache/hadoop/pull/132 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 27a4cded9e91 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 05d772e Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10893/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          There are compilation failures reported in ITestS3ABlockingThreadPool, but the patch is actually removing that file. I think we've gotten into a confused state due to not having squashed commits in the pull request. To unblock us, I created a fresh new JIRA for temporary pre-commit testing: HADOOP-13758.

          Show
          cnauroth Chris Nauroth added a comment - There are compilation failures reported in ITestS3ABlockingThreadPool , but the patch is actually removing that file. I think we've gotten into a confused state due to not having squashed commits in the pull request. To unblock us, I created a fresh new JIRA for temporary pre-commit testing: HADOOP-13758 .
          Hide
          aw Allen Wittenauer added a comment -

          I've squashed PRs before. There's probably an easier way but in the branch that PR comes from, I do:

          git rebase trunk
          git rebase -i (last commit hash)
          

          then squash indvidual commits in the -i session, leaving the top one alone. The PR should now be a single commit again.

          Show
          aw Allen Wittenauer added a comment - I've squashed PRs before. There's probably an easier way but in the branch that PR comes from, I do: git rebase trunk git rebase -i (last commit hash) then squash indvidual commits in the -i session, leaving the top one alone. The PR should now be a single commit again.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hello Allen Wittenauer. Thanks for the help. Unfortunately in this case, I'm trying to make progress toward committing a patch for Steve while he is offline. I don't own the PR to do the squashin' myself.

          Show
          cnauroth Chris Nauroth added a comment - Hello Allen Wittenauer . Thanks for the help. Unfortunately in this case, I'm trying to make progress toward committing a patch for Steve while he is offline. I don't own the PR to do the squashin' myself.
          Hide
          cnauroth Chris Nauroth added a comment - - edited

          +1

          I have committed this to trunk, branch-2 and branch-2.8 after completing pre-commit on HADOOP-13758. Steve, thank you for the contribution.

          Show
          cnauroth Chris Nauroth added a comment - - edited +1 I have committed this to trunk, branch-2 and branch-2.8 after completing pre-commit on HADOOP-13758 . Steve, thank you for the contribution.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10687 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10687/)
          HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are (cnauroth: rev 9cad3e235026dbe4658705ca85d263d0edf14521)

          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/S3AScaleTestBase.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionAlgorithmPropagation.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryption.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/yarn/ITestS3AMiniYarnCluster.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3AInputStreamPerformance.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/AbstractSTestS3AHugeFiles.java
          • (edit) hadoop-tools/hadoop-aws/pom.xml
          • (delete) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlockingThreadPool.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADirectoryPerformance.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractDistCp.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AConfiguration.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADeleteManyFiles.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFailureHandling.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3AHugeFilesClassicOutput.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/yarn/ITestS3A.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFileOperationCost.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/S3AContract.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlocksize.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ATemporaryCredentials.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestConstants.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADeleteFilesOneByOne.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/AbstractS3ATestBase.java
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFSMainOperationsLocalFileSystem.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFileSystemContract.java
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10687 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10687/ ) HADOOP-13614 . Purge some superfluous/obsolete S3 FS tests that are (cnauroth: rev 9cad3e235026dbe4658705ca85d263d0edf14521) (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/S3AScaleTestBase.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionAlgorithmPropagation.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryption.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/yarn/ITestS3AMiniYarnCluster.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3AInputStreamPerformance.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/AbstractSTestS3AHugeFiles.java (edit) hadoop-tools/hadoop-aws/pom.xml (delete) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlockingThreadPool.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADirectoryPerformance.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractDistCp.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AConfiguration.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADeleteManyFiles.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFailureHandling.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3AHugeFilesClassicOutput.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/yarn/ITestS3A.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFileOperationCost.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/S3AContract.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlocksize.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ATemporaryCredentials.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestConstants.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADeleteFilesOneByOne.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/AbstractS3ATestBase.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFSMainOperationsLocalFileSystem.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFileSystemContract.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user steveloughran closed the pull request at:

          https://github.com/apache/hadoop/pull/132

          Show
          githubbot ASF GitHub Bot added a comment - Github user steveloughran closed the pull request at: https://github.com/apache/hadoop/pull/132

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development