Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      The s3 file system has been removed. The s3a file system should be used instead.

      Description

      UPDATE: This JIRA ticket is to track the effort of cutting the s3:// from trunk branch. Please see the cloned JIRA ticket HADOOP-13239 for deprecating s3:// from branch-2.

      The fact that s3:// was broken in Hadoop 2.7 and nobody noticed until now shows that it's not being used. while invaluable at the time, s3n and especially s3a render it obsolete except for reading existing data.

      I propose

      1. Mark Java source as @deprecated
      2. Warn the first time in a JVM that an S3 instance is created, "deprecated -will be removed in future releases"
      3. in Hadoop trunk we really cut it. Maybe have an attic project (external?) which holds it for anyone who still wants it. Or: retain the code but remove the fs.s3.impl config option, so you have to explicitly add it for use.
      1. HADOOP-12709.000.patch
        119 kB
        Mingliang Liu
      2. HADOOP-12709.001.patch
        146 kB
        Mingliang Liu
      3. HADOOP-12709.002.patch
        147 kB
        Mingliang Liu
      4. HADOOP-12709.003.patch
        168 kB
        Mingliang Liu
      5. HADOOP-12709.004.patch
        172 kB
        Mingliang Liu
      6. HADOOP-12709.005.patch
        172 kB
        Mingliang Liu
      7. HADOOP-12709-006.patch
        179 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          wheat9 Haohui Mai added a comment -

          +1 on removing in trunk.

          Show
          wheat9 Haohui Mai added a comment - +1 on removing in trunk.
          Hide
          liuml07 Mingliang Liu added a comment -

          The v0 patch cuts the s3:// support in trunk branch.

          Specially:

          • The S3Credentials and S3Exception are used by both s3 and s3n. Thus they were moved to s3n package instead of being deleted. The unit test for S3Credentials is thus also renamed.
          • Updated the hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md.

          Did I miss anything?

          Show
          liuml07 Mingliang Liu added a comment - The v0 patch cuts the s3:// support in trunk branch. Specially: The S3Credentials and S3Exception are used by both s3 and s3n. Thus they were moved to s3n package instead of being deleted. The unit test for S3Credentials is thus also renamed. Updated the hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md . Did I miss anything?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          +1 mvninstall 9m 34s trunk passed
          +1 compile 0m 21s trunk passed with JDK v1.8.0_72
          +1 compile 0m 17s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 25s trunk passed
          +1 mvneclipse 0m 25s trunk passed
          +1 findbugs 0m 40s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 20s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 17s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.8.0_72
          +1 javac 2m 10s hadoop-tools_hadoop-aws-jdk1.8.0_72 with JDK v1.8.0_72 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6)
          +1 javac 0m 16s hadoop-aws in the patch passed with JDK v1.8.0_72.
          +1 compile 0m 15s the patch passed with JDK v1.7.0_95
          +1 javac 2m 25s hadoop-tools_hadoop-aws-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6)
          +1 javac 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_95.
          -1 checkstyle 0m 15s hadoop-tools/hadoop-aws: patch generated 1 new + 57 unchanged - 96 fixed = 58 total (was 153)
          +1 mvnsite 0m 21s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 0m 51s the patch passed
          -1 javadoc 1m 31s hadoop-tools_hadoop-aws-jdk1.8.0_72 with JDK v1.8.0_72 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8)
          +1 javadoc 0m 17s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 16s the patch passed with JDK v1.7.0_95
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.8.0_72.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          18m 19s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788510/HADOOP-12709.000.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux a4fd88123c4b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c7ac25e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/artifact/patchprocess/whitespace-eol.txt
          javadoc hadoop-tools_hadoop-aws-jdk1.8.0_72: https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_72.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/testReport/
          modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. +1 mvninstall 9m 34s trunk passed +1 compile 0m 21s trunk passed with JDK v1.8.0_72 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 0m 40s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 20s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 17s the patch passed +1 compile 0m 16s the patch passed with JDK v1.8.0_72 +1 javac 2m 10s hadoop-tools_hadoop-aws-jdk1.8.0_72 with JDK v1.8.0_72 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6) +1 javac 0m 16s hadoop-aws in the patch passed with JDK v1.8.0_72. +1 compile 0m 15s the patch passed with JDK v1.7.0_95 +1 javac 2m 25s hadoop-tools_hadoop-aws-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6) +1 javac 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_95. -1 checkstyle 0m 15s hadoop-tools/hadoop-aws: patch generated 1 new + 57 unchanged - 96 fixed = 58 total (was 153) +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 0m 51s the patch passed -1 javadoc 1m 31s hadoop-tools_hadoop-aws-jdk1.8.0_72 with JDK v1.8.0_72 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8) +1 javadoc 0m 17s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 16s the patch passed with JDK v1.7.0_95 +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.8.0_72. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 18m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788510/HADOOP-12709.000.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux a4fd88123c4b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c7ac25e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/artifact/patchprocess/whitespace-eol.txt javadoc hadoop-tools_hadoop-aws-jdk1.8.0_72: https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_72.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/testReport/ modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8666/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          I'm in favor of the proposal to remove s3 from trunk. I haven't had time to review the patch yet, but I'll put it in my queue.

          Show
          cnauroth Chris Nauroth added a comment - I'm in favor of the proposal to remove s3 from trunk. I haven't had time to review the patch yet, but I'll put it in my queue.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hello Mingliang Liu. Thank you for taking care of this. This looks good overall.

          Did I miss anything?

          The only missing thing I noticed is that we should remove the fs.s3.* properties from core-default.xml.

          Show
          cnauroth Chris Nauroth added a comment - Hello Mingliang Liu . Thank you for taking care of this. This looks good overall. Did I miss anything? The only missing thing I noticed is that we should remove the fs.s3.* properties from core-default.xml.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks for the review, Chris Nauroth. I'll post a patch soon with core-default.xml updated.

          Show
          liuml07 Mingliang Liu added a comment - Thanks for the review, Chris Nauroth . I'll post a patch soon with core-default.xml updated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          +1 mvninstall 6m 41s trunk passed
          +1 compile 0m 12s trunk passed with JDK v1.8.0_91
          +1 compile 0m 14s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 19s trunk passed
          +1 mvneclipse 0m 19s trunk passed
          +1 findbugs 0m 29s trunk passed
          +1 javadoc 0m 13s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 13s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 13s the patch passed
          +1 compile 0m 9s the patch passed with JDK v1.8.0_91
          +1 javac 0m 9s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6)
          +1 compile 0m 11s the patch passed with JDK v1.7.0_95
          +1 javac 0m 11s hadoop-tools_hadoop-aws-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6)
          -1 checkstyle 0m 12s hadoop-tools/hadoop-aws: The patch generated 1 new + 58 unchanged - 128 fixed = 59 total (was 186)
          +1 mvnsite 0m 16s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 0m 38s the patch passed
          -1 javadoc 0m 10s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8)
          +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95
          +1 unit 0m 9s hadoop-aws in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 11s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          12m 52s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788510/HADOOP-12709.000.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 2a912087ae79 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8ef75a7
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/artifact/patchprocess/whitespace-eol.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/testReport/
          modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. +1 mvninstall 6m 41s trunk passed +1 compile 0m 12s trunk passed with JDK v1.8.0_91 +1 compile 0m 14s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 19s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 13s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 13s the patch passed +1 compile 0m 9s the patch passed with JDK v1.8.0_91 +1 javac 0m 9s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6) +1 compile 0m 11s the patch passed with JDK v1.7.0_95 +1 javac 0m 11s hadoop-tools_hadoop-aws-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 1 unchanged - 5 fixed = 1 total (was 6) -1 checkstyle 0m 12s hadoop-tools/hadoop-aws: The patch generated 1 new + 58 unchanged - 128 fixed = 59 total (was 186) +1 mvnsite 0m 16s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 0m 38s the patch passed -1 javadoc 0m 10s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8) +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95 +1 unit 0m 9s hadoop-aws in the patch passed with JDK v1.8.0_91. +1 unit 0m 11s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 12m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788510/HADOOP-12709.000.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 2a912087ae79 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8ef75a7 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/artifact/patchprocess/whitespace-eol.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/testReport/ modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9316/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The v1 patch updates the core-default.xml config keys. As some of them are also used by s3n, I renamed them accordingly.

          The newly reported checkstyle and whitespace warnings are existing ones. Let's address them in another jira, along with other fixes. I can work on that as well.

          Show
          liuml07 Mingliang Liu added a comment - The v1 patch updates the core-default.xml config keys. As some of them are also used by s3n, I renamed them accordingly. The newly reported checkstyle and whitespace warnings are existing ones. Let's address them in another jira, along with other fixes. I can work on that as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 24 new or modified test files.
          0 mvndep 2m 12s Maven dependency ordering for branch
          +1 mvninstall 6m 57s trunk passed
          +1 compile 6m 6s trunk passed with JDK v1.8.0_91
          +1 compile 6m 55s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 29s trunk passed
          +1 mvnsite 1m 21s trunk passed
          +1 mvneclipse 1m 5s trunk passed
          +1 findbugs 2m 6s trunk passed
          +1 javadoc 1m 8s trunk passed with JDK v1.8.0_91
          +1 javadoc 1m 20s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 0m 56s the patch passed
          +1 compile 5m 52s the patch passed with JDK v1.8.0_91
          +1 javac 5m 52s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 657 unchanged - 5 fixed = 657 total (was 662)
          +1 compile 6m 44s the patch passed with JDK v1.7.0_95
          +1 javac 6m 44s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672)
          -1 checkstyle 1m 24s root: The patch generated 2 new + 102 unchanged - 134 fixed = 104 total (was 236)
          +1 mvnsite 1m 14s the patch passed
          +1 mvneclipse 0m 27s the patch passed
          -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 29s the patch passed
          -1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8)
          +1 javadoc 1m 19s the patch passed with JDK v1.7.0_95
          -1 unit 6m 57s hadoop-common in the patch failed with JDK v1.8.0_91.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91.
          -1 unit 7m 12s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          68m 52s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.conf.TestCommonConfigurationFields
          JDK v1.7.0_95 Failed junit tests hadoop.conf.TestCommonConfigurationFields



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802782/HADOOP-12709.001.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux f11c499c1ab1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6957e45
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/whitespace-eol.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 24 new or modified test files. 0 mvndep 2m 12s Maven dependency ordering for branch +1 mvninstall 6m 57s trunk passed +1 compile 6m 6s trunk passed with JDK v1.8.0_91 +1 compile 6m 55s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 29s trunk passed +1 mvnsite 1m 21s trunk passed +1 mvneclipse 1m 5s trunk passed +1 findbugs 2m 6s trunk passed +1 javadoc 1m 8s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 20s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 0m 56s the patch passed +1 compile 5m 52s the patch passed with JDK v1.8.0_91 +1 javac 5m 52s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 657 unchanged - 5 fixed = 657 total (was 662) +1 compile 6m 44s the patch passed with JDK v1.7.0_95 +1 javac 6m 44s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672) -1 checkstyle 1m 24s root: The patch generated 2 new + 102 unchanged - 134 fixed = 104 total (was 236) +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 27s the patch passed -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 29s the patch passed -1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8) +1 javadoc 1m 19s the patch passed with JDK v1.7.0_95 -1 unit 6m 57s hadoop-common in the patch failed with JDK v1.8.0_91. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91. -1 unit 7m 12s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 68m 52s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.conf.TestCommonConfigurationFields JDK v1.7.0_95 Failed junit tests hadoop.conf.TestCommonConfigurationFields Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802782/HADOOP-12709.001.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux f11c499c1ab1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6957e45 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/whitespace-eol.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9319/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The v2 patch is to address hadoop.conf.TestCommonConfigurationFields failing test. This is a good catch actually. Again, let's track the existing checkstyle/whitespace warnings along with others in s3n module by separate jira.

          Show
          liuml07 Mingliang Liu added a comment - The v2 patch is to address hadoop.conf.TestCommonConfigurationFields failing test. This is a good catch actually. Again, let's track the existing checkstyle/whitespace warnings along with others in s3n module by separate jira.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 24 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 38s trunk passed
          +1 compile 5m 56s trunk passed with JDK v1.8.0_91
          +1 compile 6m 52s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 27s trunk passed
          +1 mvnsite 1m 19s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 2m 4s trunk passed
          +1 javadoc 1m 8s trunk passed with JDK v1.8.0_91
          +1 javadoc 1m 21s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 0m 55s the patch passed
          +1 compile 5m 53s the patch passed with JDK v1.8.0_91
          +1 javac 5m 53s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 657 unchanged - 5 fixed = 657 total (was 662)
          +1 compile 6m 47s the patch passed with JDK v1.7.0_95
          +1 javac 6m 47s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672)
          -1 checkstyle 1m 24s root: The patch generated 2 new + 102 unchanged - 134 fixed = 104 total (was 236)
          +1 mvnsite 1m 14s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 30s the patch passed
          -1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8)
          +1 javadoc 1m 21s the patch passed with JDK v1.7.0_95
          +1 unit 7m 32s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91.
          +1 unit 7m 40s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          66m 56s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802807/HADOOP-12709.002.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux f2a2fa4b491a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 47c41e7
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/artifact/patchprocess/whitespace-eol.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 24 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 38s trunk passed +1 compile 5m 56s trunk passed with JDK v1.8.0_91 +1 compile 6m 52s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 27s trunk passed +1 mvnsite 1m 19s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 4s trunk passed +1 javadoc 1m 8s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 21s trunk passed with JDK v1.7.0_95 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 0m 55s the patch passed +1 compile 5m 53s the patch passed with JDK v1.8.0_91 +1 javac 5m 53s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 657 unchanged - 5 fixed = 657 total (was 662) +1 compile 6m 47s the patch passed with JDK v1.7.0_95 +1 javac 6m 47s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672) -1 checkstyle 1m 24s root: The patch generated 2 new + 102 unchanged - 134 fixed = 104 total (was 236) +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 30s the patch passed -1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 2 new + 4 unchanged - 4 fixed = 6 total (was 8) +1 javadoc 1m 21s the patch passed with JDK v1.7.0_95 +1 unit 7m 32s hadoop-common in the patch passed with JDK v1.8.0_91. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91. +1 unit 7m 40s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 66m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802807/HADOOP-12709.002.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux f2a2fa4b491a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 47c41e7 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/artifact/patchprocess/whitespace-eol.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9322/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Mingliang Liu, thank you for the updated patch.

          I spotted a few more files that need clean-ups because of the configuration property changes:

          • hadoop-common-project/hadoop-common/src/test/resources/core-site.xml
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json

          The Checkstyle and Javadoc warnings aren't really introduced by this patch, but since we're touching these files anyway, it would be good to go ahead and clean them up.

          Aside from that, this patch looks like the right overall approach to me. I would like a second review from Steve Loughran before we proceed with any commits. In particular, I'd like a second opinion on the configuration property renames and the class renames. Some of these are backwards-incompatible for S3N. I think it's the right thing to do, and we can make a backwards-incompatible change like this in trunk/3.x, but I'd like a second opinion. I know Steve won't be available to comment until mid-next week at the earliest.

          Show
          cnauroth Chris Nauroth added a comment - Mingliang Liu , thank you for the updated patch. I spotted a few more files that need clean-ups because of the configuration property changes: hadoop-common-project/hadoop-common/src/test/resources/core-site.xml hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json The Checkstyle and Javadoc warnings aren't really introduced by this patch, but since we're touching these files anyway, it would be good to go ahead and clean them up. Aside from that, this patch looks like the right overall approach to me. I would like a second review from Steve Loughran before we proceed with any commits. In particular, I'd like a second opinion on the configuration property renames and the class renames. Some of these are backwards-incompatible for S3N. I think it's the right thing to do, and we can make a backwards-incompatible change like this in trunk/3.x, but I'd like a second opinion. I know Steve won't be available to comment until mid-next week at the earliest.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 26 new or modified test files.
          0 mvndep 0m 49s Maven dependency ordering for branch
          +1 mvninstall 6m 42s trunk passed
          +1 compile 5m 48s trunk passed with JDK v1.8.0_91
          +1 compile 6m 42s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 30s trunk passed
          +1 mvnsite 2m 4s trunk passed
          +1 mvneclipse 1m 3s trunk passed
          +1 findbugs 3m 11s trunk passed
          +1 javadoc 1m 32s trunk passed with JDK v1.8.0_91
          +1 javadoc 1m 52s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 32s the patch passed
          +1 compile 5m 43s the patch passed with JDK v1.8.0_91
          +1 javac 5m 43s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 658 unchanged - 5 fixed = 658 total (was 663)
          +1 compile 6m 44s the patch passed with JDK v1.7.0_95
          +1 javac 6m 44s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672)
          +1 checkstyle 1m 26s root: The patch generated 0 new + 103 unchanged - 133 fixed = 103 total (was 236)
          +1 mvnsite 1m 59s the patch passed
          +1 mvneclipse 0m 56s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 4m 1s the patch passed
          +1 javadoc 0m 55s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 javadoc 0m 13s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91.
          +1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8)
          +1 javadoc 0m 13s hadoop-sls in the patch passed with JDK v1.8.0_91.
          +1 javadoc 1m 52s the patch passed with JDK v1.7.0_95
          +1 unit 7m 40s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 unit 5m 35s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 53s hadoop-sls in the patch passed with JDK v1.8.0_91.
          +1 unit 7m 55s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 5m 57s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 55s hadoop-sls in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          88m 54s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803057/HADOOP-12709.003.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux dbd0620fe027 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 411fb4b
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9343/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9343/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 26 new or modified test files. 0 mvndep 0m 49s Maven dependency ordering for branch +1 mvninstall 6m 42s trunk passed +1 compile 5m 48s trunk passed with JDK v1.8.0_91 +1 compile 6m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 30s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 1m 3s trunk passed +1 findbugs 3m 11s trunk passed +1 javadoc 1m 32s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 52s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 5m 43s the patch passed with JDK v1.8.0_91 +1 javac 5m 43s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 658 unchanged - 5 fixed = 658 total (was 663) +1 compile 6m 44s the patch passed with JDK v1.7.0_95 +1 javac 6m 44s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672) +1 checkstyle 1m 26s root: The patch generated 0 new + 103 unchanged - 133 fixed = 103 total (was 236) +1 mvnsite 1m 59s the patch passed +1 mvneclipse 0m 56s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 4m 1s the patch passed +1 javadoc 0m 55s hadoop-common in the patch passed with JDK v1.8.0_91. +1 javadoc 0m 13s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91. +1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8) +1 javadoc 0m 13s hadoop-sls in the patch passed with JDK v1.8.0_91. +1 javadoc 1m 52s the patch passed with JDK v1.7.0_95 +1 unit 7m 40s hadoop-common in the patch passed with JDK v1.8.0_91. +1 unit 5m 35s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91. +1 unit 0m 53s hadoop-sls in the patch passed with JDK v1.8.0_91. +1 unit 7m 55s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 5m 57s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 unit 0m 55s hadoop-sls in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 88m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803057/HADOOP-12709.003.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux dbd0620fe027 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 411fb4b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9343/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9343/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -
          1. Can we keep the original names as deprecated? Even though we can break compatibility in trunk, it doesn't mean we should.
          2. references in code (e.g. InMemoryNativeFileSystemStore), NativeS3FileSystem must refer via constant references, not inline strings.
          3. S3Credentials could switch validaton code to Preconditions checks (with the same messages).
          Show
          stevel@apache.org Steve Loughran added a comment - Can we keep the original names as deprecated? Even though we can break compatibility in trunk, it doesn't mean we should . references in code (e.g. InMemoryNativeFileSystemStore ), NativeS3FileSystem must refer via constant references, not inline strings. S3Credentials could switch validaton code to Preconditions checks (with the same messages).
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Chris Nauroth for your review and nice catches. Sorry I missed the places out of the hadoop-aws module. In the current patch, all the clean-ups you listed are addressed. The checkstyle and javadoc warnings are fixed as well.

          Thanks Steve Loughran for the review and comment. I was kind of aggressive about changing the config keys/values, but I agree that if we should keep the original names and mark them as deprecated if we can. This way, the existing applications using s3n don't have to update their configurations to run. As to the implementation, the v4 patch employs the Configuration#addDeprecations in the static block. I'm wondering if there is a better way.
          Your 2nd and 3rd comments are very valid though I was thinking of addressing them separately along with other existing places where inline strings were used. As those changes are very related but not complex, I also think it's doable in this patch. See v4 if I address them correctly.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Chris Nauroth for your review and nice catches. Sorry I missed the places out of the hadoop-aws module. In the current patch, all the clean-ups you listed are addressed. The checkstyle and javadoc warnings are fixed as well. Thanks Steve Loughran for the review and comment. I was kind of aggressive about changing the config keys/values, but I agree that if we should keep the original names and mark them as deprecated if we can. This way, the existing applications using s3n don't have to update their configurations to run. As to the implementation, the v4 patch employs the Configuration#addDeprecations in the static block. I'm wondering if there is a better way. Your 2nd and 3rd comments are very valid though I was thinking of addressing them separately along with other existing places where inline strings were used. As those changes are very related but not complex, I also think it's doable in this patch. See v4 if I address them correctly.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 26 new or modified test files.
          0 mvndep 0m 54s Maven dependency ordering for branch
          +1 mvninstall 6m 52s trunk passed
          +1 compile 6m 7s trunk passed with JDK v1.8.0_91
          +1 compile 6m 44s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 28s trunk passed
          +1 mvnsite 2m 4s trunk passed
          +1 mvneclipse 0m 55s trunk passed
          +1 findbugs 3m 11s trunk passed
          +1 javadoc 1m 32s trunk passed with JDK v1.8.0_91
          +1 javadoc 1m 50s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 30s the patch passed
          +1 compile 5m 53s the patch passed with JDK v1.8.0_91
          +1 javac 5m 53s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 657 unchanged - 5 fixed = 657 total (was 662)
          +1 compile 6m 40s the patch passed with JDK v1.7.0_95
          +1 javac 6m 40s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672)
          +1 checkstyle 1m 26s root: The patch generated 0 new + 107 unchanged - 134 fixed = 107 total (was 241)
          +1 mvnsite 1m 59s the patch passed
          +1 mvneclipse 0m 54s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 56s the patch passed
          +1 javadoc 0m 53s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 javadoc 0m 12s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91.
          +1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8)
          +1 javadoc 0m 13s hadoop-sls in the patch passed with JDK v1.8.0_91.
          +1 javadoc 1m 50s the patch passed with JDK v1.7.0_95
          -1 unit 6m 53s hadoop-common in the patch failed with JDK v1.8.0_91.
          +1 unit 5m 43s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 51s hadoop-sls in the patch passed with JDK v1.8.0_91.
          -1 unit 7m 10s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 6m 9s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 56s hadoop-sls in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          87m 59s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.net.TestDNS
          JDK v1.7.0_95 Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803477/HADOOP-12709.004.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux b512f56b9a60 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / acb509b
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 26 new or modified test files. 0 mvndep 0m 54s Maven dependency ordering for branch +1 mvninstall 6m 52s trunk passed +1 compile 6m 7s trunk passed with JDK v1.8.0_91 +1 compile 6m 44s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 28s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 0m 55s trunk passed +1 findbugs 3m 11s trunk passed +1 javadoc 1m 32s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 50s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 5m 53s the patch passed with JDK v1.8.0_91 +1 javac 5m 53s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 657 unchanged - 5 fixed = 657 total (was 662) +1 compile 6m 40s the patch passed with JDK v1.7.0_95 +1 javac 6m 40s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 667 unchanged - 5 fixed = 667 total (was 672) +1 checkstyle 1m 26s root: The patch generated 0 new + 107 unchanged - 134 fixed = 107 total (was 241) +1 mvnsite 1m 59s the patch passed +1 mvneclipse 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 56s the patch passed +1 javadoc 0m 53s hadoop-common in the patch passed with JDK v1.8.0_91. +1 javadoc 0m 12s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91. +1 javadoc 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8) +1 javadoc 0m 13s hadoop-sls in the patch passed with JDK v1.8.0_91. +1 javadoc 1m 50s the patch passed with JDK v1.7.0_95 -1 unit 6m 53s hadoop-common in the patch failed with JDK v1.8.0_91. +1 unit 5m 43s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_91. +1 unit 0m 51s hadoop-sls in the patch passed with JDK v1.8.0_91. -1 unit 7m 10s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 6m 9s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 unit 0m 56s hadoop-sls in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 87m 59s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.net.TestDNS JDK v1.7.0_95 Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803477/HADOOP-12709.004.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux b512f56b9a60 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / acb509b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9370/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          hadoop.net.TestDNS is not related and is tracked by HADOOP-13101.

          Show
          liuml07 Mingliang Liu added a comment - hadoop.net.TestDNS is not related and is tracked by HADOOP-13101 .
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch LGTM, though I will need to do a full test run of AWS to be sure things will work

          Chris, do you have any comments.

          Show
          stevel@apache.org Steve Loughran added a comment - Patch LGTM, though I will need to do a full test run of AWS to be sure things will work Chris, do you have any comments.
          Hide
          cnauroth Chris Nauroth added a comment -

          The patch looks good to me too. I don't have any further comments, but it looks like it needs to be rebased after a few recent commits.

          The current scope of this JIRA is stated as "deprecate from branch-2 and remove from trunk". I am wondering if it makes more sense to split those 2 tasks into 2 separate JIRAs. That way, we'd be able to enter a release note for 2.x that says "it's going away soon" and a separate release note for 3.0 that says "it's gone". What are your thoughts? Also, Mingliang, were you planning on doing the deprecation part of this too?

          Show
          cnauroth Chris Nauroth added a comment - The patch looks good to me too. I don't have any further comments, but it looks like it needs to be rebased after a few recent commits. The current scope of this JIRA is stated as "deprecate from branch-2 and remove from trunk". I am wondering if it makes more sense to split those 2 tasks into 2 separate JIRAs. That way, we'd be able to enter a release note for 2.x that says "it's going away soon" and a separate release note for 3.0 that says "it's gone". What are your thoughts? Also, Mingliang, were you planning on doing the deprecation part of this too?
          Hide
          liuml07 Mingliang Liu added a comment -

          Hi Chris Nauroth, thanks for your review. I'll post a rebased version soon.

          I like your proposal. I'd like to work on the branch-2 deprecation.

          Show
          liuml07 Mingliang Liu added a comment - Hi Chris Nauroth , thanks for your review. I'll post a rebased version soon. I like your proposal. I'd like to work on the branch-2 deprecation.
          Hide
          liuml07 Mingliang Liu added a comment -

          Rebase from trunk.

          Show
          liuml07 Mingliang Liu added a comment - Rebase from trunk .
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for patch 005, pending pre-commit. I'll hold off committing in case Steve wants to comment one more time or talk about the proposal to split this into 2 JIRAs.

          Show
          cnauroth Chris Nauroth added a comment - +1 for patch 005, pending pre-commit. I'll hold off committing in case Steve wants to comment one more time or talk about the proposal to split this into 2 JIRAs.
          Hide
          cnauroth Chris Nauroth added a comment -

          Also, I did a full hadoop-aws test run in parallel mode after applying patch 005 to trunk. Everything passed as expected, and total test execution time for the whole hadoop-aws module came down to 11 minutes 45 seconds.

          Show
          cnauroth Chris Nauroth added a comment - Also, I did a full hadoop-aws test run in parallel mode after applying patch 005 to trunk. Everything passed as expected, and total test execution time for the whole hadoop-aws module came down to 11 minutes 45 seconds.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks to the parallel test enhancement, Chris Nauroth. It's awesome!

          Show
          liuml07 Mingliang Liu added a comment - Thanks to the parallel test enhancement, Chris Nauroth . It's awesome!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 26 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 6m 48s trunk passed
          +1 compile 7m 12s trunk passed
          +1 checkstyle 1m 26s trunk passed
          +1 mvnsite 1m 55s trunk passed
          +1 mvneclipse 0m 46s trunk passed
          +1 findbugs 3m 1s trunk passed
          +1 javadoc 1m 36s trunk passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 32s the patch passed
          +1 compile 7m 12s the patch passed
          +1 javac 7m 12s root generated 0 new + 693 unchanged - 5 fixed = 693 total (was 698)
          +1 checkstyle 1m 24s root: The patch generated 0 new + 107 unchanged - 131 fixed = 107 total (was 238)
          +1 mvnsite 1m 58s the patch passed
          +1 mvneclipse 0m 47s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 findbugs 3m 21s the patch passed
          +1 javadoc 1m 33s the patch passed
          -1 unit 7m 54s hadoop-common in the patch failed.
          +1 unit 6m 8s hadoop-mapreduce-client-hs in the patch passed.
          +1 unit 0m 13s hadoop-aws in the patch passed.
          +1 unit 0m 52s hadoop-sls in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          57m 56s



          Reason Tests
          Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807830/HADOOP-12709.005.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux a7824df6b3b8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 97e2449
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 26 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 6m 48s trunk passed +1 compile 7m 12s trunk passed +1 checkstyle 1m 26s trunk passed +1 mvnsite 1m 55s trunk passed +1 mvneclipse 0m 46s trunk passed +1 findbugs 3m 1s trunk passed +1 javadoc 1m 36s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 7m 12s the patch passed +1 javac 7m 12s root generated 0 new + 693 unchanged - 5 fixed = 693 total (was 698) +1 checkstyle 1m 24s root: The patch generated 0 new + 107 unchanged - 131 fixed = 107 total (was 238) +1 mvnsite 1m 58s the patch passed +1 mvneclipse 0m 47s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 3m 21s the patch passed +1 javadoc 1m 33s the patch passed -1 unit 7m 54s hadoop-common in the patch failed. +1 unit 6m 8s hadoop-mapreduce-client-hs in the patch passed. +1 unit 0m 13s hadoop-aws in the patch passed. +1 unit 0m 52s hadoop-sls in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 57m 56s Reason Tests Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807830/HADOOP-12709.005.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux a7824df6b3b8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 97e2449 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9654/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Once again, the failure in TestDNS is unrelated. Otherwise, pre-commit looks clean.

          Show
          cnauroth Chris Nauroth added a comment - Once again, the failure in TestDNS is unrelated. Otherwise, pre-commit looks clean.
          Hide
          liuml07 Mingliang Liu added a comment -

          Hi Chris Nauroth, I filed JIRA ticket HADOOP-13239 for deprecating s3:// from branch-2, assigned it to myself, and updated this JIRA's description. Let's track the effort of cutting it from trunk here, as what we have been doing.

          Show
          liuml07 Mingliang Liu added a comment - Hi Chris Nauroth , I filed JIRA ticket HADOOP-13239 for deprecating s3:// from branch-2, assigned it to myself, and updated this JIRA's description. Let's track the effort of cutting it from trunk here, as what we have been doing.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 006; this just 005 in sync with trunk; tested against s3 us-east.

          If yetus is happy, I'll +1 this

          (generated with a git format-patch out of curiosity to see what yetus does)

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 006; this just 005 in sync with trunk; tested against s3 us-east. If yetus is happy, I'll +1 this (generated with a git format-patch out of curiosity to see what yetus does)
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 26 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 8m 27s trunk passed
          +1 compile 8m 45s trunk passed
          +1 checkstyle 1m 38s trunk passed
          +1 mvnsite 2m 21s trunk passed
          +1 mvneclipse 1m 9s trunk passed
          +1 findbugs 3m 30s trunk passed
          +1 javadoc 1m 45s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 55s the patch passed
          +1 compile 8m 44s the patch passed
          +1 javac 8m 44s root generated 0 new + 709 unchanged - 5 fixed = 709 total (was 714)
          +1 checkstyle 1m 37s root: The patch generated 0 new + 107 unchanged - 131 fixed = 107 total (was 238)
          +1 mvnsite 2m 19s the patch passed
          +1 mvneclipse 1m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 4s The patch has no ill-formed XML file.
          +1 findbugs 4m 16s the patch passed
          +1 javadoc 1m 52s the patch passed
          +1 unit 10m 55s hadoop-common in the patch passed.
          +1 unit 6m 37s hadoop-mapreduce-client-hs in the patch passed.
          +1 unit 0m 22s hadoop-aws in the patch passed.
          +1 unit 1m 0s hadoop-sls in the patch passed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          92m 56s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12814695/HADOOP-12709-006.patch
          JIRA Issue HADOOP-12709
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux cced514d4c66 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8113855
          Default Java 1.8.0_91
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9897/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9897/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 26 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 8m 27s trunk passed +1 compile 8m 45s trunk passed +1 checkstyle 1m 38s trunk passed +1 mvnsite 2m 21s trunk passed +1 mvneclipse 1m 9s trunk passed +1 findbugs 3m 30s trunk passed +1 javadoc 1m 45s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 55s the patch passed +1 compile 8m 44s the patch passed +1 javac 8m 44s root generated 0 new + 709 unchanged - 5 fixed = 709 total (was 714) +1 checkstyle 1m 37s root: The patch generated 0 new + 107 unchanged - 131 fixed = 107 total (was 238) +1 mvnsite 2m 19s the patch passed +1 mvneclipse 1m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 4s The patch has no ill-formed XML file. +1 findbugs 4m 16s the patch passed +1 javadoc 1m 52s the patch passed +1 unit 10m 55s hadoop-common in the patch passed. +1 unit 6m 37s hadoop-mapreduce-client-hs in the patch passed. +1 unit 0m 22s hadoop-aws in the patch passed. +1 unit 1m 0s hadoop-sls in the patch passed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 92m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12814695/HADOOP-12709-006.patch JIRA Issue HADOOP-12709 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux cced514d4c66 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8113855 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9897/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-aws hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9897/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1
          committed!

          Show
          stevel@apache.org Steve Loughran added a comment - +1 committed!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10035 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10035/)
          HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu. (stevel: rev 96fa0f848bbd623ab55092889fb7b040a2a2971c)

          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractOpen.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractRename.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestS3Credentials.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3Credentials.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestINode.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/TestS3Credentials.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/INode.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/S3Exception.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3OutputStream.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java
          • hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/FileSystemStore.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3InputStream.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractCreate.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/MigrationTool.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/package.html
          • hadoop-tools/hadoop-aws/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/package.html
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3FileSystemConfigKeys.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestInMemoryS3FileSystemContract.java
          • hadoop-common-project/hadoop-common/src/test/resources/core-site.xml
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/Jets3tS3FileSystemContractTest.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/S3Credentials.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3FileSystemException.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocal_S3FileContextURI.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestS3InMemoryFileSystem.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/S3Contract.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractSeek.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestS3_LocalFileContextURI.java
          • hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractMkdir.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/InMemoryNativeFileSystemStore.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java
          • hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/S3FileSystemContractBaseTest.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • hadoop-tools/hadoop-aws/src/test/resources/contract/s3.xml
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/VersionMismatchException.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractRootDir.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/S3NativeFileSystemConfigKeys.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3Exception.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestS3FileSystem.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractDelete.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Block.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/S3InMemoryFileSystem.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/Jets3tNativeFileSystemStore.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10035 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10035/ ) HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu. (stevel: rev 96fa0f848bbd623ab55092889fb7b040a2a2971c) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractOpen.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractRename.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestS3Credentials.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3Credentials.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestINode.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/TestS3Credentials.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/INode.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/S3Exception.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3OutputStream.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/FileSystemStore.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3InputStream.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractCreate.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/MigrationTool.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/package.html hadoop-tools/hadoop-aws/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/package.html hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3FileSystemConfigKeys.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestInMemoryS3FileSystemContract.java hadoop-common-project/hadoop-common/src/test/resources/core-site.xml hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/Jets3tS3FileSystemContractTest.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/S3Credentials.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3FileSystemException.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocal_S3FileContextURI.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestS3InMemoryFileSystem.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/S3Contract.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractSeek.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestS3_LocalFileContextURI.java hadoop-common-project/hadoop-common/src/main/resources/core-default.xml hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractMkdir.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/InMemoryNativeFileSystemStore.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/S3FileSystemContractBaseTest.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml hadoop-tools/hadoop-aws/src/test/resources/contract/s3.xml hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/VersionMismatchException.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractRootDir.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/S3NativeFileSystemConfigKeys.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/S3Exception.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/TestS3FileSystem.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3/TestS3ContractDelete.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Block.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3/S3InMemoryFileSystem.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/Jets3tNativeFileSystemStore.java
          Hide
          liuml07 Mingliang Liu added a comment -

          Thank you Steve Loughran for your review and commit!

          Show
          liuml07 Mingliang Liu added a comment - Thank you Steve Loughran for your review and commit!

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development