Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0
    • Component/s: fs/s3
    • Labels:
      None

      Description

      The fact that s3:// was broken in Hadoop 2.7 and nobody noticed until now shows that it's not being used. while invaluable at the time, s3n and especially s3a render it obsolete except for reading existing data.

      HADOOP-12709 cuts the s3:// from trunk branch, and this JIRA ticket is to deprecate it from branch-2.

      1. Mark Java source as @deprecated
      2. Warn the first time in a JVM that an S3 instance is created, "deprecated -will be removed in future releases"

      Thanks Steve Loughran for the proposal.

      1. HADOOP-13239-branch-2.000.patch
        19 kB
        Mingliang Liu
      2. HADOOP-13239-branch-2.001.patch
        19 kB
        Mingliang Liu

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 17 new or modified test files.
          +1 mvninstall 6m 46s branch-2 passed
          +1 compile 0m 14s branch-2 passed with JDK v1.8.0_91
          +1 compile 0m 15s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 16s branch-2 passed
          +1 mvnsite 0m 21s branch-2 passed
          +1 mvneclipse 0m 18s branch-2 passed
          +1 findbugs 0m 34s branch-2 passed
          +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_91
          +1 javadoc 0m 14s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 14s the patch passed
          +1 compile 0m 11s the patch passed with JDK v1.8.0_91
          -1 javac 0m 11s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 10 new + 1 unchanged - 5 fixed = 11 total (was 6)
          +1 compile 0m 13s the patch passed with JDK v1.7.0_101
          -1 javac 0m 13s hadoop-tools_hadoop-aws-jdk1.7.0_101 with JDK v1.7.0_101 generated 14 new + 1 unchanged - 5 fixed = 15 total (was 6)
          -1 checkstyle 0m 13s hadoop-tools/hadoop-aws: The patch generated 15 new + 115 unchanged - 13 fixed = 130 total (was 128)
          +1 mvnsite 0m 18s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 41s the patch passed
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 12s the patch passed with JDK v1.7.0_101
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          13m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:d1c475d
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811279/HADOOP-13239-branch-2.000.patch
          JIRA Issue HADOOP-13239
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 44d3f760062f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ddf6642
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.7.0_101.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/testReport/
          modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 17 new or modified test files. +1 mvninstall 6m 46s branch-2 passed +1 compile 0m 14s branch-2 passed with JDK v1.8.0_91 +1 compile 0m 15s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 16s branch-2 passed +1 mvnsite 0m 21s branch-2 passed +1 mvneclipse 0m 18s branch-2 passed +1 findbugs 0m 34s branch-2 passed +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_91 +1 javadoc 0m 14s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 14s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_91 -1 javac 0m 11s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 10 new + 1 unchanged - 5 fixed = 11 total (was 6) +1 compile 0m 13s the patch passed with JDK v1.7.0_101 -1 javac 0m 13s hadoop-tools_hadoop-aws-jdk1.7.0_101 with JDK v1.7.0_101 generated 14 new + 1 unchanged - 5 fixed = 15 total (was 6) -1 checkstyle 0m 13s hadoop-tools/hadoop-aws: The patch generated 15 new + 115 unchanged - 13 fixed = 130 total (was 128) +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 41s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_101 +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 13m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:d1c475d JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811279/HADOOP-13239-branch-2.000.patch JIRA Issue HADOOP-13239 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 44d3f760062f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ddf6642 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.7.0_101.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/testReport/ modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9809/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The v1 patch is to address hasWarnedDeprecation checkstyle warning. Others are expected as we're deprecating classes.

          Show
          liuml07 Mingliang Liu added a comment - The v1 patch is to address hasWarnedDeprecation checkstyle warning. Others are expected as we're deprecating classes.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 24s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 17 new or modified test files.
          +1 mvninstall 7m 43s branch-2 passed
          +1 compile 0m 21s branch-2 passed with JDK v1.8.0_91
          +1 compile 0m 19s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 15s branch-2 passed
          +1 mvnsite 0m 20s branch-2 passed
          +1 mvneclipse 0m 15s branch-2 passed
          +1 findbugs 0m 39s branch-2 passed
          +1 javadoc 0m 14s branch-2 passed with JDK v1.8.0_91
          +1 javadoc 0m 16s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 12s the patch passed with JDK v1.8.0_91
          -1 javac 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 10 new + 1 unchanged - 5 fixed = 11 total (was 6)
          +1 compile 0m 15s the patch passed with JDK v1.7.0_101
          -1 javac 0m 15s hadoop-tools_hadoop-aws-jdk1.7.0_101 with JDK v1.7.0_101 generated 14 new + 1 unchanged - 5 fixed = 15 total (was 6)
          -1 checkstyle 0m 14s hadoop-tools/hadoop-aws: The patch generated 14 new + 115 unchanged - 13 fixed = 129 total (was 128)
          +1 mvnsite 0m 19s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 52s the patch passed
          +1 javadoc 0m 11s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 14s the patch passed with JDK v1.7.0_101
          +1 unit 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          15m 28s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:d1c475d
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811289/HADOOP-13239-branch-2.001.patch
          JIRA Issue HADOOP-13239
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 32ec446d36dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ddf6642
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.7.0_101.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/testReport/
          modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 17 new or modified test files. +1 mvninstall 7m 43s branch-2 passed +1 compile 0m 21s branch-2 passed with JDK v1.8.0_91 +1 compile 0m 19s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 15s branch-2 passed +1 mvnsite 0m 20s branch-2 passed +1 mvneclipse 0m 15s branch-2 passed +1 findbugs 0m 39s branch-2 passed +1 javadoc 0m 14s branch-2 passed with JDK v1.8.0_91 +1 javadoc 0m 16s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 15s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_91 -1 javac 0m 12s hadoop-tools_hadoop-aws-jdk1.8.0_91 with JDK v1.8.0_91 generated 10 new + 1 unchanged - 5 fixed = 11 total (was 6) +1 compile 0m 15s the patch passed with JDK v1.7.0_101 -1 javac 0m 15s hadoop-tools_hadoop-aws-jdk1.7.0_101 with JDK v1.7.0_101 generated 14 new + 1 unchanged - 5 fixed = 15 total (was 6) -1 checkstyle 0m 14s hadoop-tools/hadoop-aws: The patch generated 14 new + 115 unchanged - 13 fixed = 129 total (was 128) +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 52s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_101 +1 unit 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 15m 28s Subsystem Report/Notes Docker Image:yetus/hadoop:d1c475d JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811289/HADOOP-13239-branch-2.001.patch JIRA Issue HADOOP-13239 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 32ec446d36dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ddf6642 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.8.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/artifact/patchprocess/diff-compile-javac-hadoop-tools_hadoop-aws-jdk1.7.0_101.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/testReport/ modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9811/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          tested against S3 ireland; saw a transient failure in an s3a test, but that is implicitly unrelated to this.

          Show
          stevel@apache.org Steve Loughran added a comment - +1 tested against S3 ireland; saw a transient failure in an s3a test, but that is implicitly unrelated to this.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Steve Loughran for your review. This and HADOOP-12709 together should address the proposal for abandoning s3:// in trunk and branch-2.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Steve Loughran for your review. This and HADOOP-12709 together should address the proposal for abandoning s3:// in trunk and branch-2 .
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          Committed to branch-2 and branch-2.8. Not done to trunk, as I'm going to cut s3 out there entirely

          Show
          stevel@apache.org Steve Loughran added a comment - +1 Committed to branch-2 and branch-2.8. Not done to trunk, as I'm going to cut s3 out there entirely
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Steve Loughran for the review and commit.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Steve Loughran for the review and commit.

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              liuml07 Mingliang Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development