Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14248

Retire SharedInstanceProfileCredentialsProvider in trunk.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha4
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      SharedInstanceProfileCredentialsProvider is removed after this change. Users should use InstanceProfileCredentialsProvider provided by AWS SDK instead, which itself enforces a singleton instance to reduce calls to AWS EC2 Instance Metadata Service.

      Description

      This is from the discussion in HADOOP-13050.

      So HADOOP-13727 added the SharedInstanceProfileCredentialsProvider, which effectively reduces high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. That patch, in order to prevent the throttling problem, defined new class SharedInstanceProfileCredentialsProvider as a subclass of InstanceProfileCredentialsProvider, which enforces creation of only a single instance.

      Per HADOOP-13050, we upgraded the AWS Java SDK. Since then, the InstanceProfileCredentialsProvider in SDK code internally enforces a singleton. That confirms that our effort in HADOOP-13727 makes 100% sense. Meanwhile, SharedInstanceProfileCredentialsProvider can retire gracefully in trunk branch.

      1. HADOOP-14248.000.patch
        12 kB
        Mingliang Liu
      2. HADOOP-14248-branch-2.001.patch
        11 kB
        Mingliang Liu
      3. HADOOP-14248.001.patch
        12 kB
        Mingliang Liu
      4. HADOOP-14248-branch-2.002.patch
        11 kB
        Mingliang Liu

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/)
          HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. (cnauroth: rev b8305e6d06cbb9d44c728da14506d5cf825c12e5)

          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java
          • (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • (delete) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceProfileCredentialsProvider.java
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java
          • (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/ ) HADOOP-14248 . Retire SharedInstanceProfileCredentialsProvider in trunk. (cnauroth: rev b8305e6d06cbb9d44c728da14506d5cf825c12e5) (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml (delete) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceProfileCredentialsProvider.java (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11577 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11577/)
          HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. (cnauroth: rev b8305e6d06cbb9d44c728da14506d5cf825c12e5)

          • (delete) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceProfileCredentialsProvider.java
          • (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java
          • (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11577 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11577/ ) HADOOP-14248 . Retire SharedInstanceProfileCredentialsProvider in trunk. (cnauroth: rev b8305e6d06cbb9d44c728da14506d5cf825c12e5) (delete) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceProfileCredentialsProvider.java (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 and committed to both trunk and branch-2. The new JIRA HADOOP-14301 tracks the branch-2 commit of the deprecation, for inclusion in 2.9.0. I also made that a sub-task of HADOOP-13204, which is also targeted to 2.9.0. For this one, I converted to a top-level issue, because it wouldn't make sense for an issue targeted to 3.0.0-alpha3 to be a sub-task of an issue targeted to 2.9.0. (If there is an umbrella S3A JIRA targeted to 3.x that I'm not aware of, please feel free to move it under there.)

          Thank you for the patch, Mingliang Liu.

          Show
          cnauroth Chris Nauroth added a comment - +1 and committed to both trunk and branch-2. The new JIRA HADOOP-14301 tracks the branch-2 commit of the deprecation, for inclusion in 2.9.0. I also made that a sub-task of HADOOP-13204 , which is also targeted to 2.9.0. For this one, I converted to a top-level issue, because it wouldn't make sense for an issue targeted to 3.0.0-alpha3 to be a sub-task of an issue targeted to 2.9.0. (If there is an umbrella S3A JIRA targeted to 3.x that I'm not aware of, please feel free to move it under there.) Thank you for the patch, Mingliang Liu .
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Chris Nauroth for taking care of this. I'm totally +1 on the proposal.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Chris Nauroth for taking care of this. I'm totally +1 on the proposal.
          Hide
          cnauroth Chris Nauroth added a comment -

          Mingliang Liu, this looks good now, and I confirmed a full test run against US-west-2.

          However, I'm now thinking that we need two separates JIRA issues, just for the sake of accurate tracking against target versions and release notes. HADOOP-14248 would track the removal from trunk (already covered by the current release note), and the new issue would be targeted to 2.9.0 with a different release note describing deprecation instead of removal. (No need to repeat pre-commit. I'd just comment on the new JIRA that pre-commit for branch-2 was already covered here.)

          Do you think that makes sense? If so, I'd be happy to be the JIRA janitor and finish off committing this.

          Show
          cnauroth Chris Nauroth added a comment - Mingliang Liu , this looks good now, and I confirmed a full test run against US-west-2. However, I'm now thinking that we need two separates JIRA issues, just for the sake of accurate tracking against target versions and release notes. HADOOP-14248 would track the removal from trunk (already covered by the current release note), and the new issue would be targeted to 2.9.0 with a different release note describing deprecation instead of removal. (No need to repeat pre-commit. I'd just comment on the new JIRA that pre-commit for branch-2 was already covered here.) Do you think that makes sense? If so, I'd be happy to be the JIRA janitor and finish off committing this.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 47s Maven dependency ordering for branch
          +1 mvninstall 6m 48s branch-2 passed
          +1 compile 6m 5s branch-2 passed with JDK v1.8.0_121
          +1 compile 6m 50s branch-2 passed with JDK v1.7.0_121
          +1 checkstyle 1m 29s branch-2 passed
          +1 mvnsite 1m 28s branch-2 passed
          +1 mvneclipse 1m 25s branch-2 passed
          +1 findbugs 2m 21s branch-2 passed
          +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_121
          +1 javadoc 1m 10s branch-2 passed with JDK v1.7.0_121
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 4s the patch passed
          +1 compile 5m 39s the patch passed with JDK v1.8.0_121
          -1 javac 5m 39s root-jdk1.8.0_121 with JDK v1.8.0_121 generated 2 new + 899 unchanged - 1 fixed = 901 total (was 900)
          +1 compile 6m 39s the patch passed with JDK v1.7.0_121
          -1 javac 6m 39s root-jdk1.7.0_121 with JDK v1.7.0_121 generated 2 new + 992 unchanged - 1 fixed = 994 total (was 993)
          +1 checkstyle 1m 33s the patch passed
          +1 mvnsite 1m 38s the patch passed
          +1 mvneclipse 0m 36s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 49s the patch passed
          +1 javadoc 1m 7s the patch passed with JDK v1.8.0_121
          +1 javadoc 1m 19s the patch passed with JDK v1.7.0_121
          +1 unit 8m 5s hadoop-common in the patch passed with JDK v1.7.0_121.
          +1 unit 0m 30s hadoop-aws in the patch passed with JDK v1.7.0_121.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          96m 36s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-14248
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862904/HADOOP-14248-branch-2.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 9e9121a8d467 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / b04c091
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_121.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 47s Maven dependency ordering for branch +1 mvninstall 6m 48s branch-2 passed +1 compile 6m 5s branch-2 passed with JDK v1.8.0_121 +1 compile 6m 50s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 1m 29s branch-2 passed +1 mvnsite 1m 28s branch-2 passed +1 mvneclipse 1m 25s branch-2 passed +1 findbugs 2m 21s branch-2 passed +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_121 +1 javadoc 1m 10s branch-2 passed with JDK v1.7.0_121 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 4s the patch passed +1 compile 5m 39s the patch passed with JDK v1.8.0_121 -1 javac 5m 39s root-jdk1.8.0_121 with JDK v1.8.0_121 generated 2 new + 899 unchanged - 1 fixed = 901 total (was 900) +1 compile 6m 39s the patch passed with JDK v1.7.0_121 -1 javac 6m 39s root-jdk1.7.0_121 with JDK v1.7.0_121 generated 2 new + 992 unchanged - 1 fixed = 994 total (was 993) +1 checkstyle 1m 33s the patch passed +1 mvnsite 1m 38s the patch passed +1 mvneclipse 0m 36s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 49s the patch passed +1 javadoc 1m 7s the patch passed with JDK v1.8.0_121 +1 javadoc 1m 19s the patch passed with JDK v1.7.0_121 +1 unit 8m 5s hadoop-common in the patch passed with JDK v1.7.0_121. +1 unit 0m 30s hadoop-aws in the patch passed with JDK v1.7.0_121. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 96m 36s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-14248 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862904/HADOOP-14248-branch-2.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 9e9121a8d467 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / b04c091 Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_121.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12084/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Chris Nauroth very much for your review. The v2 patch for branch-2 addresses your comment. Are you +1 on the current patch (trunk and branch-2 respectively)?

          Show
          liuml07 Mingliang Liu added a comment - Thanks Chris Nauroth very much for your review. The v2 patch for branch-2 addresses your comment. Are you +1 on the current patch (trunk and branch-2 respectively)?
          Hide
          cnauroth Chris Nauroth added a comment -

          Hello Mingliang Liu. This looks good overall. I have a comment on the branch-2 patch.

             private SharedInstanceProfileCredentialsProvider() {
          -    super();
          +    InstanceProfileCredentialsProvider.getInstance();
             }
          

          I don't think this change is necessary. The call to InstanceProfileCredentialsProvider#getInstance() returns an instance (always the same one now that we've upgraded the AWS SDK), but then it never saves a reference to that instance or does anything else with it.

          Show
          cnauroth Chris Nauroth added a comment - Hello Mingliang Liu . This looks good overall. I have a comment on the branch-2 patch. private SharedInstanceProfileCredentialsProvider() { - super (); + InstanceProfileCredentialsProvider.getInstance(); } I don't think this change is necessary. The call to InstanceProfileCredentialsProvider#getInstance() returns an instance (always the same one now that we've upgraded the AWS SDK), but then it never saves a reference to that instance or does anything else with it.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 2m 0s Maven dependency ordering for branch
          +1 mvninstall 14m 58s trunk passed
          +1 compile 19m 10s trunk passed
          +1 checkstyle 2m 4s trunk passed
          +1 mvnsite 1m 42s trunk passed
          +1 mvneclipse 0m 48s trunk passed
          +1 findbugs 2m 7s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 0s the patch passed
          +1 compile 14m 12s the patch passed
          +1 javac 14m 12s root generated 0 new + 776 unchanged - 1 fixed = 776 total (was 777)
          +1 checkstyle 1m 59s the patch passed
          +1 mvnsite 1m 33s the patch passed
          +1 mvneclipse 0m 46s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 2m 27s the patch passed
          +1 javadoc 1m 20s the patch passed
          -1 unit 7m 37s hadoop-common in the patch failed.
          +1 unit 0m 34s hadoop-aws in the patch passed.
          +1 asflicense 0m 37s The patch does not generate ASF License warnings.
          101m 19s



          Reason Tests
          Failed junit tests hadoop.security.TestRaceWhenRelogin



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-14248
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860939/HADOOP-14248.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux cc86f2b4f7c0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 01aca54
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 2m 0s Maven dependency ordering for branch +1 mvninstall 14m 58s trunk passed +1 compile 19m 10s trunk passed +1 checkstyle 2m 4s trunk passed +1 mvnsite 1m 42s trunk passed +1 mvneclipse 0m 48s trunk passed +1 findbugs 2m 7s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 0s the patch passed +1 compile 14m 12s the patch passed +1 javac 14m 12s root generated 0 new + 776 unchanged - 1 fixed = 776 total (was 777) +1 checkstyle 1m 59s the patch passed +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 46s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 27s the patch passed +1 javadoc 1m 20s the patch passed -1 unit 7m 37s hadoop-common in the patch failed. +1 unit 0m 34s hadoop-aws in the patch passed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 101m 19s Reason Tests Failed junit tests hadoop.security.TestRaceWhenRelogin Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-14248 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860939/HADOOP-14248.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux cc86f2b4f7c0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 01aca54 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Steve Loughran for your comments.

          I think it's OK to remove this class in next major version, aka 2.x -> 3.x. It still merits a lot to keep this in 2.x branches in case people have started using it (that is a really meaningful improvement). We're kind enough to warn people about this in branch-2.9, where the AWS SDK got upgraded. branch-2.8 will stay as it is. I attach the branch-2 patch. Does it make sense?

          branch-2 patch was tested against us-west-1 on my laptop (using secret credentials) and AWS EC2 (using instance profile credentials).

          Show
          liuml07 Mingliang Liu added a comment - Thanks Steve Loughran for your comments. I think it's OK to remove this class in next major version, aka 2.x -> 3.x. It still merits a lot to keep this in 2.x branches in case people have started using it (that is a really meaningful improvement). We're kind enough to warn people about this in branch-2.9, where the AWS SDK got upgraded. branch-2.8 will stay as it is. I attach the branch-2 patch. Does it make sense? branch-2 patch was tested against us-west-1 on my laptop (using secret credentials) and AWS EC2 (using instance profile credentials).
          Hide
          stevel@apache.org Steve Loughran added a comment -

          We may want to retain a reference to the class as deprecated, or at least warn people in Hadoop 2.8.1 that this provider will be removed in future

          Show
          stevel@apache.org Steve Loughran added a comment - We may want to retain a reference to the class as deprecated, or at least warn people in Hadoop 2.8.1 that this provider will be removed in future
          Hide
          liuml07 Mingliang Liu added a comment -

          Tested against us-west-1 region on laptop (using secret credentials).

          Tested against us-west-1 region on Amazon EC2 VM (using instance profiles credentials).

          Show
          liuml07 Mingliang Liu added a comment - Tested against us-west-1 region on laptop (using secret credentials). Tested against us-west-1 region on Amazon EC2 VM (using instance profiles credentials).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 47s Maven dependency ordering for branch
          +1 mvninstall 12m 56s trunk passed
          +1 compile 16m 40s trunk passed
          +1 checkstyle 1m 54s trunk passed
          +1 mvnsite 1m 36s trunk passed
          +1 mvneclipse 0m 41s trunk passed
          +1 findbugs 2m 2s trunk passed
          +1 javadoc 1m 12s trunk passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 0m 58s the patch passed
          +1 compile 14m 57s the patch passed
          +1 javac 14m 57s root generated 0 new + 776 unchanged - 1 fixed = 776 total (was 777)
          +1 checkstyle 2m 5s the patch passed
          +1 mvnsite 1m 44s the patch passed
          +1 mvneclipse 0m 45s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 52s the patch passed
          +1 javadoc 1m 34s the patch passed
          -1 unit 9m 6s hadoop-common in the patch failed.
          +1 unit 0m 38s hadoop-aws in the patch passed.
          +1 asflicense 0m 42s The patch does not generate ASF License warnings.
          99m 23s



          Reason Tests
          Failed junit tests hadoop.security.TestKDiag



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-14248
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860769/HADOOP-14248.000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 20857e63cc04 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9bae672
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11949/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11949/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11949/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 47s Maven dependency ordering for branch +1 mvninstall 12m 56s trunk passed +1 compile 16m 40s trunk passed +1 checkstyle 1m 54s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 41s trunk passed +1 findbugs 2m 2s trunk passed +1 javadoc 1m 12s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 0m 58s the patch passed +1 compile 14m 57s the patch passed +1 javac 14m 57s root generated 0 new + 776 unchanged - 1 fixed = 776 total (was 777) +1 checkstyle 2m 5s the patch passed +1 mvnsite 1m 44s the patch passed +1 mvneclipse 0m 45s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 52s the patch passed +1 javadoc 1m 34s the patch passed -1 unit 9m 6s hadoop-common in the patch failed. +1 unit 0m 38s hadoop-aws in the patch passed. +1 asflicense 0m 42s The patch does not generate ASF License warnings. 99m 23s Reason Tests Failed junit tests hadoop.security.TestKDiag Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-14248 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860769/HADOOP-14248.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 20857e63cc04 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9bae672 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11949/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11949/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11949/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Steve Loughran and Chris Nauroth, can you have a look at this if you find time? Thanks,

          Show
          liuml07 Mingliang Liu added a comment - Steve Loughran and Chris Nauroth , can you have a look at this if you find time? Thanks,

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              liuml07 Mingliang Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development