Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11694 Über-jira: S3a phase II: robustness, scale and performance
  3. HADOOP-13727

S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: fs/s3
    • Labels:
      None

      Description

      When running in an EC2 VM, S3A can make use of InstanceProfileCredentialsProvider from the AWS SDK to obtain credentials from the EC2 Instance Metadata Service. We have observed that for a highly multi-threaded application, this may generate a high number of calls to the Instance Metadata Service. The service may throttle the client by replying with an HTTP 429 response or forcibly closing connections. We can greatly reduce the number of calls to the service by enforcing that all threads use a single shared instance of InstanceProfileCredentialsProvider.

      1. HADOOP-13727-branch-2.001.patch
        36 kB
        Chris Nauroth
      2. HADOOP-13727-branch-2.002.patch
        37 kB
        Chris Nauroth
      3. HADOOP-13727-branch-2.003.patch
        48 kB
        Chris Nauroth
      4. HADOOP-13727-branch-2.004.patch
        48 kB
        Chris Nauroth
      5. HADOOP-13727-branch-2.005.patch
        48 kB
        Chris Nauroth

        Issue Links

          Activity

          Hide
          cnauroth Chris Nauroth added a comment -

          Thank you to Rajesh Balamohan for reporting this.

          Show
          cnauroth Chris Nauroth added a comment - Thank you to Rajesh Balamohan for reporting this.
          Hide
          cnauroth Chris Nauroth added a comment -

          I'm attaching patch 001. We have confirmed through load testing in our EC2 environment that this patch prevents the throttling problems we had seen. I also have completed a full S3A test run against US-west-2.

          • Define SharedInstanceProfileCredentialsProvider as a subclass of InstanceProfileCredentialsProvider, which enforces creation of only a single instance.
          • Change credential provider creation logic in S3AUtils to support use of the shared instance, both in the default case and the case that the user has configured fs.s3a.aws.credentials.provider.
          • Also change the logic of S3AUtils for more edge case validation, better error messages and better readability (I hope).
          • Update site documentation and core-default.xml to describe the new provider.
          • Set up a new unit test suite, TestS3AAWSCredentialsProvider. There were multiple tests from ITestS3AAWSCredentialsProvider that didn't really need full S3 integration, so I've moved them to the new unit test suite. Now they'll run in pre-commit. I also added new tests for the new functionality and new validation logic.

          As of AWS SDK 1.11.39, the SDK code internally enforces a singleton. After Hadoop upgrades to that version or higher, it's likely that we can remove this code.

          Also, I have proposed a change to the FileSystem cache logic in HADOOP-13726 that would have prevented this from surfacing. That's going to be a much riskier change, so I'd still like to proceed with the S3A change here.

          Show
          cnauroth Chris Nauroth added a comment - I'm attaching patch 001. We have confirmed through load testing in our EC2 environment that this patch prevents the throttling problems we had seen. I also have completed a full S3A test run against US-west-2. Define SharedInstanceProfileCredentialsProvider as a subclass of InstanceProfileCredentialsProvider , which enforces creation of only a single instance. Change credential provider creation logic in S3AUtils to support use of the shared instance, both in the default case and the case that the user has configured fs.s3a.aws.credentials.provider . Also change the logic of S3AUtils for more edge case validation, better error messages and better readability (I hope). Update site documentation and core-default.xml to describe the new provider. Set up a new unit test suite, TestS3AAWSCredentialsProvider . There were multiple tests from ITestS3AAWSCredentialsProvider that didn't really need full S3 integration, so I've moved them to the new unit test suite. Now they'll run in pre-commit. I also added new tests for the new functionality and new validation logic. As of AWS SDK 1.11.39, the SDK code internally enforces a singleton. After Hadoop upgrades to that version or higher, it's likely that we can remove this code. Also, I have proposed a change to the FileSystem cache logic in HADOOP-13726 that would have prevented this from surfacing. That's going to be a much riskier change, so I'd still like to proceed with the S3A change here.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 55s Maven dependency ordering for branch
          +1 mvninstall 7m 26s branch-2 passed
          +1 compile 6m 42s branch-2 passed with JDK v1.8.0_101
          +1 compile 7m 15s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 35s branch-2 passed
          +1 mvnsite 1m 33s branch-2 passed
          +1 mvneclipse 0m 33s branch-2 passed
          +1 findbugs 2m 32s branch-2 passed
          +1 javadoc 1m 8s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 23s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 9s the patch passed
          +1 compile 6m 27s the patch passed with JDK v1.8.0_101
          +1 javac 6m 27s the patch passed
          +1 compile 7m 22s the patch passed with JDK v1.7.0_111
          +1 javac 7m 22s the patch passed
          +1 checkstyle 1m 35s root: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3)
          +1 mvnsite 1m 37s the patch passed
          +1 mvneclipse 0m 37s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 48s the patch passed
          +1 javadoc 1m 8s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 23s the patch passed with JDK v1.7.0_111
          +1 unit 8m 27s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          99m 33s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13727
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833826/HADOOP-13727-branch-2.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 04cf4a9ae651 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / e02c756
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10816/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10816/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10816/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 55s Maven dependency ordering for branch +1 mvninstall 7m 26s branch-2 passed +1 compile 6m 42s branch-2 passed with JDK v1.8.0_101 +1 compile 7m 15s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 35s branch-2 passed +1 mvnsite 1m 33s branch-2 passed +1 mvneclipse 0m 33s branch-2 passed +1 findbugs 2m 32s branch-2 passed +1 javadoc 1m 8s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 23s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 9s the patch passed +1 compile 6m 27s the patch passed with JDK v1.8.0_101 +1 javac 6m 27s the patch passed +1 compile 7m 22s the patch passed with JDK v1.7.0_111 +1 javac 7m 22s the patch passed +1 checkstyle 1m 35s root: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) +1 mvnsite 1m 37s the patch passed +1 mvneclipse 0m 37s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 48s the patch passed +1 javadoc 1m 8s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 23s the patch passed with JDK v1.7.0_111 +1 unit 8m 27s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 99m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13727 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833826/HADOOP-13727-branch-2.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 04cf4a9ae651 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / e02c756 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10816/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10816/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10816/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          LGTM

          Could you have that 1) 2) 3) list in both the docs and md formatted for each. that is <ul> for the javadocs and 1. 2. for markdown?

          Show
          stevel@apache.org Steve Loughran added a comment - LGTM Could you have that 1) 2) 3) list in both the docs and md formatted for each. that is <ul> for the javadocs and 1. 2. for markdown?
          Hide
          cnauroth Chris Nauroth added a comment -

          Thank you, Steve. I am attaching revision 002 with your suggestions. I used <ol> for the JavaDocs, because the ordering is significant.

          Show
          cnauroth Chris Nauroth added a comment - Thank you, Steve. I am attaching revision 002 with your suggestions. I used <ol> for the JavaDocs, because the ordering is significant.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 33s Maven dependency ordering for branch
          +1 mvninstall 6m 33s branch-2 passed
          +1 compile 5m 35s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 31s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 23s branch-2 passed
          +1 mvnsite 1m 23s branch-2 passed
          +1 mvneclipse 0m 32s branch-2 passed
          +1 findbugs 2m 16s branch-2 passed
          +1 javadoc 1m 3s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 16s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 3s the patch passed
          +1 compile 5m 31s the patch passed with JDK v1.8.0_101
          +1 javac 5m 31s the patch passed
          +1 compile 6m 34s the patch passed with JDK v1.7.0_111
          +1 javac 6m 34s the patch passed
          +1 checkstyle 1m 26s root: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3)
          +1 mvnsite 1m 29s the patch passed
          +1 mvneclipse 0m 39s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 56s the patch passed
          +1 javadoc 1m 16s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 21s the patch passed with JDK v1.7.0_111
          +1 unit 8m 38s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          94m 31s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13727
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833993/HADOOP-13727-branch-2.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux f0ef52c2cc2b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 3972bb3
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10823/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10823/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10823/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 6m 33s branch-2 passed +1 compile 5m 35s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 31s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 23s branch-2 passed +1 mvnsite 1m 23s branch-2 passed +1 mvneclipse 0m 32s branch-2 passed +1 findbugs 2m 16s branch-2 passed +1 javadoc 1m 3s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 16s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 5m 31s the patch passed with JDK v1.8.0_101 +1 javac 5m 31s the patch passed +1 compile 6m 34s the patch passed with JDK v1.7.0_111 +1 javac 6m 34s the patch passed +1 checkstyle 1m 26s root: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) +1 mvnsite 1m 29s the patch passed +1 mvneclipse 0m 39s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 56s the patch passed +1 javadoc 1m 16s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 21s the patch passed with JDK v1.7.0_111 +1 unit 8m 38s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 94m 31s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13727 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833993/HADOOP-13727-branch-2.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux f0ef52c2cc2b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 3972bb3 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10823/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10823/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10823/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          It's irritating to get whitespace warnings every time a patch touches core-default.xml, so here is revision 003 to clean up all of the remaining whitespace problems. If people don't want to +1 this revision because of touching unrelated lines of code, then I understand that, and I'll separate it to a different JIRA.

          Show
          cnauroth Chris Nauroth added a comment - It's irritating to get whitespace warnings every time a patch touches core-default.xml, so here is revision 003 to clean up all of the remaining whitespace problems. If people don't want to +1 this revision because of touching unrelated lines of code, then I understand that, and I'll separate it to a different JIRA.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 57s Maven dependency ordering for branch
          +1 mvninstall 7m 0s branch-2 passed
          +1 compile 6m 30s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 59s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 31s branch-2 passed
          +1 mvnsite 1m 28s branch-2 passed
          +1 mvneclipse 0m 31s branch-2 passed
          +1 findbugs 2m 22s branch-2 passed
          +1 javadoc 1m 6s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 17s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 4s the patch passed
          +1 compile 6m 4s the patch passed with JDK v1.8.0_101
          +1 javac 6m 4s the patch passed
          +1 compile 8m 8s the patch passed with JDK v1.7.0_111
          +1 javac 8m 8s the patch passed
          +1 checkstyle 1m 45s root: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3)
          +1 mvnsite 1m 45s the patch passed
          +1 mvneclipse 0m 39s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 14s the patch passed
          +1 javadoc 1m 19s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 30s the patch passed with JDK v1.7.0_111
          +1 unit 11m 18s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 31s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          104m 52s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13727
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834020/HADOOP-13727-branch-2.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 21eec51baace 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 3972bb3
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10825/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10825/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10825/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 57s Maven dependency ordering for branch +1 mvninstall 7m 0s branch-2 passed +1 compile 6m 30s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 59s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 31s branch-2 passed +1 mvnsite 1m 28s branch-2 passed +1 mvneclipse 0m 31s branch-2 passed +1 findbugs 2m 22s branch-2 passed +1 javadoc 1m 6s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 17s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 4s the patch passed +1 compile 6m 4s the patch passed with JDK v1.8.0_101 +1 javac 6m 4s the patch passed +1 compile 8m 8s the patch passed with JDK v1.7.0_111 +1 javac 8m 8s the patch passed +1 checkstyle 1m 45s root: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) +1 mvnsite 1m 45s the patch passed +1 mvneclipse 0m 39s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 14s the patch passed +1 javadoc 1m 19s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 30s the patch passed with JDK v1.7.0_111 +1 unit 11m 18s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 31s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 104m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13727 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834020/HADOOP-13727-branch-2.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 21eec51baace 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 3972bb3 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10825/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10825/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10825/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Patch 003 really should fix the whitespace warnings that were flagged in the last pre-commit run. I confirmed by running the same check that Yetus runs, and I didn't find any whitespace endings remaining in the file.

          grep -n -I --extended-regexp '[[:blank:]]$' ~/git/hadoop/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          

          I do need to rebase this though after the commit of HADOOP-13560.

          Show
          cnauroth Chris Nauroth added a comment - Patch 003 really should fix the whitespace warnings that were flagged in the last pre-commit run. I confirmed by running the same check that Yetus runs, and I didn't find any whitespace endings remaining in the file. grep -n -I --extended-regexp '[[:blank:]]$' ~/git/hadoop/hadoop-common-project/hadoop-common/src/main/resources/core- default .xml I do need to rebase this though after the commit of HADOOP-13560 .
          Hide
          cnauroth Chris Nauroth added a comment -

          I'm attaching revision 004, which is the same thing rebased against current branch-2.

          Show
          cnauroth Chris Nauroth added a comment - I'm attaching revision 004, which is the same thing rebased against current branch-2.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 39s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 55s Maven dependency ordering for branch
          +1 mvninstall 7m 16s branch-2 passed
          +1 compile 5m 26s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 25s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 26s branch-2 passed
          +1 mvnsite 1m 19s branch-2 passed
          +1 mvneclipse 0m 31s branch-2 passed
          +1 findbugs 2m 14s branch-2 passed
          +1 javadoc 1m 0s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 14s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 0s the patch passed
          +1 compile 5m 26s the patch passed with JDK v1.8.0_101
          +1 javac 5m 26s the patch passed
          +1 compile 6m 30s the patch passed with JDK v1.7.0_111
          +1 javac 6m 30s the patch passed
          +1 checkstyle 1m 26s root: The patch generated 0 new + 4 unchanged - 3 fixed = 4 total (was 7)
          +1 mvnsite 1m 27s the patch passed
          +1 mvneclipse 0m 36s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 52s the patch passed
          +1 javadoc 1m 9s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 24s the patch passed with JDK v1.7.0_111
          +1 unit 7m 23s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          104m 40s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13727
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834228/HADOOP-13727-branch-2.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 0ed8b555eaad 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ad7d3c4
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10828/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10828/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10828/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 39s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 55s Maven dependency ordering for branch +1 mvninstall 7m 16s branch-2 passed +1 compile 5m 26s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 25s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 26s branch-2 passed +1 mvnsite 1m 19s branch-2 passed +1 mvneclipse 0m 31s branch-2 passed +1 findbugs 2m 14s branch-2 passed +1 javadoc 1m 0s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 14s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 0s the patch passed +1 compile 5m 26s the patch passed with JDK v1.8.0_101 +1 javac 5m 26s the patch passed +1 compile 6m 30s the patch passed with JDK v1.7.0_111 +1 javac 6m 30s the patch passed +1 checkstyle 1m 26s root: The patch generated 0 new + 4 unchanged - 3 fixed = 4 total (was 7) +1 mvnsite 1m 27s the patch passed +1 mvneclipse 0m 36s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 52s the patch passed +1 javadoc 1m 9s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 24s the patch passed with JDK v1.7.0_111 +1 unit 7m 23s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 104m 40s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13727 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834228/HADOOP-13727-branch-2.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 0ed8b555eaad 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ad7d3c4 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10828/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10828/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10828/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          LGTM, +1, pending

          1. it passing a full test run against your endpoint
          2. you doing a look at the generated aws site page (or a github/markdown editor) to make sure the 1. 2 ... bullet points are pulled out. I think you may need an extra line.

          Rajesh or someone running on EC2 will presumably need to check that it works too —we aren't testing that, are we? (though presumably you could run the integration tests using IAM to pick up the credentials —has someone done that?)

          Show
          stevel@apache.org Steve Loughran added a comment - LGTM, +1, pending it passing a full test run against your endpoint you doing a look at the generated aws site page (or a github/markdown editor) to make sure the 1. 2 ... bullet points are pulled out. I think you may need an extra line. Rajesh or someone running on EC2 will presumably need to check that it works too —we aren't testing that, are we? (though presumably you could run the integration tests using IAM to pick up the credentials —has someone done that?)
          Hide
          rajesh.balamohan Rajesh Balamohan added a comment -

          I have tried out this patch in AWS test environment and fixes the issue. Are you referring to running entire test suite in aws ec2?.

          Show
          rajesh.balamohan Rajesh Balamohan added a comment - I have tried out this patch in AWS test environment and fixes the issue. Are you referring to running entire test suite in aws ec2?.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Ideally it'd be good to do the whole test suite to verify that IAM works across the lot, however I think we can take the fact that your test suite, which does use IAM, scaled up, is implicitly a sign that the auth mechanism works

          Show
          stevel@apache.org Steve Loughran added a comment - Ideally it'd be good to do the whole test suite to verify that IAM works across the lot, however I think we can take the fact that your test suite, which does use IAM, scaled up, is implicitly a sign that the auth mechanism works
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          Show
          stevel@apache.org Steve Loughran added a comment - +1
          Hide
          stevel@apache.org Steve Loughran added a comment -

          actually, not applying to branch-2..can you sync it up?

          Show
          stevel@apache.org Steve Loughran added a comment - actually, not applying to branch-2..can you sync it up?
          Hide
          cnauroth Chris Nauroth added a comment -

          I'm attaching patch revision 005, rebasing to current.

          it passing a full test run against your endpoint

          The rebased patch applied to branch-2 passes a full test run against us-west-2.

          you doing a look at the generated aws site page (or a github/markdown editor) to make sure the 1. 2 ... bullet points are pulled out. I think you may need an extra line.

          Everything looks to be rendering nicely. These are actually part of Markdown code blocks that repeat snippets of core-site.xml. Thanks for being watchful of the documentation though.

          Ideally it'd be good to do the whole test suite to verify that IAM works across the lot...

          For this last test run, I relied on instance profile credentials for all of the S3A tests. FWIW, I now use instance profile credentials pretty regularly for my test runs of all patches I review.

          With the rebase and addressing the above, all feedback has been resolved. I plan to commit based on Steve's +1 after a fresh pre-commit run.

          Show
          cnauroth Chris Nauroth added a comment - I'm attaching patch revision 005, rebasing to current. it passing a full test run against your endpoint The rebased patch applied to branch-2 passes a full test run against us-west-2. you doing a look at the generated aws site page (or a github/markdown editor) to make sure the 1. 2 ... bullet points are pulled out. I think you may need an extra line. Everything looks to be rendering nicely. These are actually part of Markdown code blocks that repeat snippets of core-site.xml. Thanks for being watchful of the documentation though. Ideally it'd be good to do the whole test suite to verify that IAM works across the lot... For this last test run, I relied on instance profile credentials for all of the S3A tests. FWIW, I now use instance profile credentials pretty regularly for my test runs of all patches I review. With the rebase and addressing the above, all feedback has been resolved. I plan to commit based on Steve's +1 after a fresh pre-commit run.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 57s Maven dependency ordering for branch
          +1 mvninstall 6m 40s branch-2 passed
          +1 compile 5m 32s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 32s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 27s branch-2 passed
          +1 mvnsite 1m 23s branch-2 passed
          +1 mvneclipse 1m 20s branch-2 passed
          +1 findbugs 2m 17s branch-2 passed
          +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 15s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 3s the patch passed
          +1 compile 5m 29s the patch passed with JDK v1.8.0_101
          +1 javac 5m 29s the patch passed
          +1 compile 6m 31s the patch passed with JDK v1.7.0_111
          +1 javac 6m 31s the patch passed
          +1 checkstyle 1m 26s root: The patch generated 0 new + 4 unchanged - 3 fixed = 4 total (was 7)
          +1 mvnsite 1m 27s the patch passed
          +1 mvneclipse 0m 37s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 47s the patch passed
          +1 javadoc 1m 6s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 21s the patch passed with JDK v1.7.0_111
          +1 unit 7m 28s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          91m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13727
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835041/HADOOP-13727-branch-2.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 8fbb2c0fe7cb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 086577c
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10883/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10883/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10883/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 57s Maven dependency ordering for branch +1 mvninstall 6m 40s branch-2 passed +1 compile 5m 32s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 32s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 27s branch-2 passed +1 mvnsite 1m 23s branch-2 passed +1 mvneclipse 1m 20s branch-2 passed +1 findbugs 2m 17s branch-2 passed +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 15s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 5m 29s the patch passed with JDK v1.8.0_101 +1 javac 5m 29s the patch passed +1 compile 6m 31s the patch passed with JDK v1.7.0_111 +1 javac 6m 31s the patch passed +1 checkstyle 1m 26s root: The patch generated 0 new + 4 unchanged - 3 fixed = 4 total (was 7) +1 mvnsite 1m 27s the patch passed +1 mvneclipse 0m 37s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 47s the patch passed +1 javadoc 1m 6s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 21s the patch passed with JDK v1.7.0_111 +1 unit 7m 28s hadoop-common in the patch passed with JDK v1.7.0_111. +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 91m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13727 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835041/HADOOP-13727-branch-2.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 8fbb2c0fe7cb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 086577c Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10883/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10883/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10883/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          I have committed this to trunk, branch-2 and branch-2.8. Rajesh, thank you for reporting the issue and testing the patch. Steve, thank you for your code review.

          Show
          cnauroth Chris Nauroth added a comment - I have committed this to trunk, branch-2 and branch-2.8. Rajesh, thank you for reporting the issue and testing the patch. Steve, thank you for your code review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10670 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10670/)
          HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance (cnauroth: rev d8fa1cfa6722cbf7a4ec3d6b9c44b034da9aa351)

          • (add) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceProfileCredentialsProvider.java
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/AWSCredentialProviderList.java
          • (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
          • (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java
          • (add) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10670 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10670/ ) HADOOP-13727 . S3A: Reduce high number of connections to EC2 Instance (cnauroth: rev d8fa1cfa6722cbf7a4ec3d6b9c44b034da9aa351) (add) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceProfileCredentialsProvider.java (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/AWSCredentialProviderList.java (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java (add) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java

            People

            • Assignee:
              cnauroth Chris Nauroth
              Reporter:
              rajesh.balamohan Rajesh Balamohan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development