Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12841

Update s3-related properties in core-default.xml

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Tags:
      s3

      Description

      HADOOP-11670 deprecated fs.s3a.awsAccessKeyId/fs.s3a.awsSecretAccessKey in favor of fs.s3a.access.key/fs.s3a.secret.key in the code, but did not update core-default.xml. Also, a few S3 related properties are missing.

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev01: based on https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html,
        I added a few missing properties for S3/S3A/S3N access key and secret key.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev01: based on https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html , I added a few missing properties for S3/S3A/S3N access key and secret key.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 43s trunk passed
        +1 compile 5m 44s trunk passed with JDK v1.8.0_72
        +1 compile 6m 43s trunk passed with JDK v1.7.0_95
        +1 mvnsite 1m 5s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 javadoc 0m 52s trunk passed with JDK v1.8.0_72
        +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 40s the patch passed
        +1 compile 5m 33s the patch passed with JDK v1.8.0_72
        +1 javac 5m 33s the patch passed
        +1 compile 6m 30s the patch passed with JDK v1.7.0_95
        +1 javac 6m 30s the patch passed
        +1 mvnsite 1m 1s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 javadoc 0m 50s the patch passed with JDK v1.8.0_72
        +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95
        +1 unit 7m 7s hadoop-common in the patch passed with JDK v1.8.0_72.
        -1 unit 6m 45s hadoop-common in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        53m 35s



        Reason Tests
        JDK v1.7.0_95 Failed junit tests hadoop.fs.TestSymlinkLocalFSFileSystem



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12789981/HADOOP-12841.001.patch
        JIRA Issue HADOOP-12841
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
        uname Linux bdbad3c7f662 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8808779
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 43s trunk passed +1 compile 5m 44s trunk passed with JDK v1.8.0_72 +1 compile 6m 43s trunk passed with JDK v1.7.0_95 +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 14s trunk passed +1 javadoc 0m 52s trunk passed with JDK v1.8.0_72 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 5m 33s the patch passed with JDK v1.8.0_72 +1 javac 5m 33s the patch passed +1 compile 6m 30s the patch passed with JDK v1.7.0_95 +1 javac 6m 30s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 50s the patch passed with JDK v1.8.0_72 +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95 +1 unit 7m 7s hadoop-common in the patch passed with JDK v1.8.0_72. -1 unit 6m 45s hadoop-common in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 53m 35s Reason Tests JDK v1.7.0_95 Failed junit tests hadoop.fs.TestSymlinkLocalFSFileSystem Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12789981/HADOOP-12841.001.patch JIRA Issue HADOOP-12841 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux bdbad3c7f662 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8808779 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8721/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        test failures are unrelated.

        Show
        jojochuang Wei-Chiu Chuang added a comment - test failures are unrelated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        +1. LGTM. Committed to trunk and branch-2.

        The test failure are not related, this is a document change.

        Thanks a lot for the work, Wei-Chiu Chuang.

        Show
        eddyxu Lei (Eddy) Xu added a comment - +1. LGTM. Committed to trunk and branch-2. The test failure are not related, this is a document change. Thanks a lot for the work, Wei-Chiu Chuang .
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for the review and commit!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for the review and commit!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9375 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9375/)
        HADOOP-12841. Update s3-related properties in core-default.xml. (lei: rev 2093acf6b659d5a271b7e97f9b64652d7cf01eef)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9375 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9375/ ) HADOOP-12841 . Update s3-related properties in core-default.xml. (lei: rev 2093acf6b659d5a271b7e97f9b64652d7cf01eef) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Lei (Eddy) Xu per HADOOP-13025, should we also commit this in 2.7.x and 2.8?

        Show
        jojochuang Wei-Chiu Chuang added a comment - Lei (Eddy) Xu per HADOOP-13025 , should we also commit this in 2.7.x and 2.8?

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development