Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      There's some code in HttpFS's server class that masks sensitive configuration values when they're being logged. We should update this to use ConfigRedactor (currently used for a similar purpose when refreshing configuration) so the list of sensitive configuration parameters can be managed in one place.

      The list also needs a few additions for HttpFS.

        Issue Links

          Activity

          Hide
          mackrorysd Sean Mackrory added a comment -

          Attaching a patch. Some keys were already redacted, so added those to ConfigRedactor, and started using that for redaction instead of maintaining a separate list. Also did some manual testing to check the log output...

          Show
          mackrorysd Sean Mackrory added a comment - Attaching a patch. Some keys were already redacted, so added those to ConfigRedactor, and started using that for redaction instead of maintaining a separate list. Also did some manual testing to check the log output...
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 6m 48s trunk passed
          +1 compile 6m 55s trunk passed
          +1 checkstyle 1m 29s trunk passed
          +1 mvnsite 1m 22s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 0m 56s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 4s the patch passed
          +1 compile 6m 51s the patch passed
          +1 javac 6m 51s the patch passed
          +1 checkstyle 1m 27s the patch passed
          +1 mvnsite 1m 20s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 0s the patch passed
          +1 javadoc 0m 56s the patch passed
          +1 unit 7m 16s hadoop-common in the patch passed.
          +1 unit 3m 2s hadoop-hdfs-httpfs in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          45m 57s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11080
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836613/HDFS-11080.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 15ba397151d4 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0dc2a6a
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17387/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-httpfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17387/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 48s trunk passed +1 compile 6m 55s trunk passed +1 checkstyle 1m 29s trunk passed +1 mvnsite 1m 22s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 56s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 4s the patch passed +1 compile 6m 51s the patch passed +1 javac 6m 51s the patch passed +1 checkstyle 1m 27s the patch passed +1 mvnsite 1m 20s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 0s the patch passed +1 javadoc 0m 56s the patch passed +1 unit 7m 16s hadoop-common in the patch passed. +1 unit 3m 2s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 45m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11080 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836613/HDFS-11080.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 15ba397151d4 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0dc2a6a Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17387/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-httpfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17387/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          LGTM +1 will commit shortly

          Show
          andrew.wang Andrew Wang added a comment - LGTM +1 will commit shortly
          Hide
          andrew.wang Andrew Wang added a comment -

          Committed! thanks Sean for the contribution.

          Show
          andrew.wang Andrew Wang added a comment - Committed! thanks Sean for the contribution.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10760 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10760/)
          HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean (wang: rev 7e521c5a49fbcf88285c102051ea2522edc847b9)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigRedactor.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/server/Server.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10760 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10760/ ) HDFS-11080 . Update HttpFS to use ConfigRedactor. Contributed by Sean (wang: rev 7e521c5a49fbcf88285c102051ea2522edc847b9) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigRedactor.java (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/server/Server.java

            People

            • Assignee:
              mackrorysd Sean Mackrory
              Reporter:
              mackrorysd Sean Mackrory
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development