Hadoop Common
  1. Hadoop Common
  2. HADOOP-8857

hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 0.23.7
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The docs and default.xml state that the secret is randomly generated if the secret.file is not present, this is incorrect as the secret must be shared across all nodes in the cluster as it is used to verify the signature of the hadoop.auth cookie. If randomly generated it would be diff in all nodes.

      ORIGINAL DESCRIPTION:

      AuthenticationFilterInitializer#initFilter fails if the configured hadoop.http.authentication.signature.secret.file does not exist, eg:

      java.lang.RuntimeException: Could not read HTTP signature secret file: /var/lib/hadoop-hdfs/hadoop-http-auth-signature-secret
      

      Creating /var/lib/hadoop-hdfs/hadoop-http-auth-signature-secret (populated with a string) fixes the issue. Per the auth docs "If a secret is not provided a random secret is generated at start up time.", which sounds like it means the file should be generated at startup with a random secrete, which doesn't seem to be the case. Also the instructions in the docs should be more clear in this regard.

      1. HADOOP-8857.patch
        2 kB
        Alejandro Abdelnur

        Activity

        Hide
        Eli Collins added a comment -

        Also, the documentation refers to the config as "signature.secret" not "signature.secret.file". It should also mention that it should be created on each host in the cluster that runs an auth-enabled Web UI.

        Show
        Eli Collins added a comment - Also, the documentation refers to the config as "signature.secret" not "signature.secret.file". It should also mention that it should be created on each host in the cluster that runs an auth-enabled Web UI.
        Hide
        Owen O'Malley added a comment -

        Actually, we need to:

        • remove the default value from core-default.xml
        • if the value isn't defined, let the lower layer generate random bytes

        Saving the random bytes lowers the security of the system.

        Show
        Owen O'Malley added a comment - Actually, we need to: remove the default value from core-default.xml if the value isn't defined, let the lower layer generate random bytes Saving the random bytes lowers the security of the system.
        Hide
        Eli Collins added a comment -

        We should be able to get away with that, IIUC the only reason to save the randomly generated bytes was for users that need to share the secret across hosts, but we can require they generate their own secret.

        Show
        Eli Collins added a comment - We should be able to get away with that, IIUC the only reason to save the randomly generated bytes was for users that need to share the secret across hosts, but we can require they generate their own secret.
        Hide
        Alejandro Abdelnur added a comment -

        The secret cannot be generated randomly in the case of a cluster as the secret must be shared by ALL machines. We have to update the docs removing the 'generated randomly' comment.

        Show
        Alejandro Abdelnur added a comment - The secret cannot be generated randomly in the case of a cluster as the secret must be shared by ALL machines. We have to update the docs removing the 'generated randomly' comment.
        Hide
        Alejandro Abdelnur added a comment -

        patch updating docs and default.xml

        Show
        Alejandro Abdelnur added a comment - patch updating docs and default.xml
        Hide
        Aaron T. Myers added a comment -

        +1, this change looks good to me. I agree with Tucu that it's not reasonable to generate this value randomly given that this cookie will need to be validated by all of the hosts in the cluster.

        Show
        Aaron T. Myers added a comment - +1, this change looks good to me. I agree with Tucu that it's not reasonable to generate this value randomly given that this cookie will need to be validated by all of the hosts in the cluster.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12566424/HADOOP-8857.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2090//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2090//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566424/HADOOP-8857.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2090//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2090//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        Committed to trunk and branch-2.

        Show
        Alejandro Abdelnur added a comment - Committed to trunk and branch-2.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3280 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3280/)
        HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3280 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3280/ ) HADOOP-8857 . hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #108 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/108/)
        HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #108 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/108/ ) HADOOP-8857 . hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1297 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1297/)
        HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1297 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1297/ ) HADOOP-8857 . hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1325 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1325/)
        HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1325 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1325/ ) HADOOP-8857 . hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1438601) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438601 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/)
        HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1441567)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441567
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/ ) HADOOP-8857 . hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu) (Revision 1441567) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441567 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/apt/HttpAuthentication.apt.vm

          People

          • Assignee:
            Alejandro Abdelnur
            Reporter:
            Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development