Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14229

hadoop.security.auth_to_local example is incorrect in the documentation

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-beta1
    • Component/s: None
    • Labels:
      None

      Description

      Let's see jhs as example:

      RULE:[2:$1@$0](jhs/.*@.*REALM.TLD)s/.*/mapred/

      That means principal has 2 components (jhs/myhost@REALM).
      The second column converts this to jhs@REALM. So the regex will not match on this since regex expects / in the principal.

      My suggestion is

      RULE:[2:$1](jhs)s/.*/mapred/

      https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html

      1. HADOOP-14229.01.patch
        1 kB
        Andras Bokor
      2. HADOOP-14229.02.patch
        1.0 kB
        Andras Bokor
      3. HADOOP-14229.03.patch
        1 kB
        Andras Bokor

        Issue Links

          Activity

          Hide
          boky01 Andras Bokor added a comment -

          Thanks Ravi Prakash!

          Show
          boky01 Andras Bokor added a comment - Thanks Ravi Prakash !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12070 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12070/)
          HADOOP-14229. hadoop.security.auth_to_local example is incorrect in the (raviprak: rev 746189ad8cdf90ab35baec9364b2e02956a1e70c)

          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12070 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12070/ ) HADOOP-14229 . hadoop.security.auth_to_local example is incorrect in the (raviprak: rev 746189ad8cdf90ab35baec9364b2e02956a1e70c) (edit) hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
          Hide
          raviprak Ravi Prakash added a comment -

          Committed to trunk.

          Show
          raviprak Ravi Prakash added a comment - Committed to trunk.
          Hide
          raviprak Ravi Prakash added a comment -

          Looks good to me. +1. Committing shortly. Thank you for the contribution Andras!

          Show
          raviprak Ravi Prakash added a comment - Looks good to me. +1. Committing shortly. Thank you for the contribution Andras!
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 13m 12s trunk passed
          +1 mvnsite 1m 18s trunk passed
          +1 mvnsite 0m 50s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 14s The patch does not generate ASF License warnings.
          16m 7s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14229
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12866661/HADOOP-14229.03.patch
          Optional Tests asflicense mvnsite
          uname Linux 80ec0aec2f3b 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e4f34ec
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12251/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 13m 12s trunk passed +1 mvnsite 1m 18s trunk passed +1 mvnsite 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 14s The patch does not generate ASF License warnings. 16m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14229 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12866661/HADOOP-14229.03.patch Optional Tests asflicense mvnsite uname Linux 80ec0aec2f3b 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e4f34ec modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12251/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          boky01 Andras Bokor added a comment -

          Reattach last patch to kick Hadoop QA

          Show
          boky01 Andras Bokor added a comment - Reattach last patch to kick Hadoop QA
          Hide
          boky01 Andras Bokor added a comment -

          Allen Wittenauer,

          That is true. Skipping FQDN causes super group access between clusters (in the same realm).
          I think here we are not intended to provide a production ready mapping it should just be an example showing what is the basic syntax.
          Anyway, your comment made me feel I should provide a bit more complex (but not too complex) example to show the syntax of auth_to_local rules.
          HADOOP-14229.03.patch

          Show
          boky01 Andras Bokor added a comment - Allen Wittenauer , That is true. Skipping FQDN causes super group access between clusters (in the same realm). I think here we are not intended to provide a production ready mapping it should just be an example showing what is the basic syntax. Anyway, your comment made me feel I should provide a bit more complex (but not too complex) example to show the syntax of auth_to_local rules. HADOOP-14229.03.patch
          Hide
          aw Allen Wittenauer added a comment -

          I'm going to +1 with the caveat that this makes the docs a little more clear, but doesn't really solve a key problem:

          $ bin/hadoop kerbname nn/host2.domain@REALM.TLD
          Name: nn/host2.domain@REALM.TLD to hdfs
          

          This is sort of hinted at in the docs:

          The default rule maps the principal host/full.qualified.domain.name@REALM.TLD to system user host. The default rule will not be appropriate for most clusters.
          

          It then goes on to provide the example rule which doesn't actually fix that warning and all clusters still have super user access on every other cluster in the same realm. At which point it becomes clear the documentation is mostly an exercise in obfuscation. You're better off just using hdfs/, yarn/, etc for daemons and avoid all this mapping baloney anyway (which is what most people that I know of do).

          Show
          aw Allen Wittenauer added a comment - I'm going to +1 with the caveat that this makes the docs a little more clear, but doesn't really solve a key problem: $ bin/hadoop kerbname nn/host2.domain@REALM.TLD Name: nn/host2.domain@REALM.TLD to hdfs This is sort of hinted at in the docs: The default rule maps the principal host/full.qualified.domain.name@REALM.TLD to system user host. The default rule will not be appropriate for most clusters. It then goes on to provide the example rule which doesn't actually fix that warning and all clusters still have super user access on every other cluster in the same realm. At which point it becomes clear the documentation is mostly an exercise in obfuscation. You're better off just using hdfs/, yarn/, etc for daemons and avoid all this mapping baloney anyway (which is what most people that I know of do).
          Hide
          hanishakoneru Hanisha Koneru added a comment -

          Andras Bokor,
          Verified that the current suggested settings for hadoop.security.auth_to_local in https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html do not perform the intended action.
          As you said, the command hadoop kerbname jhs/host.domain@REALM.TLD gives the following result:

          Name: jhs/host.domain@REALM.TLD to jhs/host.domain@REALM.TLD

          whereas, the intended result is:

          Name: jhs/host.domain@REALM.TLD to mapred

          The patch LGTM.

          Show
          hanishakoneru Hanisha Koneru added a comment - Andras Bokor , Verified that the current suggested settings for hadoop.security.auth_to_local in https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html do not perform the intended action. As you said, the command hadoop kerbname jhs/host.domain@REALM.TLD gives the following result: Name: jhs/host.domain@REALM.TLD to jhs/host.domain@REALM.TLD whereas, the intended result is: Name: jhs/host.domain@REALM.TLD to mapred The patch LGTM.
          Hide
          boky01 Andras Bokor added a comment -

          Patch 02:
          Make it more compact
          Test results:

          hadoop-3.0.0-alpha2/bin/hadoop kerbname {nn,dn,jn,rm,nm,jhs}/host.domain@REALM.TLD
          Name: nn/host.domain@REALM.TLD to hdfs
          Name: dn/host.domain@REALM.TLD to hdfs
          Name: jn/host.domain@REALM.TLD to hdfs
          Name: rm/host.domain@REALM.TLD to yarn
          Name: nm/host.domain@REALM.TLD to yarn
          Name: jhs/host.domain@REALM.TLD to mapred
          Show
          boky01 Andras Bokor added a comment - Patch 02: Make it more compact Test results: hadoop-3.0.0-alpha2/bin/hadoop kerbname {nn,dn,jn,rm,nm,jhs}/host.domain@REALM.TLD Name: nn/host.domain@REALM.TLD to hdfs Name: dn/host.domain@REALM.TLD to hdfs Name: jn/host.domain@REALM.TLD to hdfs Name: rm/host.domain@REALM.TLD to yarn Name: nm/host.domain@REALM.TLD to yarn Name: jhs/host.domain@REALM.TLD to mapred
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 13m 55s trunk passed
          +1 mvnsite 1m 6s trunk passed
          +1 mvnsite 1m 1s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          17m 0s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-14229
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860355/HADOOP-14229.01.patch
          Optional Tests asflicense mvnsite
          uname Linux cc418ca36696 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ab759e9
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11914/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 13m 55s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvnsite 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 17m 0s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-14229 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860355/HADOOP-14229.01.patch Optional Tests asflicense mvnsite uname Linux cc418ca36696 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ab759e9 modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11914/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          boky01 Andras Bokor added a comment -

          Arpit Agarwal,

          Could you please check my patch?

          Show
          boky01 Andras Bokor added a comment - Arpit Agarwal , Could you please check my patch?

            People

            • Assignee:
              boky01 Andras Bokor
              Reporter:
              boky01 Andras Bokor
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development