Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7546

Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern

        Details

        • Target Version/s:

          Description

          This config is used in the SaslRpcClient, and the no-default breaks cross-realm trust principals being used at clients.

          Current location: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java#L309

          The config should be documented and the default should be set to * to preserve the prior-to-introduction behaviour.

          1. Text File
            HDFS-7546.addendum.001.patch
            0.9 kB
            Yongjun Zhang
          2. Text File
            HDFS-7546.patch
            0.7 kB
            Harsh J

            Issue Links

            relates to

            New Feature - A new feature of the product, which has yet to be developed. HADOOP-9789 Support server advertised kerberos principals

            • Critical - Crashes, loss of data, severe memory leak.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              aw Allen Wittenauer added a comment -

              Is it just namenode or is it any service that has Kerberos configured?

              Show
              aw Allen Wittenauer added a comment - Is it just namenode or is it any service that has Kerberos configured?
              Hide
              Permalink
              yzhangal Yongjun Zhang added a comment -

              Hi Harsh J,

              Thanks for reporting the issue and providing patch. I labeled it as "supportability". I reviewed the change and have a few comments.

              • The description of the property can be improved with more information. What about: 
                A client-side property that describes permitted server principal pattern. It can be configured
to control allowed realms to authenticate with, which is useful in cross-realm environment.
              • what's the current default of this property prior to your change?
              • wonder if there is any catch by changing the default pattern to "*", which essentially accepts any pattern?
              Show
              yzhangal Yongjun Zhang added a comment - Hi Harsh J , Thanks for reporting the issue and providing patch. I labeled it as "supportability". I reviewed the change and have a few comments. The description of the property can be improved with more information. What about: A client-side property that describes permitted server principal pattern. It can be configured to control allowed realms to authenticate with, which is useful in cross-realm environment. what's the current default of this property prior to your change? wonder if there is any catch by changing the default pattern to "*", which essentially accepts any pattern?
              Hide
              Permalink
              hadoopqa Hadoop QA added a comment -

              -1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12688010/HDFS-7546.patch
              against trunk revision 1050d42.

              +1 @author. The patch does not contain any @author tags.

              -1 tests included. The patch doesn't appear to include any new or modified tests.
              Please justify why no new tests are needed for this patch.
              Also please list what manual steps were performed to verify this patch.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 javadoc. There were no new javadoc warning messages.

              +1 eclipse:eclipse. The patch built with eclipse:eclipse.

              +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

              -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

              org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
              org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration

              Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9072//testReport/
              Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9072//console

              This message is automatically generated.

              Show
              hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12688010/HDFS-7546.patch against trunk revision 1050d42. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9072//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9072//console This message is automatically generated.
              Hide
              Permalink
              aw Allen Wittenauer added a comment -

              +1 lgtm

              Show
              aw Allen Wittenauer added a comment - +1 lgtm
              Hide
              Permalink
              aw Allen Wittenauer added a comment -

              Committed to trunk.

              Thanks.

              Show
              aw Allen Wittenauer added a comment - Committed to trunk. Thanks.
              Hide
              Permalink
              hudson Hudson added a comment -

              SUCCESS: Integrated in Hadoop-trunk-Commit #7053 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7053/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              Show
              hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7053 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7053/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #100 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/100/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #100 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/100/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Yarn-trunk #834 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/834/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #834 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/834/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Hdfs-trunk #2032 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2032/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2032 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2032/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #101 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/101/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #101 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/101/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Mapreduce-trunk #2051 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2051/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2051 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2051/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/)
              HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              Show
              hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/ ) HDFS-7546 . Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) (aw: rev 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
              Hide
              Permalink
              yzhangal Yongjun Zhang added a comment -

              Hi Allen Wittenauer,

              You committed this fix to trunk only, did you mean to say that the fix is incompatible change? Thanks.

              Show
              yzhangal Yongjun Zhang added a comment - Hi Allen Wittenauer , You committed this fix to trunk only, did you mean to say that the fix is incompatible change? Thanks.
              Hide
              Permalink
              aw Allen Wittenauer added a comment -

              I committed this to trunk only because my time is more valuable than branch-2.

              Show
              aw Allen Wittenauer added a comment - I committed this to trunk only because my time is more valuable than branch-2.
              Hide
              Permalink
              yzhangal Yongjun Zhang added a comment -

              Hi Allen Wittenauer,

              Thanks for the clarification. I created HDFS-8618 to cherry-pick to branch-2.

              What I will do is to modify the corresponding CHANGES.txt to reflect that it will be fixed in branch-2 (I target it as 2.7.1), and cherry pick both HDFS-7546 and HDFS-8618 to branch-2 and branch-2.7.

              Thanks.

              Show
              yzhangal Yongjun Zhang added a comment - Hi Allen Wittenauer , Thanks for the clarification. I created HDFS-8618 to cherry-pick to branch-2. What I will do is to modify the corresponding CHANGES.txt to reflect that it will be fixed in branch-2 (I target it as 2.7.1), and cherry pick both HDFS-7546 and HDFS-8618 to branch-2 and branch-2.7. Thanks.
              Hide
              Permalink
              yzhangal Yongjun Zhang added a comment -

              HI Vinod Kumar Vavilapalli,

              I attached an addendum patch to this jira, with the intention to commit the addendum to trunk, then cherry-pick both the commit Allen Wittenauer did for this jira and the the addendum to branch-2 and branch-2.7 (targetting 2.7.1), would you please help taking a look whether this makes sense?

              Thanks much.

              Show
              yzhangal Yongjun Zhang added a comment - HI Vinod Kumar Vavilapalli , I attached an addendum patch to this jira, with the intention to commit the addendum to trunk, then cherry-pick both the commit Allen Wittenauer did for this jira and the the addendum to branch-2 and branch-2.7 (targetting 2.7.1), would you please help taking a look whether this makes sense? Thanks much.
              Hide
              Permalink
              vinodkv Vinod Kumar Vavilapalli added a comment -

              Yongjun Zhang, +1, let's get this minor but useful improvement into 2.7.1. Thanks.

              Show
              vinodkv Vinod Kumar Vavilapalli added a comment - Yongjun Zhang , +1, let's get this minor but useful improvement into 2.7.1. Thanks.
              Hide
              Permalink
              yzhangal Yongjun Zhang added a comment -

              Many thanks Vinod Kumar Vavilapalli, I cherry-picked to branch-2 and branch-2.7.

              Show
              yzhangal Yongjun Zhang added a comment - Many thanks Vinod Kumar Vavilapalli , I cherry-picked to branch-2 and branch-2.7.
              Hide
              Permalink
              hudson Hudson added a comment -

              SUCCESS: Integrated in Hadoop-Yarn-trunk #964 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/964/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #964 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/964/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #234 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/234/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #234 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/234/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Hdfs-trunk #2162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2162/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2162/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #223 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/223/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #223 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/223/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-trunk-Commit #8041 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8041/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8041 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8041/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #232 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/232/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #232 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/232/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              hudson Hudson added a comment -

              FAILURE: Integrated in Hadoop-Mapreduce-trunk #2180 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2180/)
              Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5)

              • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2180 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2180/ ) Move HDFS-7546 to release 2.7.1 in CHANGES.txt. (yzhang: rev bcb3c40bed572a2dd95ce7201d893d7bf59240d5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

                People

                • Assignee:
                  qwertymaniac Harsh J
                  Reporter:
                  qwertymaniac Harsh J
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  7 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development