Hadoop Common
  1. Hadoop Common
  2. HADOOP-9374

Add tokens from -tokenCacheFile into UGI

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 3.0.0, 0.23.7
    • Fix Version/s: 3.0.0, 0.23.7, 2.0.4-alpha
    • Component/s: security
    • Labels:
      None

      Description

      GenericOptionsParser accepts a -tokenCacheFile option. However, it only sets the mapreduce.job.credentials.json conf value instead of also adding the tokens to the UGI so they are usable by the command being executed.

        Activity

        Daryn Sharp created issue -
        Hide
        Daryn Sharp added a comment -

        I have long used this patch for token testing.

        Show
        Daryn Sharp added a comment - I have long used this patch for token testing.
        Daryn Sharp made changes -
        Field Original Value New Value
        Attachment HADOOP-9374.patch [ 12572393 ]
        Daryn Sharp made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12572393/HADOOP-9374.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 tests included appear to have a timeout.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.util.TestWinUtils

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2280//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2280//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12572393/HADOOP-9374.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.util.TestWinUtils +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2280//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2280//console This message is automatically generated.
        Hide
        Daryn Sharp added a comment -

        Windows test failures are completely unrelated.

        Show
        Daryn Sharp added a comment - Windows test failures are completely unrelated.
        Hide
        Arpit Agarwal added a comment -

        TestWinUtils failure is tracked by HADOOP-9372.

        Show
        Arpit Agarwal added a comment - TestWinUtils failure is tracked by HADOOP-9372 .
        Hide
        Kihwal Lee added a comment -

        +1 It supersedes the existing feature of setting "mapreduce.job.credentials.json", since JobContext.ugi will be the same current user ugi which the option parser adds the tokens to, at least for the CLIs we have today.

        Show
        Kihwal Lee added a comment - +1 It supersedes the existing feature of setting "mapreduce.job.credentials.json", since JobContext.ugi will be the same current user ugi which the option parser adds the tokens to, at least for the CLIs we have today.
        Hide
        Daryn Sharp added a comment -

        Thanks Kihwal. I committed to branch trunk/2/23

        Show
        Daryn Sharp added a comment - Thanks Kihwal. I committed to branch trunk/2/23
        Daryn Sharp made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 3.0.0 [ 12320357 ]
        Fix Version/s 0.23.7 [ 12323956 ]
        Fix Version/s 2.0.4-alpha [ 12324135 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3434 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3434/)
        HADOOP-9374. Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3434 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3434/ ) HADOOP-9374 . Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #149 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/149/)
        HADOOP-9374. Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #149 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/149/ ) HADOOP-9374 . Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #547 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/547/)
        svn merge -c 1454019 FIXES: HADOOP-9374. Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454028)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454028
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #547 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/547/ ) svn merge -c 1454019 FIXES: HADOOP-9374 . Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454028) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454028 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1338 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1338/)
        HADOOP-9374. Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1338 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1338/ ) HADOOP-9374 . Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1366 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1366/)
        HADOOP-9374. Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1366 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1366/ ) HADOOP-9374 . Add tokens from -tokenCacheFile into UGI (daryn) (Revision 1454019) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454019 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Daryn Sharp
            Reporter:
            Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development