Hadoop Common
  1. Hadoop Common
  2. HADOOP-6472

add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      for MAPRED-1338 - we need an option to pass a file with secreteKeys (tokens) to a mapreduce Job.
      Name of the file is set into the config and will be picked up by JobSubmiter.

      1. HADOOP-6472-2.patch
        5 kB
        Boris Shkolnik
      2. HADOOP-6472-1.patch
        5 kB
        Boris Shkolnik
      3. HADOOP-6472.patch
        4 kB
        Boris Shkolnik

        Activity

        Hide
        Owen O'Malley added a comment -

        Nobody closed this when it was committed.

        Show
        Owen O'Malley added a comment - Nobody closed this when it was committed.
        Hide
        Sam Walker added a comment -

        Alpha Bravo Charlie

        Show
        Sam Walker added a comment - Alpha Bravo Charlie
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #209 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/209/)
        . add tokenCache option to GenericOptionsParser for passing
        file with secret keys to a map reduce job. (boryas)

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #209 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/209/ ) . add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job. (boryas)
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #131 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/131/)
        . add tokenCache option to GenericOptionsParser for passing
        file with secret keys to a map reduce job. (boryas)

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #131 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/131/ ) . add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job. (boryas)
        Hide
        Devaraj Das added a comment -

        Looks good. +1

        Show
        Devaraj Das added a comment - Looks good. +1
        Hide
        Boris Shkolnik added a comment -

        changed File.getAbsolutePath() to File.toString()
        removed some commented line

        Show
        Boris Shkolnik added a comment - changed File.getAbsolutePath() to File.toString() removed some commented line
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12429367/HADOOP-6472-1.patch
        against trunk revision 895467.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429367/HADOOP-6472-1.patch against trunk revision 895467. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/console This message is automatically generated.
        Hide
        Boris Shkolnik added a comment -

        addressed all the comments.
        Also modified test to have setUp and tearDown methods.

        Show
        Boris Shkolnik added a comment - addressed all the comments. Also modified test to have setUp and tearDown methods.
        Hide
        Devaraj Das added a comment -

        Looks good. Some minor comments:
        1) Remove the LOG.info calls
        2) There seems to be an unused import in the testcase (org.apache.hadoop.record.Utils).
        3) Some unnecessary whitespace changes in the testcase.
        4) Maybe we should rename the "tokenCache" option to "tokenCacheFile" to make it more specific that it needs to be a file..

        Show
        Devaraj Das added a comment - Looks good. Some minor comments: 1) Remove the LOG.info calls 2) There seems to be an unused import in the testcase (org.apache.hadoop.record.Utils). 3) Some unnecessary whitespace changes in the testcase. 4) Maybe we should rename the "tokenCache" option to "tokenCacheFile" to make it more specific that it needs to be a file..

          People

          • Assignee:
            Boris Shkolnik
            Reporter:
            Boris Shkolnik
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development