Hadoop Common
  1. Hadoop Common
  2. HADOOP-6472

add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      for MAPRED-1338 - we need an option to pass a file with secreteKeys (tokens) to a mapreduce Job.
      Name of the file is set into the config and will be picked up by JobSubmiter.

      1. HADOOP-6472.patch
        4 kB
        Boris Shkolnik
      2. HADOOP-6472-1.patch
        5 kB
        Boris Shkolnik
      3. HADOOP-6472-2.patch
        5 kB
        Boris Shkolnik

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        6d 20h 59m 1 Boris Shkolnik 04/Jan/10 20:17
        Patch Available Patch Available Resolved Resolved
        279d 8h 43m 1 Owen O'Malley 11/Oct/10 06:01
        Resolved Resolved Closed Closed
        427d 1h 17m 1 Konstantin Shvachko 12/Dec/11 06:19
        Konstantin Shvachko made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.22.0 [ 12314296 ]
        Resolution Fixed [ 1 ]
        Hide
        Owen O'Malley added a comment -

        Nobody closed this when it was committed.

        Show
        Owen O'Malley added a comment - Nobody closed this when it was committed.
        Hide
        Sam Walker added a comment -

        Alpha Bravo Charlie

        Show
        Sam Walker added a comment - Alpha Bravo Charlie
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #209 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/209/)
        . add tokenCache option to GenericOptionsParser for passing
        file with secret keys to a map reduce job. (boryas)

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #209 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/209/ ) . add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job. (boryas)
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #131 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/131/)
        . add tokenCache option to GenericOptionsParser for passing
        file with secret keys to a map reduce job. (boryas)

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #131 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/131/ ) . add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job. (boryas)
        Hide
        Devaraj Das added a comment -

        Looks good. +1

        Show
        Devaraj Das added a comment - Looks good. +1
        Boris Shkolnik made changes -
        Attachment HADOOP-6472-2.patch [ 12429379 ]
        Hide
        Boris Shkolnik added a comment -

        changed File.getAbsolutePath() to File.toString()
        removed some commented line

        Show
        Boris Shkolnik added a comment - changed File.getAbsolutePath() to File.toString() removed some commented line
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12429367/HADOOP-6472-1.patch
        against trunk revision 895467.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429367/HADOOP-6472-1.patch against trunk revision 895467. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/249/console This message is automatically generated.
        Boris Shkolnik made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Boris Shkolnik made changes -
        Attachment HADOOP-6472-1.patch [ 12429367 ]
        Hide
        Boris Shkolnik added a comment -

        addressed all the comments.
        Also modified test to have setUp and tearDown methods.

        Show
        Boris Shkolnik added a comment - addressed all the comments. Also modified test to have setUp and tearDown methods.
        Hide
        Devaraj Das added a comment -

        Looks good. Some minor comments:
        1) Remove the LOG.info calls
        2) There seems to be an unused import in the testcase (org.apache.hadoop.record.Utils).
        3) Some unnecessary whitespace changes in the testcase.
        4) Maybe we should rename the "tokenCache" option to "tokenCacheFile" to make it more specific that it needs to be a file..

        Show
        Devaraj Das added a comment - Looks good. Some minor comments: 1) Remove the LOG.info calls 2) There seems to be an unused import in the testcase (org.apache.hadoop.record.Utils). 3) Some unnecessary whitespace changes in the testcase. 4) Maybe we should rename the "tokenCache" option to "tokenCacheFile" to make it more specific that it needs to be a file..
        Boris Shkolnik made changes -
        Attachment HADOOP-6472.patch [ 12429040 ]
        Boris Shkolnik made changes -
        Field Original Value New Value
        Assignee Boris Shkolnik [ boryas ]
        Boris Shkolnik created issue -

          People

          • Assignee:
            Boris Shkolnik
            Reporter:
            Boris Shkolnik
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development