Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1566

Need to add a mechanism to import tokens and secrets into a submitted job.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: security
    • Labels:
      None

      Description

      We need to include tokens and secrets into a submitted job. I propose adding a configuration attribute that when pointed at a token storage file will include the tokens and secrets from that token storage file.

      1. MR-1566.3.patch
        15 kB
        Jitendra Nath Pandey
      2. MR-1566.2.patch
        9 kB
        Jitendra Nath Pandey
      3. MR-1566.1.patch
        9 kB
        Jitendra Nath Pandey
      4. mr-1566-1.1.patch
        7 kB
        Devaraj Das
      5. mr-1566-1.patch
        17 kB
        Devaraj Das

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          139d 1h 44m 1 Devaraj Das 23/Jul/10 01:51
          Resolved Resolved Closed Closed
          507d 5h 28m 1 Konstantin Shvachko 12/Dec/11 06:19
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
          Devaraj Das made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Owen O'Malley [ owen.omalley ] Jitendra Nath Pandey [ jnp ]
          Resolution Fixed [ 1 ]
          Hide
          Devaraj Das added a comment -

          I just committed this. Thanks, Jitendra & Owen!

          Show
          Devaraj Das added a comment - I just committed this. Thanks, Jitendra & Owen!
          Hide
          Jitendra Nath Pandey added a comment -

          When credentials object reads a token from file, it clears all exisiting credentials. Therefore, merging the two will require another job and mapper implementation. It will be cleaner to have a separate test. Also, the mechanism to pass tokens via a file is not much related to TokenCache except that it uses obtainTokensFromNamenodeInternal method.

          > Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file
          The map task in the new test gets the tokens from credentials and verifies it against the token in the file, therefore it also reads the file.

          Show
          Jitendra Nath Pandey added a comment - When credentials object reads a token from file, it clears all exisiting credentials. Therefore, merging the two will require another job and mapper implementation. It will be cleaner to have a separate test. Also, the mechanism to pass tokens via a file is not much related to TokenCache except that it uses obtainTokensFromNamenodeInternal method. > Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file The map task in the new test gets the tokens from credentials and verifies it against the token in the file, therefore it also reads the file.
          Hide
          Devaraj Das added a comment -

          Couldn't we enhance TestTokenCache instead of adding a new test. Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file...

          Show
          Devaraj Das added a comment - Couldn't we enhance TestTokenCache instead of adding a new test. Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file...
          Hide
          Jitendra Nath Pandey added a comment -

          ant test was run manually. All tests pass except TestRumenJobTraces, which is unrelated.

          Show
          Jitendra Nath Pandey added a comment - ant test was run manually. All tests pass except TestRumenJobTraces, which is unrelated.
          Jitendra Nath Pandey made changes -
          Attachment MR-1566.3.patch [ 12450111 ]
          Hide
          Jitendra Nath Pandey added a comment -

          Added a new test which uses the binary token file.

          Show
          Jitendra Nath Pandey added a comment - Added a new test which uses the binary token file.
          Jitendra Nath Pandey made changes -
          Attachment MR-1566.2.patch [ 12449867 ]
          Hide
          Jitendra Nath Pandey added a comment -

          New patch addressing the comment.

          Show
          Jitendra Nath Pandey added a comment - New patch addressing the comment.
          Hide
          Devaraj Das added a comment -

          There is an unrelated change in the file Example.java. Other than that, looks fine.

          Show
          Devaraj Das added a comment - There is an unrelated change in the file Example.java. Other than that, looks fine.
          Jitendra Nath Pandey made changes -
          Attachment MR-1566.1.patch [ 12449574 ]
          Hide
          Jitendra Nath Pandey added a comment -

          Patch for trunk.

          Show
          Jitendra Nath Pandey added a comment - Patch for trunk.
          Jitendra Nath Pandey made changes -
          Link This issue is blocked by HADOOP-6861 [ HADOOP-6861 ]
          Devaraj Das made changes -
          Attachment mr-1566-1.1.patch [ 12438376 ]
          Hide
          Devaraj Das added a comment -

          This patch is a bugfix on top of the earlier patch.

          Show
          Devaraj Das added a comment - This patch is a bugfix on top of the earlier patch.
          Devaraj Das made changes -
          Field Original Value New Value
          Attachment mr-1566-1.patch [ 12438122 ]
          Hide
          Devaraj Das added a comment -

          Attaching patch for Y20 (on behalf of Owen). Not for commit here.

          Show
          Devaraj Das added a comment - Attaching patch for Y20 (on behalf of Owen). Not for commit here.
          Owen O'Malley created issue -

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development