Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1566

Need to add a mechanism to import tokens and secrets into a submitted job.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: security
    • Labels:
      None

      Description

      We need to include tokens and secrets into a submitted job. I propose adding a configuration attribute that when pointed at a token storage file will include the tokens and secrets from that token storage file.

      1. MR-1566.1.patch
        9 kB
        Jitendra Nath Pandey
      2. MR-1566.2.patch
        9 kB
        Jitendra Nath Pandey
      3. MR-1566.3.patch
        15 kB
        Jitendra Nath Pandey
      4. mr-1566-1.1.patch
        7 kB
        Devaraj Das
      5. mr-1566-1.patch
        17 kB
        Devaraj Das

        Issue Links

          Activity

          Hide
          Devaraj Das added a comment -

          Attaching patch for Y20 (on behalf of Owen). Not for commit here.

          Show
          Devaraj Das added a comment - Attaching patch for Y20 (on behalf of Owen). Not for commit here.
          Hide
          Devaraj Das added a comment -

          This patch is a bugfix on top of the earlier patch.

          Show
          Devaraj Das added a comment - This patch is a bugfix on top of the earlier patch.
          Hide
          Jitendra Nath Pandey added a comment -

          Patch for trunk.

          Show
          Jitendra Nath Pandey added a comment - Patch for trunk.
          Hide
          Devaraj Das added a comment -

          There is an unrelated change in the file Example.java. Other than that, looks fine.

          Show
          Devaraj Das added a comment - There is an unrelated change in the file Example.java. Other than that, looks fine.
          Hide
          Jitendra Nath Pandey added a comment -

          New patch addressing the comment.

          Show
          Jitendra Nath Pandey added a comment - New patch addressing the comment.
          Hide
          Jitendra Nath Pandey added a comment -

          Added a new test which uses the binary token file.

          Show
          Jitendra Nath Pandey added a comment - Added a new test which uses the binary token file.
          Hide
          Jitendra Nath Pandey added a comment -

          ant test was run manually. All tests pass except TestRumenJobTraces, which is unrelated.

          Show
          Jitendra Nath Pandey added a comment - ant test was run manually. All tests pass except TestRumenJobTraces, which is unrelated.
          Hide
          Devaraj Das added a comment -

          Couldn't we enhance TestTokenCache instead of adding a new test. Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file...

          Show
          Devaraj Das added a comment - Couldn't we enhance TestTokenCache instead of adding a new test. Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file...
          Hide
          Jitendra Nath Pandey added a comment -

          When credentials object reads a token from file, it clears all exisiting credentials. Therefore, merging the two will require another job and mapper implementation. It will be cleaner to have a separate test. Also, the mechanism to pass tokens via a file is not much related to TokenCache except that it uses obtainTokensFromNamenodeInternal method.

          > Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file
          The map task in the new test gets the tokens from credentials and verifies it against the token in the file, therefore it also reads the file.

          Show
          Jitendra Nath Pandey added a comment - When credentials object reads a token from file, it clears all exisiting credentials. Therefore, merging the two will require another job and mapper implementation. It will be cleaner to have a separate test. Also, the mechanism to pass tokens via a file is not much related to TokenCache except that it uses obtainTokensFromNamenodeInternal method. > Also, in the mapper, we should look at the credentials via the APIs (as is done in TestTokenCache) instead of reading the file The map task in the new test gets the tokens from credentials and verifies it against the token in the file, therefore it also reads the file.
          Hide
          Devaraj Das added a comment -

          I just committed this. Thanks, Jitendra & Owen!

          Show
          Devaraj Das added a comment - I just committed this. Thanks, Jitendra & Owen!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development