Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2109

Add support for reading multiple hadoop delegation token files

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.22.0
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None

      Description

      This is the MR part of HADOOP-6988.

      1. mapreduce-2109.3.txt
        6 kB
        Aaron T. Myers
      2. mapreduce-2109.2.txt
        3 kB
        Aaron T. Myers
      3. mapreduce-2109.1.txt
        4 kB
        Aaron T. Myers
      4. mapreduce-2109.0.txt
        4 kB
        Aaron T. Myers

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          Hi Arun,

          Several possible use cases are described in HADOOP-6988, but to summarize:

          It's presently possible to fetch delegation tokens to files, and to pass these to bin/hadoop when submitting a job. I'd like to be able to specify both MR and HDFS delegation token files upfront so that job submission (eg done by Hue or Hive) doesn't require using kinit. This change could also be useful to pass delegation token files obtained from multiple distinct NNs.

          Show
          Aaron T. Myers added a comment - Hi Arun, Several possible use cases are described in HADOOP-6988 , but to summarize: It's presently possible to fetch delegation tokens to files, and to pass these to bin/hadoop when submitting a job. I'd like to be able to specify both MR and HDFS delegation token files upfront so that job submission (eg done by Hue or Hive) doesn't require using kinit. This change could also be useful to pass delegation token files obtained from multiple distinct NNs.
          Hide
          Arun C Murthy added a comment -

          What is the use case? I can't seem to find one here or in HADOOP-6988 ... thanks.

          Show
          Arun C Murthy added a comment - What is the use case? I can't seem to find one here or in HADOOP-6988 ... thanks.
          Hide
          Aaron T. Myers added a comment -

          Thanks for the review, Eli. I've updated the patch to address your comments.

          Show
          Aaron T. Myers added a comment - Thanks for the review, Eli. I've updated the patch to address your comments.
          Hide
          Eli Collins added a comment -

          mapred_tutorial.xml needs to be updated to read comma separated since that's what's being done in HADOOP-6988. Otherwise looks good.

          Not introduced in your change but "mapreduce.job.credentials.binary" should be defined MRJobConfig, if people don't mind would be good to clean that up here.

          Show
          Eli Collins added a comment - mapred_tutorial.xml needs to be updated to read comma separated since that's what's being done in HADOOP-6988 . Otherwise looks good. Not introduced in your change but "mapreduce.job.credentials.binary" should be defined MRJobConfig, if people don't mind would be good to clean that up here.
          Hide
          Aaron T. Myers added a comment -

          Removed some log output that shouldn't have been included. Apologies for the noise.

          Show
          Aaron T. Myers added a comment - Removed some log output that shouldn't have been included. Apologies for the noise.
          Hide
          Aaron T. Myers added a comment -

          Same patch, this time with -p0.

          Show
          Aaron T. Myers added a comment - Same patch, this time with -p0.
          Hide
          Aaron T. Myers added a comment -

          Adding support for HADOOP_TOKEN_FILE_LOCATION being interpreted as a comma-separated list of paths to delegation token files.

          Show
          Aaron T. Myers added a comment - Adding support for HADOOP_TOKEN_FILE_LOCATION being interpreted as a comma-separated list of paths to delegation token files.

            People

            • Assignee:
              Aaron T. Myers
              Reporter:
              Aaron T. Myers
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development