Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1624

Document the job credentials and associated details to do with delegation tokens (on the client side)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: documentation
    • Labels:
      None

      Description

      Document the job credentials and associated details to do with delegation tokens (on the client side)

      1. job-creds.2.patch
        8 kB
        Devaraj Das
      2. job-creds.patch
        5 kB
        Devaraj Das
      3. MR-1624.1.patch
        8 kB
        Jitendra Nath Pandey
      4. MR-1624.2.patch
        8 kB
        Devaraj Das
      5. MR-1624.2.patch
        8 kB
        Jitendra Nath Pandey

        Activity

        Hide
        Devaraj Das added a comment -

        Patch for Y20

        Show
        Devaraj Das added a comment - Patch for Y20
        Hide
        Owen O'Malley added a comment -

        I'd change the first paragraph to read slightly different:

        + <p>In a secure cluster, the user is authenticated via Kerberos'
        + kinit command. Because of scalability concerns, we don't push
        + the client's Kerberos' tickets in MapReduce jobs. Instead, we
        + acquire delegation tokens from each HDFS NameNode that the job
        + will use and store them in the job as part of job submission.
        + The delegation tokens are automatically obtained
        + for the HDFS that holds the staging directories, where the job
        + job files are written, and any HDFS systems referenced by
        + FileInputFormats, FileOutputFormats, DistCp, and the
        + distributed cache.
        + Other applications require to set the configuration
        + "mapreduce.job.hdfs-servers" for all NameNodes that tasks might
        + need to talk during the job execution. This is a comma separated
        + list of file system names, such as "hdfs://nn1/,hdfs://nn2/".
        + These tokens are passed to the JobTracker
        + as part of the job submission as <a href="ext:api/org/apache/hadoop/
        + security/credentials">Credentials</a>. </p>

        Show
        Owen O'Malley added a comment - I'd change the first paragraph to read slightly different: + <p>In a secure cluster, the user is authenticated via Kerberos' + kinit command. Because of scalability concerns, we don't push + the client's Kerberos' tickets in MapReduce jobs. Instead, we + acquire delegation tokens from each HDFS NameNode that the job + will use and store them in the job as part of job submission. + The delegation tokens are automatically obtained + for the HDFS that holds the staging directories, where the job + job files are written, and any HDFS systems referenced by + FileInputFormats, FileOutputFormats, DistCp, and the + distributed cache. + Other applications require to set the configuration + "mapreduce.job.hdfs-servers" for all NameNodes that tasks might + need to talk during the job execution. This is a comma separated + list of file system names, such as "hdfs://nn1/,hdfs://nn2/". + These tokens are passed to the JobTracker + as part of the job submission as <a href="ext:api/org/apache/hadoop/ + security/credentials">Credentials</a>. </p>
        Hide
        Devaraj Das added a comment -

        This incorporate Owen's comments.

        Show
        Devaraj Das added a comment - This incorporate Owen's comments.
        Hide
        Jitendra Nath Pandey added a comment -

        Patch for trunk.

        Show
        Jitendra Nath Pandey added a comment - Patch for trunk.
        Hide
        Jitendra Nath Pandey added a comment -

        ant docs run and verified.

        Show
        Jitendra Nath Pandey added a comment - ant docs run and verified.
        Hide
        Devaraj Das added a comment -

        Attaching an updated patch that has the correct links. I committed this patch.

        Show
        Devaraj Das added a comment - Attaching an updated patch that has the correct links. I committed this patch.
        Hide
        Devaraj Das added a comment -

        Thanks, Jitendra for the trunk patch.

        Show
        Devaraj Das added a comment - Thanks, Jitendra for the trunk patch.
        Hide
        Devaraj Das added a comment -

        I am reopening this issue. The links in the patch points to currently non-existing Common project Javadoc URLs. We need to see how to have the links correctly and automatically set when we make releases of the Common & MapReduce projects.

        Show
        Devaraj Das added a comment - I am reopening this issue. The links in the patch points to currently non-existing Common project Javadoc URLs. We need to see how to have the links correctly and automatically set when we make releases of the Common & MapReduce projects.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
        Hide
        Owen O'Malley added a comment -

        I believe the urls are right now and this was already committed to trunk.

        Show
        Owen O'Malley added a comment - I believe the urls are right now and this was already committed to trunk.

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development