Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6565

Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      By default, the service field of a delegation token is populated based on server IP address. Setting hadoop.security.token.service.use_ip to false changes this behavior to use host name instead of IP address. However, this configuration property is not read from job.xml. Instead, it's read from a separate Configuration instance created during static initialization of SecurityUtil. This does not work correctly with MapReduce jobs if the framework is distributed by setting mapreduce.application.framework.path and the mapreduce.application.classpath is isolated to avoid reading core-site.xml from the cluster nodes. MapReduce tasks will fail to authenticate to HDFS, because they'll try to find a delegation token based on the NameNode IP address, even though at job submission time the tokens were generated using the host name.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gtCarrera9 Li Lu
                Reporter:
                cnauroth Chris Nauroth
              • Votes:
                1 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: