Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37205

Support mapreduce.job.send-token-conf when starting containers in YARN

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • YARN
    • None

    Description

      mapreduce.job.send-token-conf is a useful feature in Hadoop (see YARN-5910 with which RM is not required to statically have config for all the secure HDFS clusters. Currently it only works for MRv2 but it'd be nice if Spark can also use this feature. I think we only need to pass the config to LaunchContainerContext in Client.createContainerLaunchContext.

      Attachments

        Issue Links

          Activity

            People

              csun Chao Sun
              csun Chao Sun
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: