Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2531

Prevent Spark trying for token which is already available

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:
      None

      Description

      As per Apache Spark documentation :
      The property spark.yarn.security.tokens.service.enabled in Apache Spark which controls whether to retrieve delegation tokens for non-HDFS services when security is enabled. By default, delegation tokens for all supported services are retrieved when those services are configured, but it's possible to disable that behavior if it somehow conflicts with the application being run.
      Currently supported services are: hive, hbase

      It would be good to have default as spark.yarn.security.tokens.hive.enabled=false to avoid having Spark redundantly retry obtaining a token when it already has one (passed originally from Oozie's hcat credentials).

        Attachments

        1. OOZIE-2531-2.patch
          2 kB
          Satish Saley
        2. OOZIE-2531-1.patch
          2 kB
          Satish Saley

          Activity

            People

            • Assignee:
              satishsaley Satish Saley
              Reporter:
              satishsaley Satish Saley
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: