Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11182

HDFS Delegation Token will be expired when calling "UserGroupInformation.getCurrentUser.addCredentials" in HA mode

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.1
    • Fix Version/s: None
    • Component/s: YARN
    • Labels:
      None

      Description

      In HA mode, DFSClient will generate HDFS Delegation Token for each Name Node automatically, which will not be updated when Spark update Credentials for the current user.
      Spark should update these tokens in order to avoid Token Expired Error.

        Issue Links

          Activity

          Show
          marsishandsome Liangliang Gu added a comment - https://github.com/apache/spark/pull/9168
          Hide
          apachespark Apache Spark added a comment -

          User 'marsishandsome' has created a pull request for this issue:
          https://github.com/apache/spark/pull/9168

          Show
          apachespark Apache Spark added a comment - User 'marsishandsome' has created a pull request for this issue: https://github.com/apache/spark/pull/9168
          Hide
          jzhuge John Zhuge added a comment -

          Is this issue fixed by HDFS-9276?

          Show
          jzhuge John Zhuge added a comment - Is this issue fixed by HDFS-9276 ?

            People

            • Assignee:
              Unassigned
              Reporter:
              marsishandsome Liangliang Gu
            • Votes:
              3 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:

                Development