Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27515

[Deploy] When application master retry after a long time running, the hdfs delegation token may be expired

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.3.2
    • Fix Version/s: None
    • Component/s: Deploy
    • Labels:
      None

      Description

      When submit a spark yarn application, we first create a container launch context and store the relative tokens.
      And for each attempt of applicationMaster, it would transfer origin tokens for connecting to yarn.
      However, it also transfer origin hdfs delegation tokens.
      For a spark streaming application, if its applicationMaster failed when it has run for a long duration.
      The hdfs token stored in container launch context may be expired.
      When the new attempt applicationMaster prepareLocalResources, it would access the hdfs and failed for token expired.
      This error occured when we rolling upgrading our cluster.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                hzfeiwang feiwang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: