[SPARK-27891] Long running spark jobs fail because of HDFS delegation token expires

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Cannot Reproduce
• Affects Version/s: 2.0.1, 2.1.0, 2.3.1, 2.4.1
• Component/s: Security
• Assignee: Unassigned
• Reporter: hemshankar sahu
• Created: 30/May/19 21:46
• Updated: 13/Nov/24 21:15
• Resolved: 24/Apr/20 00:09

Description

When the spark job runs on a secured cluster for longer then time that is mentioned in the dfs.namenode.delegation.token.renew-interval property of hdfs-site.xml the spark job fails. ** 

Following command was used to submit the spark job

bin/spark-submit --principal acekrbuser --keytab ~/keytabs/acekrbuser.keytab --master yarn --deploy-mode cluster examples/src/main/python/wordcount.py /tmp/ff1.txt

 

Application Logs attached

 

Issue Links

• relates to: SPARK-23361 Driver restart fails if it happens after 7 days from app submission