[SPARK-23781] Merge YARN and Mesos token renewal code

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0
• Component/s: Mesos, Spark Core, YARN
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 23/Mar/18 17:19
• Updated: 17/May/20 18:14
• Resolved: 31/Oct/18 18:00

Description

With the fix for SPARK-23361, the code that handles delegation tokens in Mesos and YARN ends up being very similar.

We should refactor that code so that both backends are sharing the same code, which also would make it easier for other cluster managers to use that code.

Issue Links

• is duplicated by: SPARK-25693 Fix the multiple "manager" classes in org.apache.spark.deploy.security
• relates to: SPARK-23361 Driver restart fails if it happens after 7 days from app submission
• links to: [Github] Pull Request #22624 (vanzin)
• links to: [Github] Pull Request #22624 (vanzin)