[LIVY-549] Livy submit application exist [hdfs delegation token] error

• Type: Question
• Status: Closed
• Priority: Blocker
• Resolution: Done
• Affects Version/s: 0.5.0
• Component/s: API, Batch
• Assignee: Unassigned
• Reporter: gejx
• Created: 18/Jan/19 09:22
• Updated: 11/Feb/19 02:17
• Resolved: 11/Feb/19 02:17

Description

I have already used to try spark-submit way with  --principal and --keytab parameters. This way was successd. 

I have a Spark job where part of the work done on the executors includes POSTing to a SPNEGO-enabled REST API endpoint, and unless I'm mistaken, when Spark is running a job as a proxy user, the doAs() chain doesn't have the necessary credentials to actually do a SPNEGO negotiation.

How can I use Livy to achieve this function. Thank You.

spark version:2.3.1

https://issues.cloudera.org/browse/LIVY-44?jql=labels%20%3D%20Hue