[ZEPPELIN-4722] Support User Impersonation via --proxy-user for Spark Interpreter with K8s

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.9.0
• Component/s: Kubernetes
• Assignee: Philipp Dallig
• Reporter: Andreas Weise
• Created: 03/Apr/20 20:07
• Updated: 24/Dec/20 03:17
• Resolved: 15/Apr/20 05:34

Description

Enabling User Impersonation for Spark Interpreter running on K8s is currently not supported. (propably also not on other interpreters)

While the Spark Interpreter can be configured with "User Impersonate", no --proxy-user is passed to the corresponding spark-submit command in K8s.

Interpreter launch command:  /opt/spark/bin/spark-submit --class org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer --driver-class-path ":/opt/zeppelin/interpreter/spark/*::/opt/zeppelin/interpreter/zeppelin-interpreter-shaded-0.9.0-SNAPSHOT.jar:/opt/zeppelin/interpreter/spark/spark-interpreter-0.9.0-SNAPSHOT.jar" --driver-java-options " -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///opt/zeppelin/conf/log4j.properties -Dzeppelin.log.file='/opt/zeppelin/logs/zeppelin-interpreter-spark-aweise--spark-lnsuqc.log'" --master k8s://https://kubernetes.default.svc  --deploy-mode client --driver-memory 1g --conf spark.kubernetes.namespace=spark-test --conf spark.executor.instances=1 --conf spark.kubernetes.driver.pod.name=spark-lnsuqc --conf spark.kubernetes.container.image=spark-anaconda:3.0.0 --conf spark.driver.bindAddress=0.0.0.0 --conf spark.driver.host=spark-lnsuqc.spark-test.svc --conf spark.driver.port=22321 --conf spark.blockManager.port=22322 /opt/zeppelin/interpreter/spark/spark-interpreter-0.9.0-SNAPSHOT.jar zeppelin-server 12320 "spark-aweise" 12321:12321

 

Issue Links

• links to: GitHub Pull Request #3733