[ZEPPELIN-3950] zeppelin can't retrive data from spark native k8s cluster correctly

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.0
• Component/s: zeppelin-interpreter

Environment

centos 7.5 

java 1.8

zeppelin 0.8.0 0.9.0 SNAPSHOT

spark 2.4.0

• Flags: Important
• Assignee: Unassigned
• Reporter: YuanzhangZhu
• Created: 16/Jan/19 06:35
• Updated: 16/Jan/19 07:13

Description

I set up native spark on K8S acording to http://spark.apache.org/docs/latest/running-on-kubernetes.html . and set spark interpreter's master :k8s://https://10.64.17.119:6443 

I tried 2 case:

case 1:

import spark.implicits._
val df = Seq(("Judy", "50"),("Randy", "60")).toDF("int_column", "string_column", "date_column")
df.registerTempTable("peopleTB")
%sql
select * from peopleTB

case 2:

%spark
case class WordCountTable(word: String, count: Integer)
val textFile = sc.textFile("hdfs://10.64.17.119:9000/wordcounter.txt")
val wordCount = textFile.map(line => line.split(",")).map(line => WordCountTable(line(0),line(1).toInt)).toDF()
wordCount.registerTempTable("wordCountTB")

%sql
select * from wordCountTB

 

 

Case1 works well , Case 2 alway hung at "select * from wordCountTB"

when I change spark interpreter's master to local[*], the 2 case all works well.

 

It seems  that zeppelin can't retrive data from spark's native k8s cluster correctly