[ZEPPELIN-3547] Spark scala line breaks not working (illegal start of definition)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.8.1
• Fix Version/s: 0.8.0
• Component/s: Interpreters
• Assignee: Unassigned
• Reporter: Matthew J Meyer
• Created: 16/Jun/18 18:59
• Updated: 13/Jul/18 14:48
• Resolved: 17/Jun/18 17:20

Description

If I format spark scala statements with line breaks (for readability) and then run the para I get:

<console>:1: error: illegal start of definition .read.format("com.databricks.spark.csv")

The following will produce this error:

%spark
val path = z.textbox("dataDir")
val filename = z.textbox("projectName")val dfArticles = spark
{{ .read.format("com.databricks.spark.csv")}}
{{ .option("header", "true")}}
{{ .option("inferSchema", "true")}}
{{ .load(path+"/"filename"-articles.csv"}}

If the entire statement is on one continuous line, it runs successfully.

I am a aware that I can wrap the block with curly braces to get around this but that is not really a solution since it affects the scope of the statements making things defined in the block not available in other paragraphs in the notebook.

Using continuous lines is not really a workable option when working with anything more than trivial spark queries.

I'd really like to be using things like Helium spell extensions, notebook level form fields and all of the other good stuff in 0.8.0 but this issue is so debilitating it has me sticking with 0.7.3.

Please fix this soon.

I built from branch-0.8.0 with:

mvn clean package -Pspark-2.1 -Phadoop-2.4 -Pscala-2.11 -DskipTests

 

 

 

 

Issue Links

• is related to: ZEPPELIN-1620 livy.spark interpreter formatting breaks for scala with new line