Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Not A Problem
-
0.8.1
-
None
Description
If I format spark scala statements with line breaks (for readability) and then run the para I get:
<console>:1: error: illegal start of definition .read.format("com.databricks.spark.csv")
The following will produce this error:
%spark
val path = z.textbox("dataDir")
val filename = z.textbox("projectName")val dfArticles = spark
{{ .read.format("com.databricks.spark.csv")}}
{{ .option("header", "true")}}
{{ .option("inferSchema", "true")}}
{{ .load(path+"/"filename"-articles.csv"}}
If the entire statement is on one continuous line, it runs successfully.
I am a aware that I can wrap the block with curly braces to get around this but that is not really a solution since it affects the scope of the statements making things defined in the block not available in other paragraphs in the notebook.
Using continuous lines is not really a workable option when working with anything more than trivial spark queries.
I'd really like to be using things like Helium spell extensions, notebook level form fields and all of the other good stuff in 0.8.0 but this issue is so debilitating it has me sticking with 0.7.3.
Please fix this soon.
I built from branch-0.8.0 with:
mvn clean package -Pspark-2.1 -Phadoop-2.4 -Pscala-2.11 -DskipTests
Attachments
Attachments
Issue Links
- is related to
-
ZEPPELIN-1620 livy.spark interpreter formatting breaks for scala with new line
- Open