Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-3547

Spark scala line breaks not working (illegal start of definition)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.8.0
    • Component/s: Interpreters
    • Labels:
      None

      Description

      If I format spark scala statements with line breaks (for readability) and then run the para I get:

      <console>:1: error: illegal start of definition .read.format("com.databricks.spark.csv")

      The following will produce this error:

      %spark
      val path = z.textbox("dataDir")
      val filename = z.textbox("projectName")val dfArticles = spark
      {{ .read.format("com.databricks.spark.csv")}}
      {{ .option("header", "true")}}
      {{ .option("inferSchema", "true")}}
      {{ .load(path+"/"filename"-articles.csv"}}

      If the entire statement is on one continuous line, it runs successfully.

      I am a aware that I can wrap the block with curly braces to get around this but that is not really a solution since it affects the scope of the statements making things defined in the block not available in other paragraphs in the notebook.

      Using continuous lines is not really a workable option when working with anything more than trivial spark queries.

      I'd really like to be using things like Helium spell extensions, notebook level form fields and all of the other good stuff in 0.8.0 but this issue is so debilitating it has me sticking with 0.7.3.

      Please fix this soon.

      I built from branch-0.8.0 with:

      mvn clean package -Pspark-2.1 -Phadoop-2.4 -Pscala-2.11 -DskipTests

       

       

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mjmeyer23 Matthew J Meyer
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: