Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1620

livy.spark interpreter formatting breaks for scala with new line

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.6.0
    • Fix Version/s: None
    • Component/s: livy-interpreter
    • Labels:
    • Environment:

      HDP 2.5
      Apache Zeppelin

      Version 0.6.0.2.5.0.0-1245

      Description

      When using %livy.spark

      if you have new line and comments in multi-line statement it breaks. Same code using regular spark statement works correct.

      Error that you get:
      <console>:1: error: illegal start of definition
      .format("com.databricks.spark.csv")

      Example:
      %livy.spark

      val df = sqlContext.read
      .format("com.databricks.spark.csv")
      .option("header", "true") // Use first line of all files as header
      .option("inferSchema", "true") // Automatically infer data types
      .load("/tmp/airflightsdelays/") // Read all flights

      df.printSchema

      If i consolidate the code it works fine:
      %livy.spark

      val df = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").load("/tmp/airflightsdelays/") // Read all flights

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                azeltov alex zeltov
              • Votes:
                7 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated: