Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4676

JavaSchemaRDD.schema may throw NullType MatchError if sql has null

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: SQL
    • Labels:
      None

      Description

      val jsc = new org.apache.spark.api.java.JavaSparkContext(sc)
      val jhc = new org.apache.spark.sql.hive.api.java.JavaHiveContext(jsc)
      val nrdd = jhc.hql("select null from spark_test.for_test")
      println(nrdd.schema)
      Then the error is thrown as follows:
      scala.MatchError: NullType (of class org.apache.spark.sql.catalyst.types.NullType$)
      at org.apache.spark.sql.types.util.DataTypeConversions$.asJavaDataType(DataTypeConversions.scala:43)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              yantangzhai YanTang Zhai
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: