Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33392 Align DSv2 commands to DSv1 implementation
  3. SPARK-33889

NPE from SHOW PARTITIONS for null partition values

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      The following test fails with NPE on V2 table:

        test("null and empty string as partition values") {
          import testImplicits._
          withNamespaceAndTable("ns", "tbl") { t =>
            val df = Seq((0, ""), (1, null)).toDF("a", "part")
            df.write
              .partitionBy("part")
              .format("parquet")
              .mode(SaveMode.Overwrite)
              .saveAsTable(t)
      
            runShowPartitionsSql(s"SHOW PARTITIONS $t", Row("part=null") :: Nil)
          }
        }
      
      java.lang.NullPointerException was thrown.
      java.lang.NullPointerException
      	at org.apache.spark.sql.execution.datasources.v2.ShowPartitionsExec.$anonfun$run$3(ShowPartitionsExec.scala:58)
      	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
      

        Attachments

          Activity

            People

            • Assignee:
              maxgekk Max Gekk
              Reporter:
              maxgekk Max Gekk
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: