[SPARK-3536] SELECT on empty parquet table throws exception

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: SQL
• Labels: starter
• Target Version/s: 1.2.0
• Assignee: Ravindra Pesala
• Reporter: Michael Armbrust
• Created: 15/Sep/14 21:35
• Updated: 23/Sep/14 18:52
• Resolved: 23/Sep/14 18:52

Description

Reported by matei. Reproduce as follows:

scala> case class Data(i: Int)
defined class Data

scala> createParquetFile[Data]("testParquet")
scala> parquetFile("testParquet").count()
14/09/15 14:34:17 WARN scheduler.DAGScheduler: Creating new stage failed due to exception - job: 0
java.lang.NullPointerException
	at org.apache.spark.sql.parquet.FilteringParquetRowInputFormat.getSplits(ParquetTableOperations.scala:438)
	at parquet.hadoop.ParquetInputFormat.getSplits(ParquetInputFormat.java:344)
	at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:95)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:204)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:202)

Issue Links

• links to: [Github] Pull Request #2456 (ravipesala)