[SPARK-41015] Failure of ProtobufCatalystDataConversionSuite.scala

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Epic Link: Protobuf Support
• Assignee: Raghu Angadi
• Reporter: Max Gekk
• Created: 04/Nov/22 12:08
• Updated: 08/Nov/22 17:49
• Resolved: 08/Nov/22 10:19

Description

To reproduce the issue, set the seed to 38:

diff --git a/connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala b/connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala
index 271c5b0fec..080bf1eb1f 100644
--- a/connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala
+++ b/connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala
@@ -123,7 +123,7 @@ class ProtobufCatalystDataConversionSuite
     StringType -> ("StringMsg", ""))

   testingTypes.foreach { dt =>
-    val seed = 1 + scala.util.Random.nextInt((1024 - 1) + 1)
+    val seed = 38
     test(s"single $dt with seed $seed") {

       val (messageName, defaultValue) = catalystTypesToProtoMessages(dt.fields(0).dataType)

and run the test:

build/sbt "test:testOnly *ProtobufCatalystDataConversionSuite"

which fails with NPE:

[info] - single StructType(StructField(double_type,DoubleType,true)) with seed 38 *** FAILED *** (10 milliseconds)
[info]   java.lang.NullPointerException:
[info]   at org.apache.spark.sql.protobuf.ProtobufCatalystDataConversionSuite.$anonfun$new$2(ProtobufCatalystDataConversionSuite.scala:134)
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)

Issue Links

• links to: [Github] Pull Request #38515 (SandishKumarHN)