Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26426

ExpressionInfo related unit tests fail in Windows

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 3.0.0
    • Fix Version/s: 2.4.1, 3.0.0
    • Component/s: SQL
    • Labels:
      None
    • Environment:

      Description

      Using Windows 7 OS, after run"mvn install" for latest spark version, and then run "mvn test -Dtest=none -DwildcardSuites=org.apache.spark.sql.execution.streaming.sources.ForeachBatchSinkSuite -pl :spark-sql_2.12", it fails with Assertion error as follows

      ForeachBatchSinkSuite:

      • foreachBatch with non-stateful query *** FAILED ***
        java.lang.AssertionError:
        at org.apache.spark.sql.catalyst.expressions.ExpressionInfo.<init>(ExpressionInfo.java:82)
        at org.apache.spark.sql.catalyst.analysis.FunctionRegistry$.expressionInfo(FunctionRegistry.scala:636)
        at org.apache.spark.sql.catalyst.analysis.FunctionRegistry$.expression(FunctionRegistry.scala:595)
        at org.apache.spark.sql.catalyst.analysis.FunctionRegistry$.<init>(FunctionRegistry.scala:193)
        at org.apache.spark.sql.catalyst.analysis.FunctionRegistry$.<clinit>(FunctionRegistry.scala)
        at org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$functionRegistry$2(BaseSessionStateBuilder.scala:99)
        at scala.Option.getOrElse(Option.scala:138)
        at org.apache.spark.sql.internal.BaseSessionStateBuilder.functionRegistry$lzycompute(BaseSessionStateBuilder.scala:99)
        at org.apache.spark.sql.internal.BaseSessionStateBuilder.functionRegistry(BaseSessionStateBuilder.scala:97)
        at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:290)
        ...
          • RUN ABORTED ***
            java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.sql.catalyst.analysis.FunctionRegistry$
            at org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$functionRegistry$2(BaseSessionStateBuilder.scala:99)
            at scala.Option.getOrElse(Option.scala:138)
            at org.apache.spark.sql.internal.BaseSessionStateBuilder.functionRegistry$lzycompute(BaseSessionStateBuilder.scala:99)
            at org.apache.spark.sql.internal.BaseSessionStateBuilder.functionRegistry(BaseSessionStateBuilder.scala:97)
            at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:290)
            at org.apache.spark.sql.test.TestSparkSession.sessionState$lzycompute(TestSQLContext.scala:42)
            at org.apache.spark.sql.test.TestSparkSession.sessionState(TestSQLContext.scala:41)
            at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:77)
            at org.apache.spark.sql.execution.streaming.MemoryStreamBase.toDF(memory.scala:60)
            at org.apache.spark.sql.execution.streaming.sources.ForeachBatchSinkSuite.$anonfun$new$5(ForeachBatchSinkSuite.scala:48)

        Attachments

        1. unit-test.log
          33 kB
          Wang Yanlin

          Issue Links

            Activity

              People

              • Assignee:
                yanlin-Lynn Wang Yanlin
                Reporter:
                yanlin-Lynn Wang Yanlin
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: