Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24368

Flaky tests: org.apache.spark.sql.execution.datasources.csv.UnivocityParserSuite

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: SQL, Tests
    • Labels:
      None

      Description

      org.apache.spark.sql.execution.datasources.csv.UnivocityParserSuite failed very often.

      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91020/testReport/org.apache.spark.sql.execution.datasources.csv/UnivocityParserSuite/_It_is_not_a_test_it_is_a_sbt_testing_SuiteSelector_/history/

      org.apache.spark.sql.execution.datasources.csv.UnivocityParserSuite *** ABORTED *** (1 millisecond)
      [info]   java.lang.IllegalStateException: LiveListenerBus is stopped.
      [info]   at org.apache.spark.scheduler.LiveListenerBus.addToQueue(LiveListenerBus.scala:97)
      [info]   at org.apache.spark.scheduler.LiveListenerBus.addToStatusQueue(LiveListenerBus.scala:80)
      [info]   at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:93)
      [info]   at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
      [info]   at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
      [info]   at scala.Option.getOrElse(Option.scala:121)
      [info]   at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:120)
      [info]   at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:119)
      [info]   at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:286)
      [info]   at org.apache.spark.sql.test.TestSparkSession.sessionState$lzycompute(TestSQLContext.scala:42)
      [info]   at org.apache.spark.sql.test.TestSparkSession.sessionState(TestSQLContext.scala:41)
      [info]   at org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
      [info]   at org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
      [info]   at scala.Option.map(Option.scala:146)
      [info]   at org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:95)
      [info]   at org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:94)
      [info]   at org.apache.spark.sql.internal.SQLConf$.get(SQLConf.scala:125)
      [info]   at org.apache.spark.sql.execution.datasources.csv.CSVOptions.<init>(CSVOptions.scala:84)
      [info]   at org.apache.spark.sql.execution.datasources.csv.CSVOptions.<init>(CSVOptions.scala:40)
      [info]   at org.apache.spark.sql.execution.datasources.csv.UnivocityParserSuite.<init>(UnivocityParserSuite.scala:30)
      [info]   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      [info]   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      [info]   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      [info]   at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
      [info]   at java.lang.Class.newInstance(Class.java:442)
      [info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:435)
      [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:296)
      [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:286)
      [info]   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      [info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      [info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      [info]   at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Activity

            People

            • Assignee:
              maxgekk Max Gekk
              Reporter:
              smilegator Xiao Li
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: