[SPARK-24245] Flaky test: KafkaContinuousSinkSuite

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.1
• Component/s: Structured Streaming, Tests
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Marcelo Masiero Vanzin
• Created: 10/May/18 20:35
• Updated: 08/Oct/19 05:41
• Resolved: 08/Oct/19 05:41

Description

This test seems to be broken or flaky. From jenkins:

https://amplab.cs.berkeley.edu/jenkins/user/vanzin/my-views/view/Spark/job/spark-branch-2.3-test-maven-hadoop-2.6/367/

Caused by: org.scalatest.exceptions.TestFailedException: -1 did not equal 0
        at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:528)
        at org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1560)
        at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:501)
        at org.apache.spark.sql.kafka010.KafkaContinuousTest$$anonfun$afterEach$1.apply(KafkaContinuousTest.scala:76)
        at org.apache.spark.sql.kafka010.KafkaContinuousTest$$anonfun$afterEach$1.apply(KafkaContinuousTest.scala:76)
        at org.scalatest.concurrent.Eventually$class.makeAValiantAttempt$1(Eventually.scala:395)
        at org.scalatest.concurrent.Eventually$class.tryTryAgain$1(Eventually.scala:409)
        ... 54 more
SUITE ABORTED - KafkaContinuousSinkSuite: The code passed to eventually never returned normally. Attempted 1990 times over 30.007800773 seconds. Last failure message: -1 did not equal 0.

We should fix it or disable it in 2.3.