Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30541

Flaky test: org.apache.spark.sql.kafka010.KafkaDelegationTokenSuite

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: SQL, Structured Streaming
    • Labels:
      None
    • Target Version/s:

      Description

      The test suite has been failing intermittently as of now:

      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116862/testReport/

       

      org.apache.spark.sql.kafka010.KafkaDelegationTokenSuite.(It is not a test it is a sbt.testing.SuiteSelector)
       

      Error Details
      org.scalatest.exceptions.TestFailedDueToTimeoutException: The code passed to eventually never returned normally. Attempted 3939 times over 1.0001223535333332 minutes. Last failure message: KeeperErrorCode = AuthFailed for /brokers/ids.
      
      Stack Trace
      sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedDueToTimeoutException: The code passed to eventually never returned normally. Attempted 3939 times over 1.0001223535333332 minutes. Last failure message: KeeperErrorCode = AuthFailed for /brokers/ids.
      	at org.scalatest.concurrent.Eventually.tryTryAgain$1(Eventually.scala:432)
      	at org.scalatest.concurrent.Eventually.eventually(Eventually.scala:439)
      	at org.scalatest.concurrent.Eventually.eventually$(Eventually.scala:391)
      	at org.scalatest.concurrent.Eventually$.eventually(Eventually.scala:479)
      	at org.scalatest.concurrent.Eventually.eventually(Eventually.scala:337)
      	at org.scalatest.concurrent.Eventually.eventually$(Eventually.scala:336)
      	at org.scalatest.concurrent.Eventually$.eventually(Eventually.scala:479)
      	at org.apache.spark.sql.kafka010.KafkaTestUtils.setup(KafkaTestUtils.scala:292)
      	at org.apache.spark.sql.kafka010.KafkaDelegationTokenSuite.beforeAll(KafkaDelegationTokenSuite.scala:49)
      	at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
      	at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
      	at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
      	at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:58)
      	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:317)
      	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:510)
      	at sbt.ForkMain$Run$2.call(ForkMain.java:296)
      	at sbt.ForkMain$Run$2.call(ForkMain.java:286)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: sbt.ForkMain$ForkError: org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /brokers/ids
      	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
      	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
      	at kafka.zookeeper.AsyncResponse.resultException(ZooKeeperClient.scala:554)
      	at kafka.zk.KafkaZkClient.getChildren(KafkaZkClient.scala:719)
      	at kafka.zk.KafkaZkClient.getSortedBrokerList(KafkaZkClient.scala:455)
      	at kafka.zk.KafkaZkClient.getAllBrokersInCluster(KafkaZkClient.scala:404)
      	at org.apache.spark.sql.kafka010.KafkaTestUtils.$anonfun$setup$3(KafkaTestUtils.scala:293)
      	at org.scalatest.concurrent.Eventually.makeAValiantAttempt$1(Eventually.scala:395)
      	at org.scalatest.concurrent.Eventually.tryTryAgain$1(Eventually.scala:409)
      	... 20 more
      

        Attachments

        1. consoleText_NOK.txt
          51 kB
          Gabor Somogyi
        2. consoleText_OK.txt
          60 kB
          Gabor Somogyi
        3. unit-tests_NOK.log
          192 kB
          Gabor Somogyi
        4. unit-tests_OK.log
          287 kB
          Gabor Somogyi

          Issue Links

            Activity

              People

              • Assignee:
                gsomogyi Gabor Somogyi
                Reporter:
                kabhwan Jungtaek Lim
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: