[SPARK-28770] Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.3
• Fix Version/s: 3.0.0
• Component/s: Spark Core

Environment

Community jenkins and our arm testing instance.

• Assignee: Wing Yew Poon
• Reporter: huangtianhua
• Created: 19/Aug/19 02:29
• Updated: 19/Nov/19 01:32
• Resolved: 05/Sep/19 20:56

Description

Test

org.apache.spark.scheduler.ReplayListenerSuite.End-to-end replay with compression is failed  see https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-3.2/267/testReport/junit/org.apache.spark.scheduler/ReplayListenerSuite/End_to_end_replay_with_compression/

 

And also the test is failed on arm instance, I sent email to spark-dev before, and we suspect there is something related with the commit https://github.com/apache/spark/pull/23767, we tried to revert it and the tests are passed:

ReplayListenerSuite:
       - ...
       - End-to-end replay *** FAILED ***
         "[driver]" did not equal "[1]" (JsonProtocolSuite.scala:622)
       - End-to-end replay with compression *** FAILED ***
         "[driver]" did not equal "[1]" (JsonProtocolSuite.scala:622) 

 

Not sure what's wrong, hope someone can help to figure it out, thanks very much.

Issue Links

• is part of: SPARK-29106 Add jenkins arm test for spark
• links to: GitHub Pull Request #25659
• links to: GitHub Pull Request #25673