[ARROW-17498] [CI] Nightly build failure on integration tests with spark master job

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Component/s: Continuous Integration
• Labels: Nightly pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/32759
• Assignee: Raúl Cumplido
• Reporter: Raúl Cumplido
• Created: 22/Aug/22 15:36
• Updated: 11/Jan/23 11:51
• Resolved: 24/Aug/22 10:17

Description

The nightly integration test with spark job is failing:

test-conda-python-3.9-spark-master

with the following error:

 Testing Spark:
org.apache.spark.sql.execution.arrow
org.apache.spark.sql.execution.vectorized.ColumnarBatchSuite
org.apache.spark.sql.execution.vectorized.ArrowColumnVectorSuite
Using `mvn` from path: /spark/build/apache-maven-3.8.6/bin/mvn
[WARNING] format.skipSources set, ignoring main directories
[WARNING] format.skipTestSources set, ignoring validateOnly directories
[WARNING] No sources specified, skipping formatting
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (test) on project spark-parent_2.12: No tests matching pattern "none" were executed! (Set -Dsurefire.failIfNoSpecifiedTests=false to ignore this error.) -> [Help 1]

I can reproduce locally with:

$ PYTHON=3.9 SPARK=master docker-compose build conda-python-spark
$ PYTHON=3.9 SPARK=master docker-compose run --rm -e SETUPTOOLS_SCM_PRETEND_VERSION=9.0.0.dev620 conda-python-spark 

The issue seems to be triggered by this commit on spark master https://github.com/apache/spark/commit/3ed382f391911ec4a79fb204f4986728017dfa4a which updates the version of maven-surefire-plugin to 3.0.0-M7

Issue Links

• links to: GitHub Pull Request #13942