[BIGTOP-3579] Spark smoke test fails on Debian 9 and Ubuntu 18.04

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: debian, spark, tests
• Assignee: Kengo Seki
• Reporter: Kengo Seki
• Created: 13/Aug/21 06:38
• Updated: 13/Aug/21 09:48
• Resolved: 13/Aug/21 08:18

Description

https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/COMPONENTS=hdfs.yarn.spark@spark,OS=debian-9-amd64-deploy/258/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/COMPONENTS=hdfs.yarn.spark@spark,OS=debian-9-arm64-deploy/259/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/COMPONENTS=hdfs.yarn.spark@spark,OS=ubuntu-18.04-amd64-deploy/258/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/COMPONENTS=hdfs.yarn.spark@spark,OS=ubuntu-18.04-arm64-deploy/259/console

On these distros, the default version of R is 3.3.3 but SparkR requires 3.5.0+.
The bigtop/slaves image has R 3.6.3, but the smoke test runs on bigtop/puppet, so Spark's smoke test fails on them.

Issue Links

• links to: https://github.com/apache/bigtop/pull/808