[SPARK-26096] k8s integration tests should run R tests

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.0.0
• Component/s: Kubernetes, Spark Core, Tests
• Assignee: Unassigned
• Reporter: Marcelo Masiero Vanzin
• Created: 16/Nov/18 23:55
• Updated: 17/May/20 18:25
• Resolved: 28/Nov/18 00:25

Description

Noticed while debugging a completely separate things.

• the jenkins job doesn't enable the SparkR profile
• KubernetesSuite doesn't include the RTestsSuite trait

even if you fix those two, it seems the tests are broken:

[info] - Run SparkR on simple dataframe.R example *** FAILED *** (2 minutes, 3 seconds)
[info]   at org.scalatest.concurrent.Eventually.eventually(Eventually.scala:308)
[info]   at org.scalatest.concurrent.Eventually.eventually$(Eventually.scala:307)
[info]   at org.scalatest.concurrent.Eventually$.eventually(Eventually.scala:479)
[info]   at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.runSparkApplicationAndVerifyCompletion(KubernetesSuite.scala:274)
[info]   at org.apache.spark.deploy.k8s.integrationtest.RTestsSuite.$anonfun$$init$$1(RTestsSuite.scala:26)
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)

Issue Links

• duplicates: SPARK-25152 Enable Spark on Kubernetes R Integration Tests