Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
RedHat Linux 7.3 with Spark 2.1.0 and Hadoop 2.7.3.
-
Sprint 6
Description
Use SparkContext.getOrCreate or SparkSession.builder.getOrCreate in python tests to avoid "Cannot run multiple SparkContexts at once" error observed in some environments running scripts under src/main/python/tests with spark-submit.