[BEAM-4551] Update Spark runner to Spark version 2.3.1

• Type: Improvement
• Status: Resolved
• Priority: P3
• Resolution: Fixed
• Fix Version/s: 2.5.0, 2.6.0
• Component/s: runner-spark
• Assignee: Ismaël Mejía
• Reporter: Ismaël Mejía
• Created: 13/Jun/18 07:39
• Updated: 24/Jul/20 20:14
• Resolved: 13/Jun/18 12:19

Description

The new release of Spark runner brings the classical stability fixes but important for us it brings a fix to accumulators that hitted the runner during the migration to Spark 2 SPARK-23697. Also it aligns the Parquet version with the one we use for Beam.

Issue Links

• links to: GitHub Pull Request #5629
• links to: GitHub Pull Request #5635