[SPARK-26583] Add `paranamer` dependency to `core` module

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0, 3.0.0
• Fix Version/s: 2.4.1, 3.0.0
• Component/s: Build, Spark Core
• Target Version/s: 2.4.1, 3.0.0
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 09/Jan/19 22:48
• Updated: 08/Feb/19 08:47
• Resolved: 10/Jan/19 08:45

Description

With Scala-2.12 profile, Spark application fails while Spark is okay. For example, our documented `SimpleApp` example succeeds to compile but it fails at runtime because it doesn't use `paranamer 2.8` and hits SPARK-22128.

https://dist.apache.org/repos/dist/dev/spark/3.0.0-SNAPSHOT-2019_01_09_13_59-e853afb-docs/_site/quick-start.html

$ mvn dependency:tree -Dincludes=com.thoughtworks.paranamer
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ simple ---
[INFO] my.test:simple:jar:1.0-SNAPSHOT
[INFO] \- org.apache.spark:spark-sql_2.12:jar:3.0.0-SNAPSHOT:compile
[INFO]    \- org.apache.spark:spark-core_2.12:jar:3.0.0-SNAPSHOT:compile
[INFO]       \- org.apache.avro:avro:jar:1.8.2:compile
[INFO]          \- com.thoughtworks.paranamer:paranamer:jar:2.7:compile

Issue Links

• is duplicated by: SPARK-26819 ArrayIndexOutOfBoundsException while loading a CSV to a Dataset with dependencies spark-core_2.12 and spark-sql_2.12 (with spark-core_2.11 and spark-sql_2.11 : working fine)
• relates to: SPARK-22128 Update paranamer to 2.8 to avoid BytecodeReadingParanamer ArrayIndexOutOfBoundsException with Scala 2.12 + Java 8 lambda