[ZEPPELIN-4177] org.apache.commons:commons-lang3 dependency Hell with Spark 2.4.x

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.8.1
• Fix Version/s: 0.9.0
• Assignee: Unassigned
• Reporter: antonkulaga
• Created: 04/Jun/19 09:24
• Updated: 24/Dec/20 03:16
• Resolved: 05/May/20 08:37

Description

I get a dependency Hell when I try to load any csv with Spark 2.4.3

sparkSession.read
      .option("sep", "\t")
      .option("comment", "#")
      .option("inferSchema", true)
      .option("header", true)
      .option("ignoreLeadingWhiteSpace", true)
      .option("ignoreTrailingWhiteSpace", true)
      .option("ignoreTrailingWhiteSpace", true)
      .option("maxColumns", 150000)
      .csv(pathToTheFile)

I managed to find a workaround by adding org.apache.commons:commons-lang3:3.8.1 to the spark interpreter dependencies, however the true fix will be updating Zeppelin org.apache.commons:commons-lang3 dependency to a newer one.

Issue Links

• is fixed by: ZEPPELIN-4695 High Severity CVE's on zeppeelin docker image