[SPARK-5357] Upgrade from commons-codec 1.5

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.1.0, 1.2.0
• Fix Version/s: 1.3.0
• Component/s: Spark Core
• Assignee: Matt Whelan
• Reporter: Matthew Whelan
• Created: 21/Jan/15 20:54
• Updated: 16/Feb/15 23:06
• Resolved: 16/Feb/15 23:06

Description

Spark uses commons-codec 1.5, which has a race condition in Base64. That race was introduced in commons-codec 1.4 and resolved in 1.7. The current version of commons-codec is 1.10.

Code that runs in Workers and assumes that Base64 is thread-safe will break because spark is using a non-thread-safe version. See CODEC-96

In addition, the spark.files.userClassPathFirst mechanism is currently broken, (bug to come), so there isn't a viable work around for this issue.

Issue Links

• links to: [Github] Pull Request #4153 (MattWhelan)