[SPARK-25535] Work around bad error checking in commons-crypto

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.2
• Fix Version/s: 2.4.3, 3.0.0
• Component/s: Spark Core
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 26/Sep/18 00:01
• Updated: 27/Apr/19 06:33
• Resolved: 09/Oct/18 14:28

Description

The commons-crypto library used for encryption can get confused when certain errors happen; that can lead to crashes since the Java side thinks the ciphers are still valid while the native side has already cleaned up the ciphers.

We can work around that in Spark by doing some error checking at a higher level.

Issue Links

• links to: [Github] Pull Request #22557 (vanzin)
• links to: GitHub Pull Request #22557
• links to: GitHub Pull Request #24476