[SPARK-42854] Jackson 2.15

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.4.1
• Component/s: Input/Output
• Assignee: Unassigned
• Reporter: PJ Fanning
• Created: 19/Mar/23 10:49
• Updated: 05/May/23 18:13
• Resolved: 05/May/23 18:13

Description

I'm not advocating for an upgrade to Jackson 2.15. 2.15.0-rc1 has just been released and 2.15.0 should be out soon.

There are some security focused enhancements including a new class called StreamReadConstraints. The defaults on StreamReadConstraints are pretty high but it is not inconceivable that some Spark users might need to relax them. Parsing large strings as numbers is sub-quadratic, thus the default limit of 1000 chars or bytes (depending on input context).

When the Spark team consider upgrading to Jackson 2.15 or above, you might also want to consider adding some way for users to configure the StreamReadConstraints.

Issue Links

• is duplicated by: SPARK-43263 Upgrade FasterXML jackson to 2.15.0