[SPARK-46482] Revert SPARK-43049 due to performance regression of using CLOB

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.5.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Ivan Sadikov
• Created: 21/Dec/23 21:20
• Updated: 22/Dec/23 00:07
• Resolved: 22/Dec/23 00:07

Description

SPARK-43049 causes performance regression when writing string fields to an Oracle database due to strings written as CLOB instead of VARCHAR2. CLOB is known to have bad performance in Oracle so when creating a table and writing to it using Spark, the internal SQL statement that writes 20+ string fields would take at least 5x performance hit from the original patch (2 min vs 10+ min).

I confirmed internally that running a job with the commit reverted brings back the original performance numbers.

 

 

 

Issue Links

• duplicates: SPARK-46478 Revert SPARK-43049
• links to: GitHub Pull Request #44452