[SPARK-49269] Improve performance and memory footprint of all VALUES clauses

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.4
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Costas Zarifis
• Reporter: Costas Zarifis
• Created: 16/Aug/24 21:44
• Updated: 21/Aug/24 00:04
• Resolved: 21/Aug/24 00:04

Description

This is a continuation of the following performance improvement:

https://issues.apache.org/jira/browse/SPARK-48967

 

By pushing the early evaluation of `UnresolvedUnlineTables` into `LocalRelation (whenever possible) in `visitInlineTable` of the AstBuilder we can get the benefits of the aforementioned ticket not only in `INSERT INTO ... VALUES` statements but in every statement that can contain the `VALUES()` clause.

Issue Links

• links to: GitHub Pull Request #47791