[SPARK-23580] Interpreted mode fallback should be implemented for all expressions & projections

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.0
• Component/s: SQL
• Labels: bulk-closed release-notes
• Assignee: Unassigned
• Reporter: Herman van Hövell
• Created: 05/Mar/18 12:43
• Updated: 25/May/21 01:49
• Resolved: 25/May/21 01:40

Description

Spark SQL currently does not support interpreted mode for all expressions and projections. This is a problem for scenario's where were code generation does not work, or blows past the JVM class limits. We currently cannot gracefully fallback.

This ticket is an umbrella to fix this class of problem in Spark SQL. This work can be divided into two main area's:

• Add interpreted versions for all dataset related expressions.
• Add an interpreted version of GenerateUnsafeProjection.