Details
-
Sub-task
-
Status: Closed
-
Minor
-
Resolution: Duplicate
-
None
-
None
-
None
Description
When the DataFusion planner inserts casts, today it relies on special logic to determine the valid coded casts.
The actual arrow cast kernels support a much wider range of data types, and thus DataFusion is artificially limiting the casts it supports for no particularly good reason I can see.
This ticket tracks the work to remove the extra casting checking in the datafusion planner and instead simply rely on runtime check of arrow cast compute kernel
The potential downside of this approach is that the error may be generated later in the execution process (rather than the planner), and possibly have a less specific error message, the upside is there is less code and we get several conversions immediately (like timestamp predicate casting)
I also plan to add DictionaryArray support to the casting kernels and I would like to avoid having to replicate some part of that logic in DataFusion
Attachments
Issue Links
- links to
