[SPARK-11011] UserDefinedType serialization should be strongly typed

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.5.0, 1.5.1
• Fix Version/s: 2.0.0
• Component/s: SQL
• Labels: UDT
• Epic Link: UserDefinedType enhancements
• Assignee: Jakob Odersky
• Reporter: John Muller
• Created: 08/Oct/15 19:47
• Updated: 16/Mar/16 23:59
• Resolved: 16/Mar/16 23:59

Description

UDT's serialize method takes an Any rather than the actual type parameter. The issue lies in CatalystTypeConverters convertToCatalyst(a: Any): Any method, which pattern matches against a hardcoded list of built-in SQL types.

Planned fix is to allow the UDT to supply the CatalystTypeConverter to use via a new public method on the abstract class UserDefinedType that allows the implementer to strongly type those conversions

Issue Links

• links to: [Github] Pull Request #11379 (jodersky)