[SPARK-22435] Support processing array and map type using script

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.2.0
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Jin Xing
• Created: 03/Nov/17 10:19
• Updated: 21/May/19 04:16
• Resolved: 21/May/19 04:16

Description

Currently, It is not supported to use script(e.g. python) to process array type or map type, it will complain with below message:
org.apache.spark.sql.catalyst.expressions.UnsafeArrayData cannot be cast to [Ljava.lang.Object
org.apache.spark.sql.catalyst.expressions.UnsafeMapData cannot be cast to java.util.Map
Could we support it?

Issue Links

• links to: [Github] Pull Request #19652 (jinxing64)