[SPARK-6126] Support UDTs in JSON

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.5.0
• Component/s: SQL
• Target Version/s: 1.5.0
• Assignee: Nathan Howell
• Reporter: Joseph K. Bradley
• Shepherd: Yin Huai
• Created: 03/Mar/15 00:27
• Updated: 17/Sep/15 20:32
• Resolved: 04/Aug/15 00:47

Description

This would be nice to do (in Python):

from pyspark.mllib.util import MLUtils
data = MLUtils.loadLibSVMFile(sc, "data/mllib/sample_libsvm_data.txt")
data.toDF().toJSON().count()

But UDTs are not yet supported in JSON. It would be nice to add.

Issue Links

• is related to: SPARK-10632 Cannot save DataFrame with User Defined Types
• is related to: SPARK-9581 Add test for JSON UDTs
• links to: [Github] Pull Request #6193 (drubbo)
• links to: [Github] Pull Request #7416 (drubbo)