[SPARK-2360] CSV import to SchemaRDDs

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Component/s: SQL
• Target Version/s: 1.2.0
• Assignee: Hossein Falaki
• Reporter: Michael Armbrust
• Created: 04/Jul/14 00:15
• Updated: 07/Nov/14 22:16
• Resolved: 03/Nov/14 22:56

Description

I think the first step it to design the interface that we want to present to users. Mostly this is defining options when importing. Off the top of my head:

• What is the separator?
• Provide column names or infer them from the first row.
• how to handle multiple files with possibly different schemas
• do we have a method to let users specify the datatypes of the columns or are they just strings?
• what types of quoting / escaping do we want to support?

Issue Links

• is duplicated by: SPARK-2378 Implement functionality to read csv files
• relates to: SPARK-2179 Public API for DataTypes and Schema
• links to: [Github] Pull Request #1351 (falaki)