[SPARK-18943] Avoid per-record type dispatch in CSV when reading

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 20/Dec/16 12:30
• Updated: 12/Dec/22 17:51
• Resolved: 24/Dec/16 15:34

Description

This is similar with https://issues.apache.org/jira/browse/SPARK-16434

CSVRelation.csvParser does type dispatch for each value in each row.
We can prevent this because the schema is already kept in CSVRelation.

So, converters can be created first according to the schema, and then apply them to each.

Issue Links

• links to: [Github] Pull Request #16351 (HyukjinKwon)