Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Done
-
2.0.0
-
None
-
None
Description
This was suggested in https://github.com/apache/spark/pull/12268
CSV data sources is pretty much similar with JSON data source. However, currently the structure is pretty different.
It would be great if it is refactored so that this to be consistent with JSON data source and we can fix some common issues together easily.
In addition, it seems there are some factors to improve comparing JSON data sources.
This JIRA ticket includes several sub-tasks about refactoring and performance improvement.
The main purpose of this JIRA is to tie up several sub-tasks found in the PR above.