Description
I currently use Databrick's spark-csv lib but some features don't work with Apache Spark 2.0.0-SNAPSHOT. I understand that with the addition of CSV support into spark-sql directly, that spark-csv won't be modified.
I currently read some CSV data that has been pre-processed and is in RDD[String] format.
There is sqlContext.read.json(rdd: RDD[String]) but other formats don't appear to support the creation of DataFrames based on loading from RDD[String].
Attachments
Issue Links
- is related to
-
SPARK-15615 Support for creating a dataframe from JSON in Dataset[String]
-
- Resolved
-
-
SPARK-22505 toDF() / createDataFrame() type inference doesn't work as expected
-
- Resolved
-
- links to
