Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
1.0.0-incubating
-
None
Description
Carbondata-1.0.0 support load data with spark data frame api. There is a limit that kettle is still required since DataFrameLoaderRDD still depends on kettle. We provide NewDataFrameLoaderRDD to load data with new flow .
Also,we discovered some bugs:
1. CarbonMetastoreCatalog.createTableFromThrift
```
/**
- schemaFilePath starts with file:// will not create meta files successfully
- while thriftWriter will have no complains.
- This will cause some weired error eg. No table found.
*/
val thriftWriter = new ThriftWriter(schemaFilePath, false)
thriftWriter.open()
thriftWriter.write(thriftTableInfo)
thriftWriter.close()
```
2. There are some exceptions raised even when you have set useKettle to false.