[CARBONDATA-2109] config of dataframe load with tempCSV is invalid ,such as QUOTECHAR

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.3.0
• Component/s: data-load
• Assignee: Unassigned
• Reporter: Chenjian Qiu
• Created: 31/Jan/18 14:12
• Updated: 03/Feb/18 13:48
• Resolved: 03/Feb/18 13:48

Description

use datafame to load data with tempCSV true(dataset.writer..format("carbondata").save), carbondata will generate sql to load data
the sql's configs only have SINGLE_PASS, other configs is invalid,such as QUOTECHAR.
the code is in CarbonDataFrameWriter:

private def makeLoadString(csvFolder: String, options: CarbonOption): String = {
val dbName = CarbonEnv.getDatabaseName(options.dbName)(sqlContext.sparkSession)
s"""

LOAD DATA INPATH '$csvFolder'

INTO TABLE $dbName.${options.tableName}

OPTIONS ('FILEHEADER' = '${dataFrame.columns.mkString(",")}',

'SINGLE_PASS' = '${options.singlePass}') """.stripMargin }

Issue Links

• links to: GitHub Pull Request #1899