[SPARK-18433] Improve DataSource option keys to be more case-insensitive

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: SQL
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 14/Nov/16 07:39
• Updated: 29/Jan/17 20:40
• Resolved: 16/Nov/16 09:13

Description

Currently, `DataSource.scala` partially use `CaseInsensitiveMap` in code-path.

This issue aims to make `DataSource` to use `CaseInsensitiveMap` generally except passing to the other modules (`InMemoryFileIndex` and `InsertIntoHadoopFsRelationCommand`). They creates new case-sensitive `HadoopConfs` by calling `newHadoopConfWithOptions(options)` inside.

Also, the following options should be consistent.

• CSVOptions
• FileStreamOptions
• JDBCOptions
• JSONOptions
• OrcOptions
• ParquetOptions

Issue Links

• is related to: SPARK-19396 [DOC] Options are case-insensitive since Spark 2.1
• links to: [Github] Pull Request #15884 (dongjoon-hyun)