[CARBONDATA-188] Compress CSV file while loading

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.1.0-incubating
• Fix Version/s: 0.2.0-incubating, 0.1.1-incubating
• Component/s: core, spark-integration
• Assignee: Jacky Li
• Reporter: Jacky Li
• Created: 29/Aug/16 09:09
• Updated: 24/Sep/16 10:54
• Resolved: 06/Sep/16 15:46

Description

Currently when loading CarbonData file using Spark Dataframe API, it will firstly save as CSV file then load to CarbonData file.

Sometimes CSV requires a lot of disk space, suggest to compress the CSV before loading, thus disk space required is less.

Issue Links

• links to: GitHub Pull Request #104