[SPARK-6561] Add partition support in saveAsParquet

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 1.4.0
• Component/s: SQL
• Assignee: Cheng Lian
• Reporter: Jianshi Huang
• Created: 27/Mar/15 06:21
• Updated: 13/Oct/15 17:53
• Resolved: 13/Oct/15 17:53

Description

Now ParquetRelation2 supports automatic partition discovery which is very nice.

When we save a DataFrame into Parquet files, we also want to have it partitioned.

The proposed API looks like this:

def saveAsParquetFile(path: String, partitionColumns: Seq[String])

Jianshi

Issue Links

• duplicates: SPARK-5182 Partitioning support for tables created by the data source API