[PARQUET-197] parquet-cascading and the mapred API does not create metadata file

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Tim
• Reporter: Tim
• Created: 03/Mar/15 23:27
• Updated: 07/Apr/15 20:45
• Resolved: 13/Mar/15 20:37

Description

Repro: run a scalding job that writes parquet files to a folder. no _metadata and _common_metadata file is created

Impact: potential performance problem if parquet metadata is read from client side, which is the case for sparkSQL

casue: the metatdata writing logic is in the mapreduce API but not the mapred API of parquet.

Issue Links

• is a clone of: PARQUET-206 MapredParquetOutputCommitter does not work in hadoop2