[CRUNCH-370] Update Parquet dependency in Crunch pom

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0, 0.8.2
• Fix Version/s: 0.10.0, 0.8.3
• Component/s: IO
• Assignee: Micah Whitacre
• Reporter: Anandsagar Kothapalli
• Created: 25/Mar/14 14:36
• Updated: 20/Jun/14 03:49
• Resolved: 26/Mar/14 13:31

Description

Currently crunch is supporting avro to parquet conversion using AvroParquetFileTarget, AvroParquetFileSource classes. When I used these classes to convert avro to parquet files, I got the following exception in some cases: "org.apache.crunch.CrunchRuntimeException: parquet.io.ParquetEncodingException: empty fields are illegal, the field should be ommited completely instead"

After further debugging I found out that this issue is related to AvroWriteSupport class in parquet, which was fixed as part of milestone 1.2.3 https://github.com/Parquet/parquet-mr/issues/162. Latest parquet version is 1.3.2.

But crunch is still using parquet 1.2.0 https://github.com/apache/crunch/blob/master/pom.xml#L77
As part of this improvement, parquet dependency version in crunch will be updated if not to latest then at least to 1.2.3