[PARQUET-1012] parquet-cpp and parquet-mr version parse inconsistent

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: parquet-cpp
• Assignee: Deepak Majeti
• Reporter: YE
• Created: 31/May/17 08:38
• Updated: 13/Jun/17 18:17
• Resolved: 13/Jun/17 18:17

Description

Spark 2.1 uses parquet-mr(common) 1.8.2 which requires created_by to match certain pattern. I found the following exception when using spark to read parquet file generated by parquet-cpp.

17/05/31 16:33:53 WARN CorruptStatistics: Ignoring statistics because created_by could not be parsed (see PARQUET-251): parquet-cpp version 1.0.0
org.apache.parquet.VersionParser$VersionParseException: Could not parse created_by: parquet-cpp version 1.0.0 using format: (.+) version ((.) )?(build ?(.))

Proposal to fix this issue: set created_by to the certain pattern.

Issue Links

• relates to: PARQUET-1003 [C++] Modify DEFAULT_CREATED_BY value for every new release version