[PARQUET-660] Writing Protobuf messages with extensions results in an error or data corruption.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.1
• Fix Version/s: 1.9.0, 1.8.2
• Component/s: parquet-mr
• Flags: Patch
• Assignee: Jakub Kukul
• Reporter: Jakub Kukul
• Created: 10/Jul/16 20:20
• Updated: 23/Jun/24 03:28
• Resolved: 08/Sep/16 21:50

Description

Currently, in ProtoParquetWriter there's not any support for extended protobuf messages. An attempt to write a protobuf message with extension either:

• Results in an uninformative error like this:

  parquet.proto.ProtoWriteSupport: Cannot write message 
  YOUR_EXTENSION_TYPE cannot be cast to A_BASE_FIELD_TYPE : BASE_FIELD_NAME 
  

• Results in a data corruption, if an extension type is compatible with a base field type with a corresponding index.

Imo there are two possible solutions:

1. Ignore extension fields.
2. When an extension field is detected in a message, throw an informative error.