[PARQUET-1020] Add support for Dynamic Messages in parquet-protobuf

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: parquet-protobuf
• Assignee: Alex Buck
• Reporter: Alex Buck
• Created: 06/Jun/17 15:44
• Updated: 15/Sep/22 14:21
• Resolved: 15/Sep/22 14:10

Description

Hello. We would like to pass in a DynamicMessage rather than using the generated protobuf classes to allow us to make our job very generic.

I think this could be achieved by setting the descriptor upfront, similarly to how there is a ProtoParquetOutputFormat today.

In ProtoWriteSupport in the init method it could then generate the parquet schema created by ProtoSchemaConverter using the passed in descriptor, rather than taking it from the generated proto class.

Would there be interest in incorporating this change? If so does the approach above sound sensible? I am happy to do a pull request

initial PR here: https://github.com/apache/parquet-mr/pull/414