[BEAM-11861] ParquetIO throws Coder not found when using parseGenericRecord or parseFilesGenericRecord

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.28.0
• Fix Version/s: 2.29.0
• Component/s: io-java-parquet
• Assignee: Anant Damle
• Reporter: Anant Damle
• Created: 25/Feb/21 02:47
• Updated: 16/Mar/21 08:50
• Resolved: 16/Mar/21 08:50

Description

ParquetIO is missing output coder  when  using a user-specified parsing function with `parseGenericRecord` or `parseFilesGenericRecord` feature for reading Parquet files with unknown schema.

Workaround:
Use `setCoder` directly on the output `PCollection`
for example:

SerializableFunction<GenericRecord, Foo> parseFn = ...;
Coder<Foo> fooCoder = ...;
PCollection<Foo> records = p.apply(ParquetIO.parseGenericRecords(parseFn).from(...))
.setCoder(fooCoder);

Issue Links

• links to: GitHub Pull Request #14078