[FLINK-28756] Can't write nested object to parquet flink 1.15.1

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.15.1
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / API
• Labels: Flink Flink-parquet Flink-table-api TableAPI auto-deprioritized-major flink-formats

Environment

Local running and Flink on docker

• Flags: Important

Language

JAVA java

• Assignee: Unassigned
• Reporter: Jonathan diamant
• Created: 31/Jul/22 12:25
• Updated: 21/Aug/23 10:35

Description

Hey, I'm trying to write nested objects to parquet files with Table api (by specifying the schema). I did manage to write flattened objects to parquet files. The objects I want to write contain array of objects. The schema I used is 

 

 DataTypes. ARRAY(DataTypes.ROW(

DataTypes.FIELD(...),...))

 

When I tried to run it locally I got the exception:

 

empty fields are illegal, the field should be ommited completely instead

 

And when I debugged the code I saw that the method 

 Public void write(Array Data array Data, int ordinal) 

 

 

In the class RowWriter in ParquetRowDataWriter is not implemented and thus the content of the array cannot be written.

 

I have wondered if the method can be implemented or what it takes to implement it. 

(I have thought that this can be solved by call the write(RowData row) method of the same class with the correct parameters from the arrayData).