[FLINK-21389] ParquetInputFormat should not need parquet schema as user input

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Fix Version/s: 1.13.2
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
• Labels: pull-request-available
• Assignee: Etienne Chauchot
• Reporter: Etienne Chauchot
• Created: 17/Feb/21 10:04
• Updated: 28/Aug/21 12:18
• Resolved: 15/Jun/21 06:01

Description

ParquetInputFormat takes parquet schema as user input but after split it reads the parquet schema again here https://github.com/apache/flink/blob/52dcf439bb0b8d613fff1efecf015052d5b3a10b/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/ParquetInputFormat.java#L170
it should read the provided user schema.
But better would be to read the schema automatically and not require the user to provide a schema as spark does (https://spark.apache.org/docs/latest/sql-data-sources-parquet.html).
Thus we could add a ParquetInputFormat constructor and allow ParquetTableSource with no schema parameter

Issue Links

• links to: GitHub Pull Request #15725