From my StackOverflow in relation to an issue I'm having with getting Snowflake (Cloud DB) to load Parquet files written with version 1.11.0
The problem only appears when using a map schema field in the Avro schema. For example:
When using Parquet-Avro to write the file, a bad Parquet schema ends up with, for example
From the great answer to my StackOverflow, it seems the issue is that the 1.11.0 Parquet-Avro is still using the legacy MAP_KEY_VALUE converted type, that has no logical type equivalent. From the comment on LogicalTypeAnnotation
However, it seems this is being written with the latest 1.11.0, which then causes Apache Arrow to fail with
As it appears that Arrow only looks for the new logical type of Map or List, therefore this causes an error.
I have seen in Parquet Formats that LogicalTypes should be something like
Is this on the correct path?