[PARQUET-1928] Interpret Parquet INT96 type as FIXED[12] AVRO Schema

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.12.0
• Component/s: parquet-avro
• Labels: patch
• Flags: Patch
• Assignee: Anant Damle
• Reporter: Anant Damle
• Created: 19/Oct/20 07:16
• Updated: 23/Jun/24 03:32
• Resolved: 07/Dec/20 09:48

Description

Reading Parquet files in Apache Beam using ParquetIO uses `AvroParquetReader` causing it to throw `IllegalArgumentException("INT96 not implemented and is deprecated")`

Customers have large datasets which can't be reprocessed again to convert into a supported type. An easier approach would be to convert into a byte array of 12 bytes, that can then be interpreted by the developer in any way they want to interpret it.

Issue Links

• is related to: PARQUET-323 INT96 should be marked as deprecated
• relates to: BEAM-11527 Support user configurable Hadoop Configuration flags for ParquetIO
• links to: GitHub Pull Request #831