[BEAM-11527] Support user configurable Hadoop Configuration flags for ParquetIO

• Type: Improvement
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.28.0
• Component/s: io-java-parquet
• Labels: parquet
• Flags: Patch
• Assignee: Anant Damle
• Reporter: Anant Damle
• Created: 27/Dec/20 17:29
• Updated: 13/Apr/23 11:04
• Resolved: 31/Dec/20 13:59

Description

Provide an user configurable interface to provide/input Hadoop configuration for ParquetIO.

Current behaviour supports only {code}AvroReadSupport.AVRO_COMPATIBILITY{code} flag.

 

There are now more options supported for reader as introduced through PARQUET-1928 and PR/831

Issue Links

• is related to: PARQUET-1928 Interpret Parquet INT96 type as FIXED[12] AVRO Schema
• is related to: BEAM-10284 Allow to pass config to Parquet sink
• relates to: BEAM-11913 Add support for Hadoop configuration on ParquetIO
• links to: Commit#7bba60d
• links to: GitHub Pull Request #13619