[PARQUET-1126] make it easy to read and write parquet files in java without depending on hadoop

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Oscar Boykin
• Created: 07/Oct/17 01:43
• Updated: 23/Jun/24 03:29

Description

I am happy to help with this but I'd love some guidance on:

1) likelihood of being accepted as a patch.
2) how critical it is to maintain backwards compatibility in APIs.

For instance, we probably want to introduce a new artifact that lives under the existing hadoop depending artifact, and move as much code as possible to that, keeping the hadoop apis in the old artifact.

Welcome comments on solving this issue.

Issue Links

• blocks: BEAM-4379 Make ParquetIO Read splittable
• incorporates: PARQUET-1142 Avoid leaking Hadoop API to downstream libraries
• is duplicated by: PARQUET-1953 hadoop-common is not an optional dependency
• is related to: PARQUET-1776 Add Java NIO Avro OutputFile InputFile
• links to: GitHub Pull Request #1376