[PARQUET-1647] [Java] support for Arrow's float16

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Component/s: parquet-format, parquet-mr, parquet-thrift
• Labels: pull-request-available
• Assignee: Jiashen Zhang
• Reporter: The Alchemist
• Created: 03/Sep/19 19:09
• Updated: 20/Dec/23 08:07
• Resolved: 09/Dec/23 16:15

Description

DESCRIPTION

 

I'm wondering if there's any interest in supporting Arrow's float16 type in Parquet.

There seem to be one or two float16 / halffloat tickets here (e.g., PARQUET-1403) but nothing that speaks to adding half-float support to Parquet in-general.

 

PLANS

I'm able to spend some time on this, if someone points me  in the right direction.

 

1. Add the HALFFLOAT or FLOAT16 enum (any preferred naming convention?) to https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L32
2. Add HALFFLOAT to org.apache.parquet.schema.PrimitiveType
3. Add HALFFLOAT support to org.apache.parquet.arrow.schema.SchemaConverter
4. Add encoding for new type at org.apache.parquet.column.Encoding
5. ??

If anyone has any interest in this, pointers, or comments, they would be greatly appreciated!

Issue Links

• depends upon: PARQUET-758 [Format] HALF precision FLOAT Logical type
• links to: GitHub Pull Request #1142
• links to: GitHub Pull Request #1235