[IMPALA-9482] Support BINARY columns

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: Impala 4.2.0
• Component/s: Backend, Frontend
• Epic Link: DataSketches
• Epic Color: ghx-label-4
• Assignee: Csaba Ringhofer
• Reporter: Csaba Ringhofer
• Created: 09/Mar/20 17:28
• Updated: 26/Aug/22 13:46
• Resolved: 19/Aug/22 15:32

Description

Hive's binary data types seem very similar to strings with some exceptions:

• no implicit/explicit cast to any type except string
• bit different built-in functions than for string, but some functions accept both string and binary (e.g. concat)
• written to text files with base64 encoding
  (see https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF )

I didn't look around about client support yet.

Issue Links

• is related to: IMPALA-11340 Support BINARY in persistent legacy Java UDFs
• relates to: IMPALA-5323 Support Kudu BINARY

Sub-Tasks

• Add basic BINARY support: Resolved
• Add built-in functions for BINARY columns: Resolved
• Read/write support for BINARY in Parquet: Resolved
• Read support for BINARY in ORC: Resolved
• Complete builtin aggregate support for BINARY: Open
• Codegen WriteSlot for BINARY columns in TextConverter: Open
• Support BINARY nested in complex types in select list: Resolved
• BINARY support in Iceberg: Open