[IMPALA-11745] Add Hive's ESRI geospatial functions as builtins

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: Impala 4.3.0
• Component/s: Frontend
• Labels: geospatial
• Epic Color: ghx-label-9
• Assignee: Peter Rozsa
• Reporter: Csaba Ringhofer
• Created: 24/Nov/22 15:57
• Updated: 31/Jul/24 15:08
• Resolved: 08/Feb/23 14:06

Description

Hive has added the Esri Hive UDFs as builtin functions in HIVE-26223.
The original Hive Esri source: https://github.com/Esri/spatial-framework-for-hadoop

A simple (but inefficient) way to add postgis style geospatial support in Impala is to import these functions through Hive and add them as builtins. This is possible as Impala supports Hive UDFs, but is much slower than a native implementation could be.

My plan is to add these functions depending on a flag, e.g. uf geospatial_library=hive_esri. This would allow native implementations to be added later and let users choose between Hive compatibility and speed. It could also make sense override only the more critical functions with native ones, but it may turn out that the binary format used in ESRI is suboptimal for a native implementation.

Issue Links

• is blocked by: IMPALA-11549 Incorrent return type mismatch in Hive generic UDFs
• is blocked by: IMPALA-11746 Support legacy Hive UDFs with variable length argument list
• is blocked by: IMPALA-11718 Improve support for Hive Generic UDFs
• is related to: IMPALA-13263 Single argument overload is missing for st_convexhull()
• relates to: IMPALA-11861 Impala*Writable methods are not overridden to handle the heap-stored values
• relates to: IMPALA-11854 ImpalaStringWritable's underlying array can't be changed in UDFs
• relates to: IMPALA-11918 Fix test_java_udfs_from_impala after IMPALA-11745

Sub-Tasks

• Add ESRI geospatial function's test cases as Impala tests: Resolved
• Document changes introduced by Hive's ESRI geospatial functions: Open