[IMPALA-9962] Implement ds_kll_quantiles() function

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Epic Link: unlabelled-IMPALA-9594
• Epic Color: ghx-label-7
• Assignee: Gabor Kaszab
• Reporter: Gabor Kaszab
• Created: 16/Jul/20 08:10
• Updated: 26/Aug/20 13:43
• Resolved: 26/Aug/20 13:43

Description

Requirements for ds_kll_quantiles()

• Receives a serialized KLL sketch in BINARY type (in Impala it can be STRING as long as we don't have BINARY) as first parameter.
• Receives one or more double values to represent the quantile points.
• In Hive the return type is an array of doubles. However, Impala can't return complex types from functions at this point so we have to find some alternative approaches to implement this function.
  • One would be to return as many columns as many quantile points were given.
  • Another approach is to create a comma separated string from the results of this function and return that string instead of an array.

Hive example:

select ds_kll_quantiles(ds_kll_sketch(cast(int_col as float)), 0, 0.1, 0.5, 1) from table_name
+--------------------+
|        _c0         |
+--------------------+
| [1.0,1.0,1.0,1.0]  |
+--------------------+