[HIVE-18079] Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0
• Component/s: Standalone Metastore, Statistics
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 16/Nov/17 05:28
• Updated: 31/Jul/18 01:33
• Resolved: 08/Jun/18 06:27

Description

HyperLogLog can merge a 14 bit HLL into a 10 bit HLL bitset, because of its mathematical hash distribution & construction.

Allow the squashing of a 14 bit HLL -> 10 bit HLL without needing a second scan over the data-set.

Sub-Tasks

• Statistics: switch to 10bit HLL by default for Hive: Resolved