[DRILL-4119] Skew in hash distribution for varchar (and possibly other) types of data

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.0
• Component/s: Functions - Drill
• Assignee: Aman Sinha
• Reporter: Aman Sinha
• Reviewer: Victoria Markman
• Created: 21/Nov/15 19:05
• Updated: 11/Dec/15 23:19
• Resolved: 01/Dec/15 01:04

Description

We are seeing substantial skew for an Id column that contains varchar data of length 32. It is easily reproducible by a group-by query:

Explain plan for SELECT SomeId From table GROUP BY SomeId;
...
01-02          HashAgg(group=[{0}])
01-03            Project(SomeId=[$0])
01-04              HashToRandomExchange(dist0=[[$0]])
02-01                UnorderedMuxExchange
03-01                  Project(SomeId=[$0], E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($0))])
03-02                    HashAgg(group=[{0}])
03-03                      Project(SomeId=[$0])

The string id happens to be of the following type:

e4b4388e8865819126cb0e4dcaa7261d