[HIVE-222] Group by on a combination of disitinct and non distinct aggregates can return serialization errors with map side aggregations.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.3.0
• Component/s: Query Processor
• Hadoop Flags: Reviewed

Release Note

HIVE-222. Fixed Group by on a combination of disitinct and non distinct aggregates. (Ashish Thusoo via zshao)

• Assignee: Ashish Thusoo
• Reporter: Ashish Thusoo
• Created: 10/Jan/09 15:24
• Updated: 17/Dec/11 00:08
• Resolved: 13/Jan/09 03:53

Description

For queries of the form (groupby2_map.q in the source)

SELECT x, count(DISTINCT y), SUM FROM t GROUP BY x

when map side aggregation is on

hive.map.aggr=true (This is off by default)

The following exception can occur:
[junit] Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Double
[junit] at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeTypeDouble.serialize(DynamicSerDeTypeDouble.java:60)
[junit] at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeFieldList.serialize(DynamicSerDeFieldList.java:235)
[junit] at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeStructBase.serialize(DynamicSerDeStructBase.java:81)
[junit] at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.serialize(DynamicSerDe.java:174)

Issue Links

• blocks: HIVE-179 SUBSTR function should work like other databases
• is duplicated by: HIVE-215 Map-side aggregate with DISTINCT generates bad intermediate table