[HIVE-7956] When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Spark
• Assignee: Rui Li
• Reporter: Rui Li
• Created: 03/Sep/14 08:36
• Updated: 15/Sep/14 06:25
• Resolved: 15/Sep/14 06:25

Description

I created a bucketed table:

create table testBucket(x int,y string) clustered by(x) into 10 buckets;

Then I run a query like:

set hive.enforce.bucketing = true;
insert overwrite table testBucket select intCol,stringCol from src;

Here src is a simple textfile-based table containing 40000000 records (not bucketed). The query launches 10 reduce tasks but all the data goes to only one of them.

Issue Links

• depends upon: HIVE-8017 Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch]
• is part of: HIVE-7292 Hive on Spark