Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Consider the following query:
create table smb_bucket_1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 6 BUCKETS STORED AS TEXTFILE;
create table smb_bucket_2(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 6 BUCKETS STORED AS TEXTFILE;
– load the above tables
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
explain
select count
from
(
select /+mapjoin(a)/ a.key as key1, b.key as key2, a.value as value1, b.value as value2
from smb_bucket_1 a join smb_bucket_2 b on a.key = b.key)
subq;
The above query does not use sort-merge join. This would be very useful as we automatically convert the queries to use sorting and bucketing properties for join.