[HIVE-3554] Hive List Bucketing - Query logic

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.10.0
• Hadoop Flags: Reviewed
• Assignee: Gang Tim Liu
• Reporter: Gang Tim Liu
• Created: 08/Oct/12 21:41
• Updated: 02/May/13 02:30
• Resolved: 02/Nov/12 11:20

Description

This is part of efforts for list bucketing feature: https://cwiki.apache.org/Hive/listbucketing.html

This patch includes:
1. Query logic: hive chooses right sub-directory instead of partition directory.
2. alter table grammar which is required to support query logic

This patch doesn't include list bucketing DML. Main reasons:
1. risk. w/o DML, this patch won't impact any existing hive regression features since no touch on any data manipulation so that very low risk.
2. manageability. w/ DML, patch is getting bigger and hard to review. Removing DML, it's easy to review.

We still disable hive feature by default since DML is not in yet.

DML will be in follow-up patch.

Issue Links

• depends upon: HIVE-3451 map-reduce jobs does not work for a partition containing sub-directories
• depends upon: HIVE-3556 Test "Path -> Alias" for explain extended
• depends upon: HIVE-3514 Refactor Partition Pruner so that logic can be reused.
• is related to: HIVE-3641 Hive List Bucketing - Query logic supports <>
• is related to: HIVE-3567 Hive List Bucketing - Query logic supports <, <=, > , >= and in
• is related to: HIVE-3642 Hive List Bucketing - optimize transform()
• is related to: HIVE-3026 List Bucketing in Hive
• is related to: HIVE-3072 Hive List Bucketing - DDL support
• is related to: HIVE-3073 Hive List Bucketing - DML support
• is related to: HIVE-3643 Hive List Bucketing - set hive.mapred.supports.subdirectories
• is related to: HIVE-3566 Hive List Bucketing - Query Logic Optimization