[HIVE-16746] Reduce number of index lookups for same table in IndexWhereTaskDispatcher

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Hadoop Flags: Reviewed
• Assignee: Rajesh Balamohan
• Reporter: Rajesh Balamohan
• Created: 24/May/17 04:33
• Updated: 22/May/18 23:58
• Resolved: 24/May/17 22:35

Description

IndexWhereTaskDispatcher is used when hive.optimize.index.filter=true. It lists all indices for the table and depending on the query complexity, this ends up being in the hotpath. For e.g, Q14 explain plan takes 180-200 seconds and this index querying multiple times for same tables take up 30-40 seconds. This function was invoked around 24000 times for same set of tables.