[HIVE-10175] DynamicPartitionPruning lacks a fast-path exit for large IN() queries

• Type: Bug
• Status: Reopened
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.2.0
• Component/s: Physical Optimizer, Tez
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 01/Apr/15 04:17
• Updated: 27/Feb/24 22:23

Description

TezCompiler::runDynamicPartitionPruning() & ppr.PartitionPruner() calls the graph walker even if all tables provided to the optimizer are unpartitioned (or temporary) tables.

This makes it extremely slow as it will walk & inspect a large/complex FilterOperator later in the pipeline.

Issue Links

• is related to: HIVE-11315 Compilation performance issues for large queries