Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10175

DynamicPartitionPruning lacks a fast-path exit for large IN() queries

Log workAgile BoardRank to TopRank to BottomAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.2.0
    • Fix Version/s: None
    • Component/s: Physical Optimizer, Tez
    • Labels:
      None

      Description

      TezCompiler::runDynamicPartitionPruning() & ppr.PartitionPruner() calls the graph walker even if all tables provided to the optimizer are unpartitioned (or temporary) tables.

      This makes it extremely slow as it will walk & inspect a large/complex FilterOperator later in the pipeline.

        Attachments

        1. HIVE-10175.1.patch
          1 kB
          Gopal Vijayaraghavan
        2. HIVE-10175.profile.html
          928 kB
          Gopal Vijayaraghavan

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              gopalv Gopal Vijayaraghavan Assign to me
              Reporter:
              gopalv Gopal Vijayaraghavan

              Dates

              • Created:
                Updated:

                Issue deployment