Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11726

Pushed IN predicates to the metastore

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • None
    • None

    Description

      The PointLookupOptimizer can turn off some of the optimizations due to its use of tuple IN() clauses.

      HIVE-11573 introduced the extraction of sub-clauses that could be pushed down till the TableScan operators, though they wouldn't be pushed down to the metastore.

      In this issue, we tackle this problem by extending the filter parser of the metastore to support IN clauses, including multiple columns. This allows to push those additional predicates down throw directSQL to the metastore.

      Attachments

        1. HIVE-11726.01.patch
          31 kB
          jcamachorodriguez
        2. HIVE-11726.patch
          42 kB
          jcamachorodriguez

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-11634 Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue SPARK-33416 Avoid Hive metastore stack overflow when InSet predicate have many values

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-11573 PointLookupOptimizer can be pessimistic at a low nDV

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB link

          Activity

            People

              jcamacho Jesús Camacho Rodríguez
              jcamacho Jesús Camacho Rodríguez
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: