Details

    • Hadoop Flags:
      Reviewed

      Description

      For partition predicates that cannot be pushed down to JDO filtering (HIVE-2049), we should fall back to the old approach of listing all partition names first and use Hive's expression evaluation engine to select the correct partitions. Then the partition pruner should hand Hive a list of partition names and return a list of Partition Object (this should be added to the Hive API).

      A possible optimization is that the the partition pruner should give Hive a set of ranges of partition names (say [ts=01, ts=11], [ts=20, ts=24]), and the JDO query should be formulated as range queries. Range queries are possible because the first step list all partition names in sorted order. It's easy to come up with a range and it is guaranteed that the JDO range query results should be equivalent to the query with a list of partition names.

      1. HIVE-2050.patch
        161 kB
        Ning Zhang
      2. HIVE-2050.4.patch
        166 kB
        Ning Zhang
      3. HIVE-2050.3.patch
        166 kB
        Ning Zhang
      4. HIVE-2050.2.patch
        170 kB
        Ning Zhang

        Activity

        Ning Zhang created issue -
        Ning Zhang made changes -
        Field Original Value New Value
        Assignee Ning Zhang [ nzhang ]
        Ning Zhang made changes -
        Attachment HIVE-2050.patch [ 12474360 ]
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ning Zhang made changes -
        Attachment HIVE-2050.2.patch [ 12474754 ]
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ning Zhang made changes -
        Attachment HIVE-2050.3.patch [ 12474799 ]
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ning Zhang made changes -
        Attachment HIVE-2050.4.patch [ 12474846 ]
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Fix Version/s 0.8.0 [ 12316178 ]
        Component/s Metastore [ 12312584 ]
        Component/s Query Processor [ 12312586 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Carl Steinbach made changes -
        Labels PartitionPruner

          People

          • Assignee:
            Ning Zhang
            Reporter:
            Ning Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development