Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15269

Dynamic Min-Max/BloomFilter runtime-filtering for Tez

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.2.0
    • Tez

    Description

      If a dimension table and fact table are joined:

      select *
      from store join store_sales on (store.id = store_sales.store_id)
      where store.s_store_name = 'My Store'
      

      One optimization that can be done is to get the min/max store id values that come out of the scan/filter of the store table, and send this min/max value (via Tez edge) to the task which is scanning the store_sales table.
      We can add a BETWEEN(min, max) predicate to the store_sales TableScan, where this predicate can be pushed down to the storage handler (for example for ORC formats). Pushing a min/max predicate to the ORC reader would allow us to avoid having to entire whole row groups during the table scan.

      Attachments

        1. HIVE-15269.1.patch
          219 kB
          Deepak Jaiswal
        2. HIVE-15269.10.patch
          6.82 MB
          Deepak Jaiswal
        3. HIVE-15269.11.patch
          3.20 MB
          Deepak Jaiswal
        4. HIVE-15269.12.patch
          4.52 MB
          Deepak Jaiswal
        5. HIVE-15269.13.patch
          274 kB
          Deepak Jaiswal
        6. HIVE-15269.14.patch
          246 kB
          Deepak Jaiswal
        7. HIVE-15269.15.patch
          258 kB
          Deepak Jaiswal
        8. HIVE-15269.16.patch
          369 kB
          Deepak Jaiswal
        9. HIVE-15269.17.patch
          309 kB
          Deepak Jaiswal
        10. HIVE-15269.18.patch
          373 kB
          Deepak Jaiswal
        11. HIVE-15269.19.patch
          373 kB
          Deepak Jaiswal
        12. HIVE-15269.2.patch
          218 kB
          Deepak Jaiswal
        13. HIVE-15269.3.patch
          219 kB
          Deepak Jaiswal
        14. HIVE-15269.4.patch
          221 kB
          Deepak Jaiswal
        15. HIVE-15269.5.patch
          240 kB
          Deepak Jaiswal
        16. HIVE-15269.6.patch
          246 kB
          Deepak Jaiswal
        17. HIVE-15269.7.patch
          246 kB
          Deepak Jaiswal
        18. HIVE-15269.8.patch
          246 kB
          Deepak Jaiswal
        19. HIVE-15269.9.patch
          6.82 MB
          Deepak Jaiswal

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            djaiswal Deepak Jaiswal Assign to me
            jdere Jason Dere
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment