Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9188

BloomFilter support in ORC

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.15.0
    • Fix Version/s: 1.2.0
    • Component/s: File Formats
    • Labels:
    • Release Note:
      Support for Bloom Filters in ORC internal index.

      Description

      BloomFilters are well known probabilistic data structure for set membership checking. We can use bloom filters in ORC index for better row group pruning. Currently, ORC row group index uses min/max statistics to eliminate row groups (stripes as well) that do not satisfy predicate condition specified in the query. But in some cases, the efficiency of min/max based elimination is not optimal (unsorted columns with wide range of entries). Bloom filters can be an effective and efficient alternative for row group/split elimination for point queries or queries with IN clause.

        Attachments

        1. HIVE-9188.1.patch
          124 kB
          Prasanth Jayachandran
        2. HIVE-9188.10.patch
          315 kB
          Prasanth Jayachandran
        3. HIVE-9188.11.patch
          317 kB
          Prasanth Jayachandran
        4. HIVE-9188.2.patch
          135 kB
          Prasanth Jayachandran
        5. HIVE-9188.3.patch
          136 kB
          Prasanth Jayachandran
        6. HIVE-9188.4.patch
          192 kB
          Prasanth Jayachandran
        7. HIVE-9188.5.patch
          296 kB
          Prasanth Jayachandran
        8. HIVE-9188.6.patch
          302 kB
          Prasanth Jayachandran
        9. HIVE-9188.7.patch
          298 kB
          Prasanth Jayachandran
        10. HIVE-9188.8.patch
          298 kB
          Prasanth Jayachandran
        11. HIVE-9188.9.patch
          298 kB
          Prasanth Jayachandran

          Issue Links

            Activity

              People

              • Assignee:
                prasanth_j Prasanth Jayachandran
                Reporter:
                prasanth_j Prasanth Jayachandran
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: