Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.5
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Hadoop Flags:
      Reviewed

      Description

      Add bloomfiltering to hfile. Can be enabled on a family-level basis. Ability to configure a row vs row+col level bloom. We size the bloomfilter with the number of entries we are about to flush which seems like usually we'd be making a filter too big, so our implementation needs to take that into account.

      1. 1200-trunk.txt
        137 kB
        stack
      2. Bloom_Filters_in_HBase.pdf
        91 kB
        stack
      3. Bloom Filters in HBase.docx
        106 kB
        Nicolas Spiegelberg
      4. HBASE-1200-0.20.5.patch
        129 kB
        Nicolas Spiegelberg
      5. HBASE-1200-0.20.5-4.patch
        140 kB
        Nicolas Spiegelberg
      6. hbase-1200-0.20branch-v2.txt
        120 kB
        stack
      7. ryan_bloomfilter.patch
        20 kB
        stack
      8. Store.java.rej
        2 kB
        stack
      9. StoreScanner.java.rej
        2 kB
        stack

        Issue Links

          Activity

          stack created issue -
          stack made changes -
          Field Original Value New Value
          Link This issue blocks HBASE-61 [ HBASE-61 ]
          ryan rawson made changes -
          Assignee stack [ stack ] ryan rawson [ ryanobjc ]
          Jonathan Gray made changes -
          Fix Version/s 0.21.0 [ 12313607 ]
          Fix Version/s 0.20.0 [ 12313474 ]
          stack made changes -
          Summary Add bloomfilters to hfile; use dynamicbloomfilter instead of base bloomfilter; depend on hadoop 0.20 Add bloomfilters; use dynamicbloomfilter instead of base bloomfilter
          stack made changes -
          Attachment ryan_bloomfilter.patch [ 12409564 ]
          Nicolas Spiegelberg made changes -
          Assignee ryan rawson [ ryanobjc ] Nicolas Spiegelberg [ nspiegelberg ]
          Nicolas Spiegelberg made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Nicolas Spiegelberg made changes -
          Summary Add bloomfilters; use dynamicbloomfilter instead of base bloomfilter Add bloomfilters
          Description Add bloomfiltering to hfile. Should it be optional or on always? Currently, we bloom filter rows only, not the column + ts component, which seems good place to start but we size the bloomfilter with the number of entries we are about to flush which seems like usually we'd be making a filter too big. How to figure how many rows in the flush? We should use the DynamicBloomFilter as Andrezj does up in hadoop BloomFilterMapFile. Start small and let it resize as entries are added. Add bloomfiltering to hfile. Can be enabled on a family-level basis. Ability to configure a row vs row+col level bloom. We size the bloomfilter with the number of entries we are about to flush which seems like usually we'd be making a filter too big, so our implementation needs to take that into account.
          Nicolas Spiegelberg made changes -
          Link This issue relates to HBASE-2501 [ HBASE-2501 ]
          Nicolas Spiegelberg made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Affects Version/s 0.20.5 [ 12314800 ]
          Fix Version/s 0.20.5 [ 12314800 ]
          Fix Version/s 0.21.0 [ 12313607 ]
          Nicolas Spiegelberg made changes -
          Attachment HBASE-1200-0.20.5.patch [ 12443991 ]
          Nicolas Spiegelberg made changes -
          Attachment Bloom Filters in HBase.docx [ 12443992 ]
          stack made changes -
          Attachment Bloom_Filters_in_HBase.pdf [ 12444007 ]
          stack made changes -
          Attachment hbase-1200-0.20branch-v2.txt [ 12444011 ]
          Attachment Store.java.rej [ 12444012 ]
          Attachment StoreScanner.java.rej [ 12444013 ]
          Nicolas Spiegelberg made changes -
          Attachment HBASE-1200-0.20.5-4.patch [ 12444221 ]
          stack made changes -
          Fix Version/s 0.21.0 [ 12313607 ]
          Fix Version/s 0.20.5 [ 12314800 ]
          Labels moved_from_0_20_5
          stack made changes -
          Attachment 1200-trunk.txt [ 12444963 ]
          stack made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Nicolas Spiegelberg made changes -
          Link This issue is related to HBASE-2842 [ HBASE-2842 ]
          Lars Francke made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Nicolas Spiegelberg
              Reporter:
              stack
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development