Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Not A Problem
-
None
-
None
-
None
-
None
Description
The current dynamic filter works by having an array of filters which grows as the filters reach 50% capacity (I believe). This functions, but creates an excessive amount of overhead relative to some other bloom filter techniques. We should look into alternative bloom filter implementations, probably some form of a compacting bloom filter. We should look into HBases' implementation, the hadoop 0.23 implementation, or possibly some other open source implementations which are already available.