-
Type:
New Feature
-
Status: Open
-
Priority:
Minor
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
Add a high speed compression algorithm like BMDiff.
It gives speeds ~100MB/s for writes and ~1000MB/s for reads, compressing 2.1billions web pages from 45.1TB in 4.2TB
Reference:
http://norfolk.cs.washington.edu/htbin-post/unrestricted/colloq/details.cgi?id=437
2005 Jeff Dean talk about google architecture - around 46:00.
http://feedblog.org/2008/10/12/google-bigtable-compression-zippy-and-bmdiff/
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=755678
A reference implementation exists in HyperTable.
- is depended upon by
-
HBASE-2655 2-pass compression support
-
- Resolved
-