I'm +1 on a commit (All tests pass for me). There is work to do stil integration – in particular mapping the HColumnDescriptor configurations to match new hfile for bloomfilters, compression, and blocksizing – but I'd suggest we do these as separate issues; the patch is big enough already.
Primitive performance eval. shows random reads up by about 60%, writes up about 25% but scans are down. Will do some profiling over next few days.
Other notes on the patch:
+ The change to hbase-site.xml is not yet hooked up.
+ This patch breaks binary keys because it undoes the ugly stuff we did to make them work. Will fix again when we address hbase-859 – thats next. In other words, this patch has already started the reworking of HStoreKey removing all the crap where every key had a HREgionInfo reference. One thing in particular that it adds is rawcomparator comparing store keys; that is, no object instantiation.. pure byte compare).
+ The patch is basically a rewrite from HStore down. A few files were renamed because they changed so much – HStore becomes Store, HStoreFile becomes StoreFile, etc.
+ Some pieces of this patch are taken from tfile, hadoop-3315. In particular the hfile tests and much of the compression facility: e.g. BoundedRangeFileInputStream, and Compression types.
+ A few files are missing apache license – we can add one when we commit (simple block cache).