Google has released it's Snappy compression codec.
Avro should support this for data file compression.
add checksums to Snappy codec
There is also an acknowledged issue ticket for adding Java support. Link for reference: https://code.google.com/p/snappy/issues/detail?id=15
Here's a patch that implements this.
Here's a new version of the patch that adds support for Snappy to Avro's MapReduce API and also updates the spec, adding Snappy as an optional codec for now.
Any reason not to commit this soon?
+1 Straightforward, looks good.
I wonder about the maven scope. Should this be 'provided' scope due to its use of JNI? See http://code.google.com/p/snappy-java/ towards the bottom. A maven user can override the current scope as needed or exclude the jar, so this isn't critical and can be changed easily later.
I've thus far avoided learning too much about Maven, so don't have an opinion.
Hold off on my +1. I have one other concern. What JNI library is embedded here? Linux? Windows? Do users have to compile their own?
I'm going to check this more thoroughly on my mac.
+1 OK, tests pass. They embed compiled libraries for Mac, Windows, and Linux. So this feature is available only for those platforms at this time.
I committed this.