Google has released it's Snappy compression codec.
Avro should support this for data file compression.
add checksums to Snappy codec
I committed this.
+1 OK, tests pass. They embed compiled libraries for Mac, Windows, and Linux. So this feature is available only for those platforms at this time.
Hold off on my +1. I have one other concern. What JNI library is embedded here? Linux? Windows? Do users have to compile their own?
I'm going to check this more thoroughly on my mac.
I've thus far avoided learning too much about Maven, so don't have an opinion.
+1 Straightforward, looks good.
I wonder about the maven scope. Should this be 'provided' scope due to its use of JNI? See http://code.google.com/p/snappy-java/ towards the bottom. A maven user can override the current scope as needed or exclude the jar, so this isn't critical and can be changed easily later.
Any reason not to commit this soon?
Here's a new version of the patch that adds support for Snappy to Avro's MapReduce API and also updates the spec, adding Snappy as an optional codec for now.
Here's a patch that implements this.
There is also an acknowledged issue ticket for adding Java support. Link for reference: https://code.google.com/p/snappy/issues/detail?id=15