[AVRO-1414] Compression with C++ DataFile

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.6
• Component/s: c++
• Assignee: Daniel Russel
• Reporter: Daniel Russel
• Created: 16/Dec/13 22:07
• Updated: 23/Jan/14 21:58
• Resolved: 07/Jan/14 21:54

Description

There is no way to use compression with the C++ DataFileReader and C++ DataFileWriter, from what I can tell. Adding compression of the written blocks using boost streams is relatively straight forward and I can provide a patch if people are interested.

However, there are a couple caveats:

• the windows builds of boost don't currently include zlib support (required for compression) by default. You have to do extra work to get it.
• I don't know if doing it that way is compatible with other avro implementations