Description
A checksum might be included with each compressed block to better detect errors. While some filesystems (e.g. HDFS) may checksum data, not all do. Data files may also accumulate errors when copied between filesystems. For back-compatibility, we cannot easily add checksums to existing data files, but a new codec provides us with the opportunity to do this.
Attachments
Attachments
Issue Links
- relates to
-
AVRO-788 add Snappy compression
- Closed