[AVRO-679] Improved encodings for arrays

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Incomplete
• Component/s: spec
• Assignee: Unassigned
• Reporter: Stu Hood
• Created: 10/Oct/10 20:48
• Updated: 06/May/11 19:18
• Resolved: 22/Apr/11 18:20

Description

There are better ways to encode arrays of varints [1] which are faster to decode, and more space efficient than encoding varints independently.

Extending the idea to other types of variable length data like 'bytes' and 'string', you could encode the entries for an array block as an array of lengths, followed by contiguous byte/utf8 data.

[1] group varint encoding: slides 57-63 of http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/people/jeff/WSDM09-keynote.pdf