[MAHOUT-391] Make vector more space efficient with variable-length encoding, et al

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.3
• Fix Version/s: 0.4
• Labels: double encoding float length optimization variable vector zag zig
• Assignee: Sean R. Owen
• Reporter: Sean R. Owen
• Due: 14/May/10
• Created: 05/May/10 11:34
• Updated: 31/Oct/10 15:49
• Resolved: 11/May/10 19:23

Description

There are a few things we can do to make Vector representations smaller on disk:

• Use variable-length encoding for integer values like size and element indices in sparse representations
• Further, delta-encode indices in sequential representations
• Let caller specify that precision isn't crucial in values, allowing it to store values as floats

Since indices are usually small-ish, I'd guess this saves 2 bytes or so on average, out of 12 bytes per element now.
Using floats where applicable saves another 4. Not bad.