Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
Docs Required, Release Notes Required
Description
Motivation.
For now, TupleAssembler writes offsets for varlen columns as 2-byte {{short }}type.
This implicitly restricts key/value sizes down to 64 kB in total.
On another side, for small rows that columns can be addressed with byte type, we will waste few bytes.
Description.
Let's
- allow 4 byte types (byte, short, int) for offsets.
- implement and benchmark different approaches that allow us to write rows in the most compact way.
- then choose and merge the best one.
We can introduce several formats for writing Vartable (using byte/short/int offsets).
Additional information about Vartable format can be coded into chunk flags.
The first approach is to precalculate chunk total size, then choose the most compact format and write a chunk.
The second approach is to write a chunk with the widest format then convert the chunk into the most compact format in place.
Attachments
Attachments
Issue Links
- causes
-
-
- Open
-
- is a child of
-
-
- Open
-
- links to
