Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-6418

Binary: optionally write integer datatypes with varint encoding

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1
    • Fix Version/s: None
    • Component/s: binary
    • Labels:

      Description

      Currently all integer data types are written as is. Integer always takes 4 bytes, Long - 8 bytes, etc.

      There is well-known technique called "varint encoding" which can compress integer values [1]. When used, Integer can take 1-5 bytes, Long - 1-10 bytes. So when values are small enough we can save a lot of space.

      But this technique is not unversal, as big encoded values might require more bytes comparing to plain form. Also it might cause slowdowns in SQL engine. So this approach cannot be applied globally. Instead, we should allow users to control whether they want to use this technique or not.

      One possible approach is to add some annotation and several new methods to BinaryWriter and BinaryReader, which will control whether varint is used or not.

      [1] https://developers.google.com/protocol-buffers/docs/encoding#varints

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                vozerov Vladimir Ozerov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: