Uploaded image for project: 'Crunch (Retired)'
  1. Crunch (Retired)
  2. CRUNCH-368

TupleWritable.Comparator

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.10.0, 0.8.3
    • 0.10.0, 0.8.3
    • Core
    • None

    Description

      This patch should improve comparison performance on TupleWritables. It saves the deserialization overhead. It is particularly useful when the input tuple are large, e.g. contains long strings.

      Please note that this changes the binary format of TupleWritable. It adds a var-int indicating size of field after each type code. This is a limitation of the writable system. We do not know the size of each field until fully desalinizing it.

      Attachments

        1. crunch-368.patch
          12 kB
          Chao Shi
        2. crunch-368 benchmark.pdf
          58 kB
          Chao Shi
        3. gen_data.py
          0.3 kB
          Chao Shi

        Activity

          People

            stepinto Chao Shi
            stepinto Chao Shi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: