Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-329

Re-add type info to TupleWritable to make fields sort correctly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.8.3
    • Fix Version/s: 0.10.0, 0.8.3
    • Component/s: Core
    • Labels:
      None

      Description

      Secondary sorts aren't currently working correctly for Writable types after we hacked the TupleWritable impl to make all of the fields BytesWritables (e.g., secondary IntWritable values will no longer be sorted correctly, even though everything is still grouped correctly.)

      The least-bad way that I came up with to fix this is to use integer codes for each possible WritableComparable type in a pipeline that we can use to decode what Writable type each tuple field corresponds to. This allows us to keep the various fields sortable while still doing a reasonable job of minimizing the serialization required to pass the type information along.

        Attachments

        1. CRUNCH-329b.patch
          29 kB
          Josh Wills
        2. CRUNCH-329.patch
          26 kB
          Josh Wills
        3. fix-ss-writables.patch
          25 kB
          Josh Wills

          Activity

            People

            • Assignee:
              jwills Josh Wills
              Reporter:
              jwills Josh Wills
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: