Uploaded image for project: 'Crunch (Retired)'
  1. Crunch (Retired)
  2. CRUNCH-268

Crunch's internal Avro tuple schemas should have stable names

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.7.0
    • 0.8.0
    • Core, IO
    • None

    Description

      A long time ago, I made a change that used random names for the custom Avro schemas that Crunch generates for processing tuple types (pairs, trips, etc.). I recently hit a use case where that randomization burned me when I was re-running some pipelines over checkpointed data that I serialized using Crunch's Avro schemas (Pair, in particular), so I think that we should change the tuple schemas to have stable names based on their constituent field schemas via an MD5 hash.

      Attachments

        1. CRUNCH-268.patch
          3 kB
          Josh Wills
        2. CRUNCH-268v2.patch
          4 kB
          Josh Wills

        Activity

          People

            jwills Josh Wills
            jwills Josh Wills
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: