Uploaded image for project: 'Crunch (Retired)'
  1. Crunch (Retired)
  2. CRUNCH-300

Support reflected Avro record writing from MemPipeline

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.9.0, 0.8.2
    • Core
    • None

    Description

      MemPipeline doesn't support writing Avro records via reflection. It seems that this was half implemented but never finished, but I needed it to create some test data to run through a cluster MapReduce test. The current implementation correctly reflects the schema, but then uses a GenericDatumWriter to try and write the record, causing a ClassCastException. The correct way would be to get a ReflectDatumWriter from the ReflectDataFactory.

      Attachments

        1. 0001-Allow-MemPipeline-to-write-Avro-files-by-reflection.patch
          2 kB
          David Whiting
        2. CRUNCH-300b.patch
          26 kB
          Gabriel Reid

        Activity

          People

            gabriel.reid Gabriel Reid
            davw David Whiting
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: