Uploaded image for project: 'Avro'
  1. Avro
  2. AVRO-1067

Include OutputFormat for a specified Avro schema that works with Streaming

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.6.3
    • Fix Version/s: None
    • Component/s: java
    • Labels:
      None

      Description

      Avro currently includes AvroTextOutputFormat, which will only write an Avro file with a bytes schema. It would be nice if there were an OutputFormat that took a JSON string as a value, and a schema defined with -Davro.output.schema=... and wrote an Avro file based on that.

      This has been implemented with TypedBytes in a third-party repository already, but I'd imagine it would be trivial to make it work with text: https://github.com/tomslabs/avro-utils/blob/master/src/main/java/com/tomslabs/grid/avro/TextTypedBytesToAvroOutputFormat.java

      It would also be nice to just add the functionality into the existing AvroTextOutputFormat by just checking whether avro.output.schema is defined or not, but that could break backwards-compatibility.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              aripollak Ari Pollak
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: