Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-1239

AvroMultipleOutput ignores schemas

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 1.7.3
    • Fix Version/s: None
    • Component/s: java
    • Labels:

      Description

      Adding namedOutputs with AvroMultipleOutputs.addNamedOutput just adds them to a static map which is of course not available on the cluster during reduce execution.

      The unit tests pass though since the Instance of AvroMultipleOutputs is the same in the Reducer as in the Job's main class, so the added schemas there are present.

      Fix would be to add the namedOutput schemas to the job configuration so they can be parsed in the reducers. Example patch for the new mapreduce api is attached, but I suspect the problem is present in the mapred api also. What is the general approach for this? Fix both?

      Cheers,

      Johannes

        Attachments

        1. 1239.patch
          3 kB
          Johannes Schulte

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                baunz Johannes Schulte
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: