[AVRO-1356] AvroMultipleOutputs map only jobs do not use NamedOutput schemas

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.4
• Fix Version/s: 1.7.5
• Component/s: java
• Assignee: Alan Paulsen
• Reporter: Alan Paulsen
• Created: 21/Jul/13 14:38
• Updated: 03/Apr/14 07:11
• Resolved: 07/Aug/13 19:47

Description

AvroMultipleOutputs sets the MapOutputKeySchema when running a map only job, as follows:

boolean isMaponly = job.getNumReduceTasks() == 0;
    if (keySchema != null) {
      if (isMaponly)
        AvroJob.setMapOutputKeySchema(job, keySchema);
      else
        AvroJob.setOutputKeySchema(job, keySchema);
    }
    if (valSchema != null) {
      if (isMaponly)
        AvroJob.setMapOutputValueSchema(job, valSchema);
      else
        AvroJob.setOutputValueSchema(job, valSchema);
    }

Unfortunately, AvroKeyOutputFormat and AvroKeyValueOutputFormat never check if the job is map only, and uses the OutputKeySchema and OutputValueSchema regardless.

We can fix this by either

• Changing AvroKeyOutputFormat and AvroKeyValueOutputFormat to check if the job is map only and use the appropriate schema. (Seems right)
• Change AvroMultipleOutputs to always use the OutputKeySchema and OutputValueSchema