[AVRO-983] maven-avro-plugin: Allow maven schema compiler to support external reference in an another avro schema file

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.6.1
• Component/s: java
• Hadoop Flags: Reviewed
• Assignee: Thomas White
• Reporter: Cédric Torcq
• Created: 19/Dec/11 17:32
• Updated: 08/Nov/18 15:03
• Resolved: 13/Nov/12 09:57

Description

Avro Maven plugin doesn't support an external reference in an another Avro schema file even this schema is available in the same src directory

Example :

 
[
	{
		"namespace": "com.afklm.karma.demand.inter",
		"name": "DatedFlightLegInterType",
		"type": "record",
		"fields":
			[
				{ "name": "legOrgArp", "type": "string" },
				{ "name": "legDestArp", "type": "string" },
				{ "name": "legDepDate", "type": "long" },
				{ "name": "flightNb", "type": "string" },
				{ "name": "dfl", "type": "com.afklm.karma.demand.output.DatedFlightLegType" }
			]
	},
	{
		"namespace": "com.afklm.karma.demand.inter",
		"name": "AirportAirportDatedFlightLegInterType",
		"type": "record",
		"fields":
			[
				{ "name": "aa", "type": "AirportAirportInterType" },
				{ "name": "legOrgArp", "type": "string" },
				{ "name": "legDestArp", "type": "string" },
				{ "name": "legDepDate", "type": "long" },
				{ "name": "flightNb", "type": "string" }
			]
	}
]

generate a fatal error :

 
org.apache.avro.SchemaParseException: "com.afklm.karma.demand.output.DatedFlightLegType" is not a defined name. The type of t
he "dfl" field must be a defined name or a {"type": ...} expression.
        at org.apache.avro.Schema.parse(Schema.java:1131)
        at org.apache.avro.Schema.parse(Schema.java:1200)
        at org.apache.avro.Schema$Parser.parse(Schema.java:968)
        at org.apache.avro.Schema$Parser.parse(Schema.java:945)
        at org.apache.avro.mojo.SchemaMojo.doCompile(SchemaMojo.java:58)
        at org.apache.avro.mojo.AbstractAvroMojo.compileFiles(AbstractAvroMojo.java:137)
        at org.apache.avro.mojo.AbstractAvroMojo.execute(AbstractAvroMojo.java:107)

I propose a patch to support also this case :
In org.apache.avro.mojo.SchemaMojo (line 55):

// File src = new File(sourceDirectory, filename);
// Schema.Parser parser = new Schema.Parser();
// Schema schema = parser.parse(src);
// SpecificCompiler compiler = new SpecificCompiler(schema);
// compiler.setTemplateDir(templateDirectory);
// compiler.setStringType(StringType.valueOf(stringType));
// compiler.compileSchema(src,
// outputDirectory);

DirectoryScanner scanner = new DirectoryScanner();
scanner.setBasedir(sourceDirectory);
scanner.setIncludes( getIncludes() );
scanner.addDefaultExcludes();
scanner.scan();
File[] avroFiles = new File[scanner.getIncludedFiles().length];
for (int i = 0; i < avroFiles.length; i++)

{ avroFiles[i] = new File( scanner.getBasedir(), scanner.getIncludedFiles()[i] ); }

SpecificCompiler.compileSchema(avroFiles, outputDirectory);

Issue Links

• is related to: AVRO-1188 External Schema Imports via AVSC Schema
• relates to: AVRO-1188 External Schema Imports via AVSC Schema