[PIG-3633] AvroStorage tests are failing when running against Avro 1.7.5 - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 0.12.0
Fix Version/s: 0.13.0
Component/s: None
Labels:
None

Description

AvroStorage tests executed against latest Avro release 1.7.5 are failing:

ant clean test -Dtestcase=TestAvroStorage -Davro.version=1.7.5
...
   [junit] Running org.apache.pig.builtin.TestAvroStorage
   [junit] Tests run: 33, Failures: 15, Errors: 0, Time elapsed: 14.211 sec

With following exception:

Failed to parse: Pig script failed to parse: 
<line 1, column 5> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -r]'
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
	at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
	at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
	at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
	at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
	at org.apache.pig.PigServer.registerScript(PigServer.java:649)
	at org.apache.pig.PigServer.registerScript(PigServer.java:726)
	at org.apache.pig.PigServer.registerScript(PigServer.java:699)
	at org.apache.pig.builtin.TestAvroStorage.testAvroStorage(TestAvroStorage.java:775)
	at org.apache.pig.builtin.TestAvroStorage.testLoadRecursiveRecordsOptionOn(TestAvroStorage.java:588)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
Caused by: 
<line 1, column 5> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -r]'
	at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:849)
	at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3479)
	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536)
	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
	... 34 more
Caused by: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -r]'
	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:748)
	at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:837)
	... 40 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:716)
	... 41 more
Caused by: java.lang.NullPointerException
	at org.apache.avro.Schema.parse(Schema.java:1072)
	at org.apache.avro.Schema$Parser.parse(Schema.java:950)
	at org.apache.avro.Schema$Parser.parse(Schema.java:940)
	at org.apache.pig.builtin.AvroStorage.<init>(AvroStorage.java:136)
	... 46 more
3701 [main] WARN  org.apache.pig.tools.parameters.PreprocessorContext  - Warning : Multiple values found for INFILE. Using value test/org/apache/pig/builtin/avro/data/trevni/uncompressed/simpleRecordsTrevni.trevni
3701 [main] WARN  org.apache.pig.tools.parameters.PreprocessorContext  - Warning : Multiple values found for AVROSTORAGE_OUT_2. Using value -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc
3702 [main] WARN  org.apache.pig.tools.parameters.PreprocessorContext  - Warning : Multiple values found for OUTFILE. Using value /home/jarcec/cloudera/repos/pig/build/test/TestAvroStorage/testLoadTrevniRecords
3732 [main] ERROR org.apache.pig.PigServer  - exception during parsing: Error during parsing. Pig script failed to parse: 
<line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
Failed to parse: Pig script failed to parse: 
<line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
	at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
	at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
	at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
	at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
	at org.apache.pig.PigServer.registerScript(PigServer.java:649)
	at org.apache.pig.PigServer.registerScript(PigServer.java:726)
	at org.apache.pig.PigServer.registerScript(PigServer.java:699)
	at org.apache.pig.builtin.TestAvroStorage.testAvroStorage(TestAvroStorage.java:775)
	at org.apache.pig.builtin.TestAvroStorage.testLoadTrevniRecords(TestAvroStorage.java:709)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
Caused by: 
<line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
	at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:936)
	at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7691)
	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1580)
	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
	... 34 more
Caused by: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:748)
	at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:915)
	... 40 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:716)
	... 41 more
Caused by: java.lang.NullPointerException
	at org.apache.avro.Schema.parse(Schema.java:1072)
	at org.apache.avro.Schema$Parser.parse(Schema.java:950)
	at org.apache.avro.Schema$Parser.parse(Schema.java:940)
	at org.apache.pig.builtin.AvroStorage.<init>(AvroStorage.java:136)
	... 46 more

tomwhite has looked into the problem and found out that Avro in version 1.7.5 has updated the JSON parser library Jackson from 1.8.8 to 1.9.13. The new version is changing the way empty strings are processed.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

PIG-3633.patch
18/Dec/13 09:33
0.5 kB
Jarek Jarcec Cecho

Activity

People

Assignee:: Jarek Jarcec Cecho

Reporter:: Jarek Jarcec Cecho

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 18/Dec/13 09:30

Updated:: 07/Jul/14 18:08

Resolved:: 18/Dec/13 18:21