Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
0.12.0
-
None
-
None
Description
AvroStorage tests executed against latest Avro release 1.7.5 are failing:
ant clean test -Dtestcase=TestAvroStorage -Davro.version=1.7.5 ... [junit] Running org.apache.pig.builtin.TestAvroStorage [junit] Tests run: 33, Failures: 15, Errors: 0, Time elapsed: 14.211 sec
With following exception:
Failed to parse: Pig script failed to parse: <line 1, column 5> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -r]' at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196) at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623) at org.apache.pig.PigServer.registerQuery(PigServer.java:575) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198) at org.apache.pig.PigServer.registerScript(PigServer.java:649) at org.apache.pig.PigServer.registerScript(PigServer.java:726) at org.apache.pig.PigServer.registerScript(PigServer.java:699) at org.apache.pig.builtin.TestAvroStorage.testAvroStorage(TestAvroStorage.java:775) at org.apache.pig.builtin.TestAvroStorage.testLoadRecursiveRecordsOptionOn(TestAvroStorage.java:588) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) Caused by: <line 1, column 5> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -r]' at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:849) at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3479) at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536) at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013) at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553) at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421) at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188) ... 34 more Caused by: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -r]' at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:748) at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:837) ... 40 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:716) ... 41 more Caused by: java.lang.NullPointerException at org.apache.avro.Schema.parse(Schema.java:1072) at org.apache.avro.Schema$Parser.parse(Schema.java:950) at org.apache.avro.Schema$Parser.parse(Schema.java:940) at org.apache.pig.builtin.AvroStorage.<init>(AvroStorage.java:136) ... 46 more 3701 [main] WARN org.apache.pig.tools.parameters.PreprocessorContext - Warning : Multiple values found for INFILE. Using value test/org/apache/pig/builtin/avro/data/trevni/uncompressed/simpleRecordsTrevni.trevni 3701 [main] WARN org.apache.pig.tools.parameters.PreprocessorContext - Warning : Multiple values found for AVROSTORAGE_OUT_2. Using value -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc 3702 [main] WARN org.apache.pig.tools.parameters.PreprocessorContext - Warning : Multiple values found for OUTFILE. Using value /home/jarcec/cloudera/repos/pig/build/test/TestAvroStorage/testLoadTrevniRecords 3732 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Pig script failed to parse: <line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]' Failed to parse: Pig script failed to parse: <line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]' at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196) at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623) at org.apache.pig.PigServer.registerQuery(PigServer.java:575) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198) at org.apache.pig.PigServer.registerScript(PigServer.java:649) at org.apache.pig.PigServer.registerScript(PigServer.java:726) at org.apache.pig.PigServer.registerScript(PigServer.java:699) at org.apache.pig.builtin.TestAvroStorage.testAvroStorage(TestAvroStorage.java:775) at org.apache.pig.builtin.TestAvroStorage.testLoadTrevniRecords(TestAvroStorage.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) Caused by: <line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]' at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:936) at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7691) at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1580) at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013) at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553) at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421) at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188) ... 34 more Caused by: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]' at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:748) at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:915) ... 40 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:716) ... 41 more Caused by: java.lang.NullPointerException at org.apache.avro.Schema.parse(Schema.java:1072) at org.apache.avro.Schema$Parser.parse(Schema.java:950) at org.apache.avro.Schema$Parser.parse(Schema.java:940) at org.apache.pig.builtin.AvroStorage.<init>(AvroStorage.java:136) ... 46 more
tomwhite has looked into the problem and found out that Avro in version 1.7.5 has updated the JSON parser library Jackson from 1.8.8 to 1.9.13. The new version is changing the way empty strings are processed.