[PIG-2527] ILLUSTRATE fails for relations LOADed with the AvroStorage UDF

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.9.2, 0.10.0, 0.11, 0.10.1
• Fix Version/s: 0.10.1
• Component/s: piggybank
• Labels: avro avro_udf avrostorage happy pig storage udf

Release Note

The fix for this wasn't committed for 0.10.0, and I'm not sure where the fix is/if it was committed at all.

Can anyone remember what happened with the fixes for AvroStorage that didn't go in 0.10.0?

• Assignee: Jonathan Coveney
• Reporter: Russell Jurney
• Created: 11/Feb/12 02:16
• Updated: 18/Mar/13 22:51
• Resolved: 19/Dec/12 16:33

Description

grunt> describe emails
emails: {message_id: chararray,from:

{PIG_WRAPPER: (ARRAY_ELEM: chararray)}

,to:

{PIG_WRAPPER: (ARRAY_ELEM: chararray)}

,cc:

{PIG_WRAPPER: (ARRAY_ELEM: chararray)}

,bcc:

{PIG_WRAPPER: (ARRAY_ELEM: chararray)}

,reply_to:

{PIG_WRAPPER: (ARRAY_ELEM: chararray)}

,in_reply_to:

{PIG_WRAPPER: (ARRAY_ELEM: chararray)}

,subject: chararray,body: chararray,date: chararray}

grunt> illustrate emails
2012-02-10 18:15:01,591 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
2012-02-10 18:15:01,592 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2012-02-10 18:15:01,649 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2012-02-10 18:15:01,649 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2012-02-10 18:15:01,649 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2012-02-10 18:15:01,649 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2012-02-10 18:15:01,668 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 5
2012-02-10 18:15:02,719 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2012-02-10 18:15:02,719 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2012-02-10 18:15:02,719 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2012-02-10 18:15:02,720 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2012-02-10 18:15:02,720 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2012-02-10 18:15:02,733 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2012-02-10 18:15:02,734 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2012-02-10 18:15:02,734 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2012-02-10 18:15:02,734 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2012-02-10 18:15:02,734 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2012-02-10 18:15:02,749 [main] ERROR org.apache.pig.pen.AugmentBaseDataVisitor - No (valid) input data found!
java.lang.RuntimeException: No (valid) input data found!
at org.apache.pig.pen.AugmentBaseDataVisitor.visit(AugmentBaseDataVisitor.java:579)
at org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:82)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:66)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:180)
at org.apache.pig.PigServer.getExamples(PigServer.java:1245)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:698)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:591)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:306)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
2012-02-10 18:15:02,752 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encountered IOException. Exception : No (valid) input data found!

================================================================================
Pig Stack Trace
---------------
ERROR 2997: Encountered IOException. Exception : No (valid) input data found!

java.io.IOException: Exception : No (valid) input data found!
at org.apache.pig.PigServer.getExamples(PigServer.java:1251)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:698)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:591)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:306)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
================================================================================