[PIG-226] Problem with streaming optimization

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.1.0
• Assignee: Arun Murthy
• Reporter: Olga Natkovich
• Created: 01/May/08 20:53
• Updated: 24/Mar/10 22:01
• Resolved: 02/May/08 04:07

Description

Current optimization code assumes that every storeage function implements both LoadFunc and StoreFunc interfaces and causes an exception if it does not.

2008-05-01 13:31:23,662 [main] ERROR org.apache.pig.tools.grunt.Grunt - java.lang.RuntimeException: could not instantiate 'org.apache.pig.test.udf.storefunc.DumpLoader' with arguments '[]'
at org.apache.pig.impl.PigContext.instantiateFunc(PigContext.java:515)
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:521)
at org.apache.pig.impl.logicalLayer.optimizer.streaming.StoreOptimizer.visitStore(StoreOptimizer.java:103)
at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:121)
at org.apache.pig.impl.logicalLayer.optimizer.streaming.StoreOptimizer.optimize(StoreOptimizer.java:145)
at org.apache.pig.PigServer.optimizeAndRunQuery(PigServer.java:411)
at org.apache.pig.PigServer.registerQuery(PigServer.java:297)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:450)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:233)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:62)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:60)
at org.apache.pig.Main.main(Main.java:296)

The code that needs to change is in impl/logicalLayer/optimizer/streaming/LoadOptimizer.java and impl/logicalLayer/optimizer/streaming/StoreOptimizer.java.

The logic that needs to be added is if the load/store interface is not implemented then disable optimization.