[PIG-1247] Error Number makes it hard to debug: ERROR 2999: Unexpected internal error. org.apache.pig.backend.datastorage.DataStorageException cannot be cast to java.lang.Error

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.9.0
• Component/s: impl
• Assignee: Xuefu Zhang
• Reporter: Viraj Bhat
• Created: 20/Feb/10 02:33
• Updated: 04/Aug/11 00:35
• Resolved: 06/Apr/11 21:22

Description

I have a large script in which there are intermediate stores statements, one of them writes to a directory I do not have permission to write to.

The stack trace I get from Pig is this:

2010-02-20 02:16:32,055 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. org.apache.pig.backend.datastorage.DataStorageException cannot be cast to java.lang.Error

Details at logfile: /home/viraj/pig_1266632145355.log

Pig Stack Trace
---------------

ERROR 2999: Unexpected internal error. org.apache.pig.backend.datastorage.DataStorageException cannot be cast to java.lang.Error
java.lang.ClassCastException: org.apache.pig.backend.datastorage.DataStorageException cannot be cast to java.lang.Error
at org.apache.pig.impl.logicalLayer.parser.QueryParser.StoreClause(QueryParser.java:3583)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1407)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:949)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:762)
at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1036)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:986)
at org.apache.pig.PigServer.registerQuery(PigServer.java:386)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:720)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)
================================================================================

The only way to find the error was to look at the javacc generated QueryParser.java code and do a System.out.println()

Here is a script to reproduce the problem:

A = load '/user/viraj/three.txt' using PigStorage();
B = foreach A generate ['a'#'12'] as b:map[] ;
store B into '/user/secure/pigtest' using PigStorage();

"three.txt" has 3 lines which contain nothing but the number "1".

$ hadoop fs -ls /user/secure/

ls: could not get get listing for 'hdfs://mynamenode/user/secure' : org.apache.hadoop.security.AccessControlException: Permission denied: user=viraj, access=READ_EXECUTE, inode="secure":secure:users:rwx------

Viraj