Pig
  1. Pig
  2. PIG-3881

When loading Accumulo table, I see AccumuloSecurityException ClassNotFoundException

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.13.0
    • Fix Version/s: None
    • Component/s: grunt
    • Labels:
      None

      Description

      I've added the Accumulo jar files to my pig class like this:

      export PIG_CLASSPATH=/opt/accumulo/lib/accumulo-core-1.4.2.jar
      export PIG_CLASSPATH=$PIG_CLASSPATH:/opt/accumulo/lib/libthrift-0.6.1.jar
      export PIG_CLASSPATH=$PIG_CLASSPATH:/opt/accumulo/lib/cloudtrace-1.4.2.jar
      export PIG_CLASSPATH=$PIG_CLASSPATH:/opt/zookeeper/zookeeper-3.3.3.jar

      And then I start pig (using the fat jar). When I run

      A = LOAD 'accumulo://TABLE?instance=default&user=root&password=XXXX&zookeepers=localhost:2181' USING org.apache.pig.backend.hadoop.accumulo.AccumuloStorage();

      I ran into the following exception:

      ava.lang.NoClassDefFoundError: org/apache/accumulo/core/client/AccumuloSecurityException
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:264)
      at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:659)

        Activity

        David Medinets created issue -
        Hide
        Daniel Dai added a comment -

        Are you using Accumulo 1.4.2? My understanding is AccumuloStorage only works with Accumulo 1.5+, Josh Elser, is that right?

        Show
        Daniel Dai added a comment - Are you using Accumulo 1.4.2? My understanding is AccumuloStorage only works with Accumulo 1.5+, Josh Elser , is that right?
        Hide
        Josh Elser added a comment -

        Are you using Accumulo 1.4.2? My understanding is AccumuloStorage only works with Accumulo 1.5+

        I definitely tested with 1.5+, at the very least YMMV with 1.4 – things may happen to work but I wouldn't necessarily trust it. However, from the looks of things, that may be unrelated to the above error as the AccumuloSecurityException class exists across both versions. David Medinets, can you re-run things with the verbose flag on your pig invocation so we can get a full picture? The only other idea I have is that the paths you added might not actually exist (typo, perhaps).

        Show
        Josh Elser added a comment - Are you using Accumulo 1.4.2? My understanding is AccumuloStorage only works with Accumulo 1.5+ I definitely tested with 1.5+, at the very least YMMV with 1.4 – things may happen to work but I wouldn't necessarily trust it. However, from the looks of things, that may be unrelated to the above error as the AccumuloSecurityException class exists across both versions. David Medinets , can you re-run things with the verbose flag on your pig invocation so we can get a full picture? The only other idea I have is that the paths you added might not actually exist (typo, perhaps).
        Hide
        David Medinets added a comment -

        I am running with v1.4.2 with no option to upgrade. I ran pig with the -v option which produced this exception:

        2014-04-11 08:56:18,010 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org/apache/accumulo/core/client/AccumuloSecurityException
        2014-04-11 08:56:18,010 [main] ERROR org.apache.pig.tools.grunt.Grunt - java.lang.NoClassDefFoundError: org/apache/accumulo/core/client/AccumuloSecurityException
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:659)
        at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1296)
        at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1284)
        at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5158)
        at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1675)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1648)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:625)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1075)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:228)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
        at org.apache.pig.Main.run(Main.java:542)
        at org.apache.pig.Main.main(Main.java:156)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
        Caused by: java.lang.ClassNotFoundException: org.apache.accumulo.core.client.AccumuloSecurityException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        ... 27 more

        The code at http://people.apache.org/~elserj/accumulo-pig/ was working with pig. Well, at least I could read row values. But if 1.4.x can't be supported I'll resolve this issue and reluctantly move to another approach.

        Show
        David Medinets added a comment - I am running with v1.4.2 with no option to upgrade. I ran pig with the -v option which produced this exception: 2014-04-11 08:56:18,010 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org/apache/accumulo/core/client/AccumuloSecurityException 2014-04-11 08:56:18,010 [main] ERROR org.apache.pig.tools.grunt.Grunt - java.lang.NoClassDefFoundError: org/apache/accumulo/core/client/AccumuloSecurityException at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:659) at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1296) at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1284) at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5158) at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515) at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625) at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102) at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560) at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421) at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188) at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1675) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1648) at org.apache.pig.PigServer.registerQuery(PigServer.java:625) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1075) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:228) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66) at org.apache.pig.Main.run(Main.java:542) at org.apache.pig.Main.main(Main.java:156) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by: java.lang.ClassNotFoundException: org.apache.accumulo.core.client.AccumuloSecurityException at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 27 more The code at http://people.apache.org/~elserj/accumulo-pig/ was working with pig. Well, at least I could read row values. But if 1.4.x can't be supported I'll resolve this issue and reluctantly move to another approach.
        Hide
        Josh Elser added a comment -

        Can you post your script too? I can try to poke at this – knowing exactly what your script was doing would help.

        Show
        Josh Elser added a comment - Can you post your script too? I can try to poke at this – knowing exactly what your script was doing would help.
        Hide
        David Medinets added a comment -

        The pig script is just the load statement.

        Show
        David Medinets added a comment - The pig script is just the load statement.
        Hide
        Josh Elser added a comment -

        Oh, right - I should've remembered this earlier. Accumulo 1.5 deprecated using an explicit username and password authentication scheme in favor of using Token classes. As such, this isn't going to work with <1.5.0. Funny that you get the classnotfound exception on AccumuloSecurityException and not AuthenticationToken (as that's where it broke for me).

        IIRC, the old code you referenced above that I had linked on my people page was still based on the original accumulo-contrib module. I updated the dependency since Accumulo 1.4 is going away very quickly so the AccumuloStorage in Pig now should work with Accumulo 1.5 and 1.6. Sorry you're stuck on 1.4.2.

        Show
        Josh Elser added a comment - Oh, right - I should've remembered this earlier. Accumulo 1.5 deprecated using an explicit username and password authentication scheme in favor of using Token classes. As such, this isn't going to work with <1.5.0. Funny that you get the classnotfound exception on AccumuloSecurityException and not AuthenticationToken (as that's where it broke for me). IIRC, the old code you referenced above that I had linked on my people page was still based on the original accumulo-contrib module. I updated the dependency since Accumulo 1.4 is going away very quickly so the AccumuloStorage in Pig now should work with Accumulo 1.5 and 1.6. Sorry you're stuck on 1.4.2.
        Hide
        David Medinets added a comment -

        Accumulo releases prior to 1.5 are not supported.

        Show
        David Medinets added a comment - Accumulo releases prior to 1.5 are not supported.
        David Medinets made changes -
        Field Original Value New Value
        Resolution Won't Fix [ 2 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        21h 58m 1 David Medinets 11/Apr/14 18:45

          People

          • Assignee:
            Unassigned
            Reporter:
            David Medinets
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development