Accumulo
  1. Accumulo
  2. ACCUMULO-911

Get ClassNotFoundException when running Accumulo 1.4.0 Bulk Ingest Example

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Trying to follow the steps in the README.bulkIngest for Accumulo 1.4.0 but get a ClassNotFound Exception. Has anyone else seen this?

        Issue Links

          Activity

          Hide
          Pete Carlson added a comment -

          And to finish the example, I verified the 1000 rows are in accumulo by typing the following command line:

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.VerifyIngest cybersa zookeeper1 root **** test_bulk 0 1000
          

          Here are the results along with this command line:

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.VerifyIngest cybersa zookeeper1 root **** test_bulk 0 1000
          OK
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          Show
          Pete Carlson added a comment - And to finish the example, I verified the 1000 rows are in accumulo by typing the following command line: [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.VerifyIngest cybersa zookeeper1 root **** test_bulk 0 1000 Here are the results along with this command line: [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.VerifyIngest cybersa zookeeper1 root **** test_bulk 0 1000 OK [root@ip-10-151-7-8 accumulo-1.4.0]#
          Hide
          Pete Carlson added a comment -

          Cool! That worked! Here's output:

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh lib/examples-simple-*[^cs].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.5-1301095, built on 03/15/2012 19:48 GMT
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:host.name=master
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_24
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/hadoop-0.20.2/bin/../conf:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/lib/tools.jar:/opt/hadoop-0.20.2/bin/..:/opt/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar:/opt/hadoop-0.20.2/bin/../lib/commons-cli-1.2.jar:/opt/hadoop-0.20.2/bin/../lib/commons-codec-1.3.jar:/opt/hadoop-0.20.2/bin/../lib/commons-el-1.0.jar:/opt/hadoop-0.20.2/bin/../lib/commons-httpclient-3.0.1.jar:/opt/hadoop-0.20.2/bin/../lib/commons-logging-1.0.4.jar:/opt/hadoop-0.20.2/bin/../lib/commons-logging-api-1.0.4.jar:/opt/hadoop-0.20.2/bin/../lib/commons-net-1.4.1.jar:/opt/hadoop-0.20.2/bin/../lib/core-3.1.1.jar:/opt/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0.10.jar:/opt/hadoop-0.20.2/bin/../lib/jasper-compiler-5.5.12.jar:/opt/hadoop-0.20.2/bin/../lib/jasper-runtime-5.5.12.jar:/opt/hadoop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/opt/hadoop-0.20.2/bin/../lib/jetty-6.1.14.jar:/opt/hadoop-0.20.2/bin/../lib/jetty-util-6.1.14.jar:/opt/hadoop-0.20.2/bin/../lib/junit-3.8.1.jar:/opt/hadoop-0.20.2/bin/../lib/kfs-0.2.2.jar:/opt/hadoop-0.20.2/bin/../lib/log4j-1.2.15.jar:/opt/hadoop-0.20.2/bin/../lib/mockito-all-1.8.0.jar:/opt/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/opt/hadoop-0.20.2/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/hadoop-0.20.2/bin/../lib/slf4j-api-1.4.3.jar:/opt/hadoop-0.20.2/bin/../lib/slf4j-log4j12-1.4.3.jar:/opt/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/opt/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/opt/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-api-2.1.jar:/opt/accumulo-1.4.0/lib/libthrift-0.6.1.jar:/opt/accumulo-1.4.0/lib/accumulo-core-1.4.0.jar:/opt/zookeeper-3.3.5/zookeeper-3.3.5.jar:/opt/accumulo-1.4.0/lib/cloudtrace-1.4.0.jar:/opt/accumulo-1.4.0/lib/commons-collections-3.2.jar:/opt/accumulo-1.4.0/lib/commons-configuration-1.5.jar:/opt/accumulo-1.4.0/lib/commons-io-1.4.jar:/opt/accumulo-1.4.0/lib/commons-jci-core-1.0.jar:/opt/accumulo-1.4.0/lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.4.0/lib/commons-lang-2.4.jar:/opt/accumulo-1.4.0/lib/commons-logging-1.0.4.jar:/opt/accumulo-1.4.0/lib/commons-logging-api-1.0.4.jar:
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/hadoop-0.20.2/bin/../lib/native/Linux-amd64-64
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-276.el6.x86_64
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:user.name=root
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:user.dir=/opt/accumulo-1.4.0
          12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=zookeeper1 sessionTimeout=30000 watcher=org.apache.accumulo.core.zookeeper.ZooSession$AccumuloWatcher@4839e5b5
          12/12/17 20:33:38 INFO zookeeper.ClientCnxn: Opening socket connection to server zookeeper1/10.151.3.138:2181
          12/12/17 20:33:38 INFO zookeeper.ClientCnxn: Socket connection established to zookeeper1/10.151.3.138:2181, initiating session
          12/12/17 20:33:38 INFO zookeeper.ClientCnxn: Session establishment complete on server zookeeper1/10.151.3.138:2181, sessionid = 0x13b9fb152989c2a, negotiated timeout = 30000
          12/12/17 20:33:40 INFO input.FileInputFormat: Total input paths to process : 1
          12/12/17 20:33:40 INFO mapred.JobClient: Running job: job_201212151250_0002
          12/12/17 20:33:41 INFO mapred.JobClient:  map 0% reduce 0%
          12/12/17 20:33:51 INFO mapred.JobClient:  map 100% reduce 0%
          12/12/17 20:34:03 INFO mapred.JobClient:  map 100% reduce 33%
          12/12/17 20:34:05 INFO mapred.JobClient:  map 100% reduce 55%
          12/12/17 20:34:11 INFO mapred.JobClient:  map 100% reduce 100%
          12/12/17 20:34:13 INFO mapred.JobClient: Job complete: job_201212151250_0002
          12/12/17 20:34:13 INFO mapred.JobClient: Counters: 17
          12/12/17 20:34:13 INFO mapred.JobClient:   Job Counters
          12/12/17 20:34:13 INFO mapred.JobClient:     Launched reduce tasks=3
          12/12/17 20:34:13 INFO mapred.JobClient:     Launched map tasks=1
          12/12/17 20:34:13 INFO mapred.JobClient:     Data-local map tasks=1
          12/12/17 20:34:13 INFO mapred.JobClient:   FileSystemCounters
          12/12/17 20:34:13 INFO mapred.JobClient:     FILE_BYTES_READ=30018
          12/12/17 20:34:13 INFO mapred.JobClient:     HDFS_BYTES_READ=28000
          12/12/17 20:34:13 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=60116
          12/12/17 20:34:13 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=5849
          12/12/17 20:34:13 INFO mapred.JobClient:   Map-Reduce Framework
          12/12/17 20:34:13 INFO mapred.JobClient:     Reduce input groups=1000
          12/12/17 20:34:13 INFO mapred.JobClient:     Combine output records=0
          12/12/17 20:34:13 INFO mapred.JobClient:     Map input records=1000
          12/12/17 20:34:13 INFO mapred.JobClient:     Reduce shuffle bytes=30018
          12/12/17 20:34:13 INFO mapred.JobClient:     Reduce output records=1000
          12/12/17 20:34:13 INFO mapred.JobClient:     Spilled Records=2000
          12/12/17 20:34:13 INFO mapred.JobClient:     Map output bytes=28000
          12/12/17 20:34:13 INFO mapred.JobClient:     Combine input records=0
          12/12/17 20:34:13 INFO mapred.JobClient:     Map output records=1000
          12/12/17 20:34:13 INFO mapred.JobClient:     Reduce input records=1000
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          Show
          Pete Carlson added a comment - Cool! That worked! Here's output: [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh lib/examples-simple-*[^cs].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.5-1301095, built on 03/15/2012 19:48 GMT 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:host.name=master 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_24 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/hadoop-0.20.2/bin/../conf:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/lib/tools.jar:/opt/hadoop-0.20.2/bin/..:/opt/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar:/opt/hadoop-0.20.2/bin/../lib/commons-cli-1.2.jar:/opt/hadoop-0.20.2/bin/../lib/commons-codec-1.3.jar:/opt/hadoop-0.20.2/bin/../lib/commons-el-1.0.jar:/opt/hadoop-0.20.2/bin/../lib/commons-httpclient-3.0.1.jar:/opt/hadoop-0.20.2/bin/../lib/commons-logging-1.0.4.jar:/opt/hadoop-0.20.2/bin/../lib/commons-logging-api-1.0.4.jar:/opt/hadoop-0.20.2/bin/../lib/commons-net-1.4.1.jar:/opt/hadoop-0.20.2/bin/../lib/core-3.1.1.jar:/opt/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0.10.jar:/opt/hadoop-0.20.2/bin/../lib/jasper-compiler-5.5.12.jar:/opt/hadoop-0.20.2/bin/../lib/jasper-runtime-5.5.12.jar:/opt/hadoop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/opt/hadoop-0.20.2/bin/../lib/jetty-6.1.14.jar:/opt/hadoop-0.20.2/bin/../lib/jetty-util-6.1.14.jar:/opt/hadoop-0.20.2/bin/../lib/junit-3.8.1.jar:/opt/hadoop-0.20.2/bin/../lib/kfs-0.2.2.jar:/opt/hadoop-0.20.2/bin/../lib/log4j-1.2.15.jar:/opt/hadoop-0.20.2/bin/../lib/mockito-all-1.8.0.jar:/opt/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/opt/hadoop-0.20.2/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/hadoop-0.20.2/bin/../lib/slf4j-api-1.4.3.jar:/opt/hadoop-0.20.2/bin/../lib/slf4j-log4j12-1.4.3.jar:/opt/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/opt/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/opt/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-api-2.1.jar:/opt/accumulo-1.4.0/lib/libthrift-0.6.1.jar:/opt/accumulo-1.4.0/lib/accumulo-core-1.4.0.jar:/opt/zookeeper-3.3.5/zookeeper-3.3.5.jar:/opt/accumulo-1.4.0/lib/cloudtrace-1.4.0.jar:/opt/accumulo-1.4.0/lib/commons-collections-3.2.jar:/opt/accumulo-1.4.0/lib/commons-configuration-1.5.jar:/opt/accumulo-1.4.0/lib/commons-io-1.4.jar:/opt/accumulo-1.4.0/lib/commons-jci-core-1.0.jar:/opt/accumulo-1.4.0/lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.4.0/lib/commons-lang-2.4.jar:/opt/accumulo-1.4.0/lib/commons-logging-1.0.4.jar:/opt/accumulo-1.4.0/lib/commons-logging-api-1.0.4.jar: 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/hadoop-0.20.2/bin/../lib/native/Linux-amd64-64 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-276.el6.x86_64 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:user.name=root 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:user.home=/root 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Client environment:user.dir=/opt/accumulo-1.4.0 12/12/17 20:33:38 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=zookeeper1 sessionTimeout=30000 watcher=org.apache.accumulo.core.zookeeper.ZooSession$AccumuloWatcher@4839e5b5 12/12/17 20:33:38 INFO zookeeper.ClientCnxn: Opening socket connection to server zookeeper1/10.151.3.138:2181 12/12/17 20:33:38 INFO zookeeper.ClientCnxn: Socket connection established to zookeeper1/10.151.3.138:2181, initiating session 12/12/17 20:33:38 INFO zookeeper.ClientCnxn: Session establishment complete on server zookeeper1/10.151.3.138:2181, sessionid = 0x13b9fb152989c2a, negotiated timeout = 30000 12/12/17 20:33:40 INFO input.FileInputFormat: Total input paths to process : 1 12/12/17 20:33:40 INFO mapred.JobClient: Running job: job_201212151250_0002 12/12/17 20:33:41 INFO mapred.JobClient: map 0% reduce 0% 12/12/17 20:33:51 INFO mapred.JobClient: map 100% reduce 0% 12/12/17 20:34:03 INFO mapred.JobClient: map 100% reduce 33% 12/12/17 20:34:05 INFO mapred.JobClient: map 100% reduce 55% 12/12/17 20:34:11 INFO mapred.JobClient: map 100% reduce 100% 12/12/17 20:34:13 INFO mapred.JobClient: Job complete: job_201212151250_0002 12/12/17 20:34:13 INFO mapred.JobClient: Counters: 17 12/12/17 20:34:13 INFO mapred.JobClient: Job Counters 12/12/17 20:34:13 INFO mapred.JobClient: Launched reduce tasks=3 12/12/17 20:34:13 INFO mapred.JobClient: Launched map tasks=1 12/12/17 20:34:13 INFO mapred.JobClient: Data-local map tasks=1 12/12/17 20:34:13 INFO mapred.JobClient: FileSystemCounters 12/12/17 20:34:13 INFO mapred.JobClient: FILE_BYTES_READ=30018 12/12/17 20:34:13 INFO mapred.JobClient: HDFS_BYTES_READ=28000 12/12/17 20:34:13 INFO mapred.JobClient: FILE_BYTES_WRITTEN=60116 12/12/17 20:34:13 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=5849 12/12/17 20:34:13 INFO mapred.JobClient: Map-Reduce Framework 12/12/17 20:34:13 INFO mapred.JobClient: Reduce input groups=1000 12/12/17 20:34:13 INFO mapred.JobClient: Combine output records=0 12/12/17 20:34:13 INFO mapred.JobClient: Map input records=1000 12/12/17 20:34:13 INFO mapred.JobClient: Reduce shuffle bytes=30018 12/12/17 20:34:13 INFO mapred.JobClient: Reduce output records=1000 12/12/17 20:34:13 INFO mapred.JobClient: Spilled Records=2000 12/12/17 20:34:13 INFO mapred.JobClient: Map output bytes=28000 12/12/17 20:34:13 INFO mapred.JobClient: Combine input records=0 12/12/17 20:34:13 INFO mapred.JobClient: Map output records=1000 12/12/17 20:34:13 INFO mapred.JobClient: Reduce input records=1000 [root@ip-10-151-7-8 accumulo-1.4.0]#
          Hide
          Billie Rinaldi added a comment -

          Try

          ./bin/tool.sh lib/examples-simple-*[^cs].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          

          That regex is designed to eliminate the sources and javadoc jars so that only examples-simple-1.4.0.jar is used. "-libjars jarname" is not necessary here. The issue with the regex being incorrect is addressed in ACCUMULO-889.

          Show
          Billie Rinaldi added a comment - Try ./bin/tool.sh lib/examples-simple-*[^cs].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork That regex is designed to eliminate the sources and javadoc jars so that only examples-simple-1.4.0.jar is used. "-libjars jarname" is not necessary here. The issue with the regex being incorrect is addressed in ACCUMULO-889 .
          Hide
          Pete Carlson added a comment -

          I tried what you said by comma separating the three jar files, but now I get an error in opening zip file:

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0.jar,lib/examples-simple-1.4.0-javadoc.jar,lib/examples-simple-1.4.0-sources.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          Exception in thread "main" java.io.IOException: Error opening job jar: lib/examples-simple-1.4.0.jar,lib/examples-simple-1.4.0-javadoc.jar,lib/examples-simple-1.4.0-sources.jar
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
          Caused by: java.util.zip.ZipException: error in opening zip file
                  at java.util.zip.ZipFile.open(Native Method)
                  at java.util.zip.ZipFile.<init>(ZipFile.java:131)
                  at java.util.jar.JarFile.<init>(JarFile.java:150)
                  at java.util.jar.JarFile.<init>(JarFile.java:87)
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          

          I also tried an inidividual jar file like you mentioned, with wierd results (each individual shown below):

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork                                        java.io.FileNotFoundException: File cybersa does not exist.
                  at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:338)
                  at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:259)
                  at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:375)
                  at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
                  at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:138)
                  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
                  at org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample.main(BulkIngestExample.java:163)
                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                  at java.lang.reflect.Method.invoke(Method.java:616)
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
          
          ERROR: Wrong number of parameters: 6 instead of 7.
          accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample <instanceName> <zooKeepers> <username> <password> <table> <input dir> <work dir>
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0-javadoc.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          Exception in thread "main" java.lang.ClassNotFoundException: org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample
                  at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
                  at java.lang.Class.forName0(Native Method)
                  at java.lang.Class.forName(Class.java:264)
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0-sources.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          Exception in thread "main" java.lang.ClassNotFoundException: org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample
                  at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
                  at java.lang.Class.forName0(Native Method)
                  at java.lang.Class.forName(Class.java:264)
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          Show
          Pete Carlson added a comment - I tried what you said by comma separating the three jar files, but now I get an error in opening zip file: [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0.jar,lib/examples-simple-1.4.0-javadoc.jar,lib/examples-simple-1.4.0-sources.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork Exception in thread "main" java.io.IOException: Error opening job jar: lib/examples-simple-1.4.0.jar,lib/examples-simple-1.4.0-javadoc.jar,lib/examples-simple-1.4.0-sources.jar at org.apache.hadoop.util.RunJar.main(RunJar.java:90) Caused by: java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:131) at java.util.jar.JarFile.<init>(JarFile.java:150) at java.util.jar.JarFile.<init>(JarFile.java:87) at org.apache.hadoop.util.RunJar.main(RunJar.java:88) [root@ip-10-151-7-8 accumulo-1.4.0]# I also tried an inidividual jar file like you mentioned, with wierd results (each individual shown below): [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork java.io.FileNotFoundException: File cybersa does not exist. at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:338) at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:259) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:375) at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:138) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59) at org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample.main(BulkIngestExample.java:163) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) ERROR: Wrong number of parameters: 6 instead of 7. accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample <instanceName> <zooKeepers> <username> <password> <table> <input dir> <work dir> [root@ip-10-151-7-8 accumulo-1.4.0]# [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0-javadoc.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork Exception in thread "main" java.lang.ClassNotFoundException: org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.util.RunJar.main(RunJar.java:149) [root@ip-10-151-7-8 accumulo-1.4.0]# [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars lib/examples-simple-1.4.0-sources.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork Exception in thread "main" java.lang.ClassNotFoundException: org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.util.RunJar.main(RunJar.java:149) [root@ip-10-151-7-8 accumulo-1.4.0]#
          Hide
          William Slacum added a comment -

          The tool.sh let's you run something similar to `hadoop job`. The first command you issued said the class/Tool you wanted to execute was "/lib/examples-simple-1.4.0.jar", which is why you got a different error when you passed it as a libjar parameter. Can you try explicitly giving the name of the jar instead of the pattern? So something like:

          `./bin/tool.sh -libjars /lib/examples-simple-1.4.0.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork`

          It looks like your shell isn't expanding the pattern on the jar name.

          Show
          William Slacum added a comment - The tool.sh let's you run something similar to `hadoop job`. The first command you issued said the class/Tool you wanted to execute was "/lib/examples-simple-1.4.0.jar", which is why you got a different error when you passed it as a libjar parameter. Can you try explicitly giving the name of the jar instead of the pattern? So something like: `./bin/tool.sh -libjars /lib/examples-simple-1.4.0.jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork` It looks like your shell isn't expanding the pattern on the jar name.
          Hide
          Pete Carlson added a comment -

          I also tried adding the -libjars switch which gives me an error about opening the job jar:

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh
          Usage: tool.sh path/to/myTool.jar my.tool.class.Name [-libjars my1.jar,my2.jar]
          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars /lib/examples-simple-*[^c].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          Exception in thread "main" java.io.IOException: Error opening job jar: /lib/examples-simple-*[^c].jar
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
          Caused by: java.util.zip.ZipException: error in opening zip file
                  at java.util.zip.ZipFile.open(Native Method)
                  at java.util.zip.ZipFile.<init>(ZipFile.java:131)
                  at java.util.jar.JarFile.<init>(JarFile.java:150)
                  at java.util.jar.JarFile.<init>(JarFile.java:87)
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          Show
          Pete Carlson added a comment - I also tried adding the -libjars switch which gives me an error about opening the job jar: [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh Usage: tool.sh path/to/myTool.jar my.tool.class.Name [-libjars my1.jar,my2.jar] [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh -libjars /lib/examples-simple-*[^c].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork Exception in thread "main" java.io.IOException: Error opening job jar: /lib/examples-simple-*[^c].jar at org.apache.hadoop.util.RunJar.main(RunJar.java:90) Caused by: java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:131) at java.util.jar.JarFile.<init>(JarFile.java:150) at java.util.jar.JarFile.<init>(JarFile.java:87) at org.apache.hadoop.util.RunJar.main(RunJar.java:88) [root@ip-10-151-7-8 accumulo-1.4.0]#
          Hide
          Pete Carlson added a comment -

          But then when I try to run the command to actually ingest the 1000 rows of data into accumulo I get a ClassNotFoundException (again I obscured my password as **** ):

          [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh lib/examples-simple-*[^c].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork
          Exception in thread "main" java.lang.ClassNotFoundException: lib.examples-simple-1.4.0-sources.jar
                  at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
                  at java.lang.Class.forName0(Native Method)
                  at java.lang.Class.forName(Class.java:264)
                  at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          

          Yet if I look in the /opt/accumulo-1.4.0/lib directory I see the examples-simple-1.4.0-sources.jar

          [root@ip-10-151-7-8 accumulo-1.4.0]# cd lib
          [root@ip-10-151-7-8 lib]# ll
          total 19332
          -rw-r-xr-x. 1 1000 1000 2901405 Mar 26  2012 accumulo-core-1.4.0.jar
          -rw-r-xr-x. 1 1000 1000 6479410 Mar 26  2012 accumulo-core-1.4.0-javadoc.jar
          -rw-r-xr-x. 1 1000 1000 1018249 Mar 26  2012 accumulo-core-1.4.0-sources.jar
          -rw-r-xr-x. 1 1000 1000 1822292 Mar 26  2012 accumulo-server-1.4.0.jar
          -rw-r-xr-x. 1 1000 1000 2586973 Mar 26  2012 accumulo-server-1.4.0-javadoc.jar
          -rw-r-xr-x. 1 1000 1000 1027817 Mar 26  2012 accumulo-server-1.4.0-sources.jar
          -rw-r-xr-x. 1 1000 1000   24416 Mar 26  2012 accumulo-start-1.4.0.jar
          -rw-r-xr-x. 1 1000 1000   54229 Mar 26  2012 accumulo-start-1.4.0-javadoc.jar
          -rw-r-xr-x. 1 1000 1000   15583 Mar 26  2012 accumulo-start-1.4.0-sources.jar
          -rw-r-xr-x. 1 1000 1000   97979 Mar 26  2012 cloudtrace-1.4.0.jar
          -rw-r-xr-x. 1 1000 1000  315869 Mar 26  2012 cloudtrace-1.4.0-javadoc.jar
          -rw-r-xr-x. 1 1000 1000   45517 Mar 26  2012 cloudtrace-1.4.0-sources.jar
          -rw-r-xr-x. 1 1000 1000  571259 Mar 26  2012 commons-collections-3.2.jar
          -rw-r-xr-x. 1 1000 1000  271849 Mar 26  2012 commons-configuration-1.5.jar
          -rw-r-xr-x. 1 1000 1000  109043 Mar 26  2012 commons-io-1.4.jar
          -rw-r-xr-x. 1 1000 1000   32094 Mar 26  2012 commons-jci-core-1.0.jar
          -rw-r-xr-x. 1 1000 1000   20262 Mar 26  2012 commons-jci-fam-1.0.jar
          -rw-r-xr-x. 1 1000 1000  261809 Mar 26  2012 commons-lang-2.4.jar
          -rw-r-xr-x. 1 1000 1000   38015 Mar 26  2012 commons-logging-1.0.4.jar
          -rw-r-xr-x. 1 1000 1000   26202 Mar 26  2012 commons-logging-api-1.0.4.jar
          -rw-r-xr-x. 1 1000 1000  136779 Mar 26  2012 examples-simple-1.4.0.jar
          -rw-r-xr-x. 1 1000 1000  337289 Mar 26  2012 examples-simple-1.4.0-javadoc.jar
          -rw-r-xr-x. 1 1000 1000   85774 Mar 26  2012 examples-simple-1.4.0-sources.jar
          drw-r-xr-x. 2 1000 1000    4096 Mar 26  2012 ext
          -rw-r-xr-x. 1 1000 1000   87325 Mar 26  2012 jline-0.9.94.jar
          -rw-r-xr-x. 1 1000 1000  295922 Mar 26  2012 libthrift-0.6.1.jar
          -rw-r-xr-x. 1 1000 1000  481535 Mar 26  2012 log4j-1.2.16.jar
          drw-r-xr-x. 4 1000 1000    4096 Mar 26  2012 native
          -rw-r-xr-x. 1 1000 1000  244243 Mar 26  2012 wikisearch-ingest-1.4.0-javadoc.jar
          -rw-r-xr-x. 1 1000 1000  351677 Mar 26  2012 wikisearch-query-1.4.0-javadoc.jar
          [root@ip-10-151-7-8 lib]#
          

          Originally the permissions were 644 which I thought was causing the issue. But even when I changed the permissions to 655 as shown above there is no change. Has anyone else seen this problem?

          Show
          Pete Carlson added a comment - But then when I try to run the command to actually ingest the 1000 rows of data into accumulo I get a ClassNotFoundException (again I obscured my password as **** ): [root@ip-10-151-7-8 accumulo-1.4.0]# ./bin/tool.sh lib/examples-simple-*[^c].jar org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample cybersa zookeeper1 root **** test_bulk bulk tmp/bulkWork Exception in thread "main" java.lang.ClassNotFoundException: lib.examples-simple-1.4.0-sources.jar at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.util.RunJar.main(RunJar.java:149) [root@ip-10-151-7-8 accumulo-1.4.0]# Yet if I look in the /opt/accumulo-1.4.0/lib directory I see the examples-simple-1.4.0-sources.jar [root@ip-10-151-7-8 accumulo-1.4.0]# cd lib [root@ip-10-151-7-8 lib]# ll total 19332 -rw-r-xr-x. 1 1000 1000 2901405 Mar 26 2012 accumulo-core-1.4.0.jar -rw-r-xr-x. 1 1000 1000 6479410 Mar 26 2012 accumulo-core-1.4.0-javadoc.jar -rw-r-xr-x. 1 1000 1000 1018249 Mar 26 2012 accumulo-core-1.4.0-sources.jar -rw-r-xr-x. 1 1000 1000 1822292 Mar 26 2012 accumulo-server-1.4.0.jar -rw-r-xr-x. 1 1000 1000 2586973 Mar 26 2012 accumulo-server-1.4.0-javadoc.jar -rw-r-xr-x. 1 1000 1000 1027817 Mar 26 2012 accumulo-server-1.4.0-sources.jar -rw-r-xr-x. 1 1000 1000 24416 Mar 26 2012 accumulo-start-1.4.0.jar -rw-r-xr-x. 1 1000 1000 54229 Mar 26 2012 accumulo-start-1.4.0-javadoc.jar -rw-r-xr-x. 1 1000 1000 15583 Mar 26 2012 accumulo-start-1.4.0-sources.jar -rw-r-xr-x. 1 1000 1000 97979 Mar 26 2012 cloudtrace-1.4.0.jar -rw-r-xr-x. 1 1000 1000 315869 Mar 26 2012 cloudtrace-1.4.0-javadoc.jar -rw-r-xr-x. 1 1000 1000 45517 Mar 26 2012 cloudtrace-1.4.0-sources.jar -rw-r-xr-x. 1 1000 1000 571259 Mar 26 2012 commons-collections-3.2.jar -rw-r-xr-x. 1 1000 1000 271849 Mar 26 2012 commons-configuration-1.5.jar -rw-r-xr-x. 1 1000 1000 109043 Mar 26 2012 commons-io-1.4.jar -rw-r-xr-x. 1 1000 1000 32094 Mar 26 2012 commons-jci-core-1.0.jar -rw-r-xr-x. 1 1000 1000 20262 Mar 26 2012 commons-jci-fam-1.0.jar -rw-r-xr-x. 1 1000 1000 261809 Mar 26 2012 commons-lang-2.4.jar -rw-r-xr-x. 1 1000 1000 38015 Mar 26 2012 commons-logging-1.0.4.jar -rw-r-xr-x. 1 1000 1000 26202 Mar 26 2012 commons-logging-api-1.0.4.jar -rw-r-xr-x. 1 1000 1000 136779 Mar 26 2012 examples-simple-1.4.0.jar -rw-r-xr-x. 1 1000 1000 337289 Mar 26 2012 examples-simple-1.4.0-javadoc.jar -rw-r-xr-x. 1 1000 1000 85774 Mar 26 2012 examples-simple-1.4.0-sources.jar drw-r-xr-x. 2 1000 1000 4096 Mar 26 2012 ext -rw-r-xr-x. 1 1000 1000 87325 Mar 26 2012 jline-0.9.94.jar -rw-r-xr-x. 1 1000 1000 295922 Mar 26 2012 libthrift-0.6.1.jar -rw-r-xr-x. 1 1000 1000 481535 Mar 26 2012 log4j-1.2.16.jar drw-r-xr-x. 4 1000 1000 4096 Mar 26 2012 native -rw-r-xr-x. 1 1000 1000 244243 Mar 26 2012 wikisearch-ingest-1.4.0-javadoc.jar -rw-r-xr-x. 1 1000 1000 351677 Mar 26 2012 wikisearch-query-1.4.0-javadoc.jar [root@ip-10-151-7-8 lib]# Originally the permissions were 644 which I thought was causing the issue. But even when I changed the permissions to 655 as shown above there is no change. Has anyone else seen this problem?
          Hide
          Pete Carlson added a comment -

          We have Accumulo 1.4.0 installed on our Amazon RHEL 6.3 64-bit EC2 instance, and I'm having problems running the Bulk Ingest example.

          Our Accumulo 1.4.0 install is under /opt/accumulo-1.4.0

          I found the README.bulkIngest under /opt/accumulo-1.4.0/docs/examples and was trying to follow this tutorial.

          The first two commands to set up the table test_bulk and 1000 rows of data ran without any errors. Note, in my case the instance is 'cybersa', zookeeper is 'zookeeper1', username is root, and password is obscured as ****.

          [root@ip-10-151-7-8 accumulo-1.4.0]# pwd
          /opt/accumulo-1.4.0
          [root@ip-10-151-7-8 accumulo-1.4.0]#./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.SetupTable cybersa zookeeper1 root **** test_bulk row_00000333 row_00000666
          [root@ip-10-151-7-8 accumulo-1.4.0]#./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.GenerateTestData 0 1000 bulk/test_1.txt
          [root@ip-10-151-7-8 accumulo-1.4.0]#
          
          Show
          Pete Carlson added a comment - We have Accumulo 1.4.0 installed on our Amazon RHEL 6.3 64-bit EC2 instance, and I'm having problems running the Bulk Ingest example. Our Accumulo 1.4.0 install is under /opt/accumulo-1.4.0 I found the README.bulkIngest under /opt/accumulo-1.4.0/docs/examples and was trying to follow this tutorial. The first two commands to set up the table test_bulk and 1000 rows of data ran without any errors. Note, in my case the instance is 'cybersa', zookeeper is 'zookeeper1', username is root, and password is obscured as ****. [root@ip-10-151-7-8 accumulo-1.4.0]# pwd /opt/accumulo-1.4.0 [root@ip-10-151-7-8 accumulo-1.4.0]#./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.SetupTable cybersa zookeeper1 root **** test_bulk row_00000333 row_00000666 [root@ip-10-151-7-8 accumulo-1.4.0]#./bin/accumulo org.apache.accumulo.examples.simple.mapreduce.bulk.GenerateTestData 0 1000 bulk/test_1.txt [root@ip-10-151-7-8 accumulo-1.4.0]#

            People

            • Assignee:
              Unassigned
              Reporter:
              Pete Carlson
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development