Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Hi,
Help message from ./gobblin-dist/bin/gobblin-mapreduce.sh says
> --jars <comma-separated list of job jars> Job jar(s): *if not set, /home/michalw/gobblin-dist/lib is examined*
So I didn't set it and get in mappers:
```
2016-12-16 20:36:07,106 INFO [ForkExecutor-0] gobblin.runtime.Fork-0: Wrapping writer gobblin.writer.PartitionedDataWriter@12910662
2016-12-16 20:36:07,108 ERROR [ForkExecutor-0] gobblin.runtime.Fork-0: Fork 0 of task task_GobblinKafkaQuickStart_1481949335882_0 failed to process data records
*java.lang.NoClassDefFoundError: com/github/rholder/retry/RetryListener*
at gobblin.writer.DataWriterWrapperBuilder.build(DataWriterWrapperBuilder.java:49)
at gobblin.runtime.Fork.buildWriter(Fork.java:377)
at gobblin.runtime.Fork.buildWriterIfNotPresent(Fork.java:382)
at gobblin.runtime.Fork.processRecords(Fork.java:399)
at gobblin.runtime.Fork.run(Fork.java:170)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: *java.lang.ClassNotFoundException: com.github.rholder.retry.RetryListener*
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more
```
However if I do specify
--jars /home/michalw/gobblin-dist/lib/guava-retrying-2.0.0.jar
then it works. So to be clear: guava-retrying-2.0.0.jar is in lib, but is not included by default which is misleading regarding help text. If that is an intentional behaviour, I think it's worth to change help msg and add some notes in tutorials in docs. It would speed up a bit setting up.
Probably connected: #1321
The same issues described on groups.google forum couple of times.
I use 0.8.0, kafka->hdfs ingestion, MR
Github Url : https://github.com/linkedin/gobblin/issues/1466
Github Reporter : wosiu
Github Created At : 2016-12-17T05:42:03Z
Github Updated At : 2017-01-12T04:12:58Z